How much should you spend on an external PenTest?

External Penetration Tests today come in all shapes and sizes, from the rudimentary highly automated scanning to the more detailed and human-driven PenTests, with often widely ranging costs to boot (3 to 4 times difference is not unusual for essentially the same thing).

Sometimes it’s difficult to work out what form of external PenTest is needed and hence what should be spent annually on such an independent audit of security controls, in relation to your in-house capabilities and risks the business faces. Remember, an external PenTest is just a ‘moment in time’ test of security, it says nothing about the security of the product long after it is done; that’s up to you. This article will help in such decision making by introducing a ‘PenTest SeeSaw’ I have created to assist with the analysis process and provide a framework to perform decision making with.

The PenTest Seesaw

The way the Pentest Seesaw works is that there are factors on the left, which if present in your cybersecurity stance, help reduce the need for such an in-depth or expensive external PenTest; whilst those factors on the right, if present, increase the need for an in-depth (and possibly expensive) external PenTest.

Think of the seesaw as a way to weigh up the internal and external factors that should be considered when looking at external PenTest services and what degree of coverage is really needed to be sure security controls and hence the risk coverage is at the right level. Each of these factors are explained below.

Security Certifications

This is a measure of how many security certifications the business has and what the employees have. For instance, if you have ISO27001:2013 and have people with a mixture of CISSP certifications or equivalent throughout the organisation – consider yourself quite strong in how security is enacted and that you have the right people in your business. Whereas a lack of certification indicates that operational security risks could be higher as nobody has been trained on how to approach security and there is likely a lack of policy and procedures in place.

Cyber Aware Management

Simply put, this is a measure of how much management takes cybersecurity seriously within the organisation. Do you have an ISMS in place? Do the board or senior management regularly discuss security concerns and act on them?

Staff Cyber Skills

Do the ‘boots on the ground’ know what they are doing with respect to security? Is there a day to day ‘security culture’ in place? Do people feel empowered to raise security concerns and enact security in their work? Do people get access to security training and support materials?

Internal PenTesting

Do you perform in-house PenTests and are the results of those tests acted upon and dealt with in a timely fashion? Does the release process include security tests?

Cyber Infrastructure Features

In the business infrastructure have you actually turned on or included cybersecurity controls? This indicates utilization of what is at hand and taking a ‘defence in depth‘ mindset to protect the business assets.

Inhouse Developer Skills

Do developers have specific security training? Do they know what OWASP is and how to apply it? Do they find and fix security issues on their own? Are they capable to take PenTests analysis results and applying the findings effectively & quickly?

Bad Actor Interest

How much ‘interest’ are hackers taking in your services? Are you under constant attack? Do you know if you are under attack or not? Do you get lots of phishing attempts or have you had any malware issues?

Competition Security Stance

What approach to security is the competition taking? Do they have certifications and the right staff with the right qualifications? Is security seen as a differentiator by the customers?

Customer Data Value

Is there a lot of sensitive customer data in the business systems? For instance lots of PII, SPI or HPI? Do your customers place a high value on the data they hold with you? Is that data something of interest to hackers?

Business Data Value

What about the value of the data about the business & its assets. For instance employee and financial records, sensitive product and service information. M&A activity, business contracts, etc? Would these be something of interest to a hacker?

Business Sector Risk Factors

Does the business operate in a sector that is known to encounter a lot of security risks? For instance: government, military & financial sectors. Do you have a lot of specific security requirements to implement?

How to use the Seesaw

For each of the above factors rate from 0 to 2, where 0 is Low value or confidence, and 2 is High value or confidence. If a factor doesn’t apply you, skip it; e.g. you do not any development or have no customer data in your systems (not a SaaS provider for instance). Then sum up each side and see if the seesaw balances or tips one way or the other.

If the seesaw tips to the left, it’s likely you can shop around and spend less on external PenTests as you have a good handle on managing security risks within the business and the processes, procedures and people to do so going forward in a fast cycle of ongoing improvements. In the diagram, the PenTest ‘ball’ rolls down the slope to the left.

Whereas, if the seesaw tips to the right and the PenTest ball goes to the right; you do not have the in-house ability to find and deal with security risks on the business products yourself. So thereby requiring more in-depth PenTests to fulfil your security obligations to the market and the sector the business operates in, and hence you should be spending more on your external PenTests (all things being equal).

If the seesaw balances out, then you don’t really have a driver to shop on value as such; but you still need reasonably comprehensive external PenTests.

BTW – If your total score on the left is less than 5 and the total score on the right is greater than 7; you need to do more than buy more covering PenTests – you have an operational security risk ‘gap’ that needs specific attention by investing in the LHS. In other words, the PenTest will highlight issues you are not operationally able to resolve in a timely and confident ongoing manner appropriate to the business risk exposure levels.

Also, I recommend shopping around and do not settle with the first Pentest provider you come across, perform a competitive tender and give each provider exactly the same information (i.e. systems under test, approximate size, technical footprint, number of roles, etc). Also, I suggest focussing only on PenTester’s who are CREST members (for Australia see here ), this sets an appropriate baseline. Now you may think this will limit your ability to make savings, in my experience, this is not the case at all. Also, remember what you save on PenTests could be applied to other security endeavours in the business.

Also, with recent local moves to combine a number of security firms into one ‘super firm’ – you need to be careful when shopping around to make sure that the Pen Test providers themselves are actually independent of each other. As concerns why the need to combine and join forces in such a way, I’m 50/50 on it, as on the one hand, it reduces true competition in the market place, so increasing the likelihood of costs to businesses going up for such services. On the other hand, it might make them better able to compete in a larger market space. Although I suspect places like the USA are not undergoing such a wave of collaboration, the usual model there is takeovers, so it might indicate we have reached saturation point.

I also suspect as pen test technology improves, and the quality of what is available via open-source channels improves, it will become a race to the bottom. You also need to keep in mind that a Pen Test says nothing about your security stance in other areas (such as back-office processes, development practices, social engineering opportunities, etc). Basically do not depend on a Pen Test alone to ensure your security.

Please feel free to use the Seesaw to help model or demonstrate why an increased investment on the LHS can have benefits in reducing external costs as well, whilst maintaining control over the risk exposure.

If you would like to know more, please get in touch, or you would like to know more about how to value your data as a hacker would see it, buy my book.

Please feel free to use the PenTest Seesaw, although do provide reference back to this, thanks!