Usually a check in the box means that you comply with some sort of regulation, but that doesn't mean that you are actually secure. As a matter of fact, almost all networks that contain credit card information and are successfully hacked are PCI compliant (a real example). That goes to show that compliance doesn't protect you from hackers, it only protects you from auditors and the fines that they can impose. Whats more is that those fines are only a small fraction of the cost of the damages that can be caused by a single successful hack.
When a computer system is hacked, the hacker doesn't stop at one computer. Standard hacker practice is to perform Distributed Metastasis and propagate the penetration throughout the rest of the network. This means that within a matter of minutes the hacker will likely have control over the most or all of the critical aspects of your IT infrastructure and will also have access to your sensitive data. At that point you've lost the battle... but you were compliant, you paid for the scan and now you've got a negative Return on that Investment ("ROI").
So what are the damages? Its actually impossible to determine the exact cost in damages that result from a single successful hack because its impossible to be certain of the full extent of the compromise. Never the less, here are some of the areas to consider when attempting to calculate damages:
- Man hours to identify every compromised device
- Man hours to reinstall and configure every device
- Man hours required to check source code for malicious alterations
- Man hours to monitor network traffic for hits of malicious traffic or access
- Man hours to educate customers
- Penalties and fines.
- The cost of downtime
- The cost of lost customers
- The cost of a damaged reputation
- etc.
Now lets consider the Return on Investment of *good* security. An Advanced Penetration Test against a small IT Infrastructure (~50 computers in total) might cost something around $16,000.00-$25,000 for an 80 hour project. If that service is delivered by a quality vendor then it will enable you to identify and eliminate your risks before they are exploited by a malicious hacker. The ROI of the quality service would be equal to the cost in damages of a single successful compromise minus the cost of the services. Not to mention you'd be complaint too...
(Note: the actual cost of services varies quite a bit depending on what needs to be done, etc.)
So why is it that some vendors will do this work for $500.00 or $2,000.00, etc? Its simple, they are not delivering the same quality service as the quality vendor. When you pay $500.00 for a vulnerability scan you are paying for something that you could do yourself for free (go download nessus). Never the less, when you pay $500.00 you are really only paying for about 5 minutes of manual labor, the rest of the work is automated and done by the tools. (If you broke that down to an hourly rate you'd be paying something like $6000.00 an hour since you're paying $500.00 per 5 minutes). In the end you might end up with a check in your compliance box but you'll still just as vulnerable as you were in the beginning.
Do you have any figures on the usual rates being charged for a "good" penetration testing service? And how the prices would vary depending on scope.. I've tried looking around but I can never find information on what the average prices are (unless you talk with the vendor)
ReplyDeleteYes we do have some figures on the usual rates for what we consider to be a good penetration test. Those figures were collected over the past two years from independent security consultants, businesses that meet our quality testing standards, and some of their respective customers. We're not free to share that research but the range does vary from $200/hr to $500/hr depending on service requirements and talent involved.
ReplyDeleteWhat if the poor quality guys charge the same as the high quality penetration testers, how does the customer tell the difference?
ReplyDeleteCan I use this blog post in my blog as I'm doing security research and penetration testing, so to say, "for living"?
ReplyDelete[copied from my pen-test[at]sf.com reply]
ReplyDeleteFirst, I wanted to say that I agree with the large majority of what you've said (written) since your posting activity increased, and especially appreciate the visibility you've brought to the issue of sub-par consultants and services - the fact is, they create yet another layer of crap we have to wade through on the road to building a case for justification of cost and _why_ our services are needed in the first place. Cheers.
Now, when I clicked on the link, I thought I might actually land on a post with hard numbers and tangible Pros and Cons, but sadly such information remains elusive. In response to your post, I have a few comments:
o As I was reading the opening paragraph, the first thing that came to mind was, "Duh!". Who exactly is your intended audience? Because you're preaching to the choir, brother.
o The "Check in the Box Mentality" is not only alive and well in corporate settings, it is the prevailing attitude. As we know, (good) security is an intangible benefit, a COST center, and it's virtually impossible to put a face on "how much we're saving you when you don't have a breach" - unless the client or prospect has suffered a breach in the past. I have primarily a corporate IT Security background, and speak from experience about management's "Check in the Box Mentality" (emphasis on *management*, because I don't know any techies with this mentality). The problem is - they're perfectly happy with it, and will continue to be until they have a breach, and some are hard-headed enough to still not demand sweeping change *after* a breach. /boggle
o The statement about credit card data breaches and the fact that most victims are PCI compliant is an interesting one. I have to admit that Hannaford (OTOH) is the only recent breach (<2 yrs) I know of where the victim claimed PCI compliance at the time of the incident.
o I hear your message loud and clear. It's similar to a message I've been preaching for at least a couple of years now: Practicing fundamentally sound security will not only greatly decrease the risk of a breach, but implicitly render compliance with any standard, regulation, or law.
o Although it may be impossible to determine the *exact* cost of a breach, I'm disappointed that none of us has come up with an approximation, which is entirely doable with a bit of data. Surely we could come up with a formula that could be customized with specific client data points to arrive at a ballpark figure? I'm not necessarily volunteering, but it's something to cogitate on until next time...
Cheers,
-jp
In response to Alex and anyone who wants to use my blog as an informational source:
ReplyDeleteYes, it is fine if you use the content from our blog so long as you cite us as the source and provide a clickable link to your reader.
In response to:
ReplyDelete"What if the poor quality guys charge the same as the high quality penetration testers, how does the customer tell the difference?"
You can download our our white paper (upper right side of this blog) and they will help you to find the right vendor. Some low quality vendors do charge high prices but they won't be able to answer difficult questions on the fly. Ask them questions like, how can you use directory traversal to execute commands on a web server? Or how can you use LFI to launch a back door. Those are basic questions... difficult ones would be focused more around R&D and vulnerability exploitation. Maybe I should write a blog post about this... ;]
While it is important to select the appropriate vendor for your security assessment services... The fact remains that cost and quality do not correlate. It's possible to provide a low cost service and still provide a valuable, accurate and skilled service to the client.
ReplyDeleteI do agree though that the ones providing simply an automated Nessus scan should take a hike.
I agree with your post (as stated earlier, you're preacin to the choir!), but wanted to address two issues. 1) I take exception to your use of the term ROI. 2) I wanted to address what sounds like exasperation over business accepting poor or zero security assurance work.
ReplyDelete1) Do you consider insurance to be an investment? There is no ROI on security spending unless you are a security provider in some form or other. It is a cost, and a way to avoid loss. If a business pays you $xxx a year for 10 years to run tests and they never experience an incident of significance, did you add value or were you a cost?
2) If all business was rational and you could guarantee that in the next year they would suffer an attack, that might sell your services quite easily.
But that's the big gamble, right? Not every year does an entity suffer an incident of some significance, and every year entities accept the risk of that gamble. Likewise a part of that gamble is to only do just enough to avoid negligence should that incident occur. Hence, taking the cheapest services that put a check-mark in the box.
On a personal level, it is no different from home security. Many, many people do not invest time or money into very effective security measures for their home, despite the fact that they can so easily be robbed. It is a gamble that we have to realize is present, and that business stakeholders make all the time.
I am not saying this situation is right, but it is reality. It all comes back down to human nature and decisions and risk acceptance.
LonerVamp:
ReplyDelete1-) You and I already discussed this question and we both agreed that there was no actual ROI as ROI's are defined. We both agreed that security was a method for avoiding significant loss thereby saving the money that could result from a loss. Is insurance an investment? My opinion is that it is a "kind" of investment.
2-) Any average business that has more than 10 employees will suffer a compromise of some sort within a one year period. That compromise can come in the form of a worm, new-hire, malware, internet based threat, etc. The problem isn't that the threat is unreal, it is that businesses aren't educated about the real threat. If they were then they'd be able to avoid damages by purchasing quality security services and products.
Do people care about quality? Compare GM and Toyota...
When someone tells me that they've never suffered a compromise, I ask "How do you know?". The fact of the matter is that they don't. Even more scary is that most good hacks go totally unnoticed and those hacks do cause damage. The problem is that you can't easily study a loss and determine that its the result of a hack. Its difficult enough to study a hack to determine the loss.
1) Oops, my bad, I didn't remember! I'll leave that be! :)
ReplyDelete2) Even if a company is educated, that doesn't mean they will be rational and address the issue. I mean, most people know that their home can be broken into and they have not implemented good enough security. So why doesn't every home with a family capable of affording a central security system have one?
Of note, I'm not tackling the comparison between "good" security and "bad" security in quality and how "good" security has more ROI. I'm talking whether companies want to take on that cost of security even when educated, especially when that cost does not prohibit the business functions directly.
My guess is that we will continue to disagree on a fundamental level. We certainly play on the same team, but we're just looking at it differently and accepting different levels of reality as "ok" while focusing on different areas. :) I think neither of us is right and both of us are right, which is part of the reason why a topic like this neither has nor ever will have agreement (hence why it continues to be a topic, even a possibly passionate one). Almost religious-like. :)