Vulnerability scanning can have a detrimental negative impact on the security posture of your IT infrastructure if used improperly. This negative impact is due to a perceptional issue that has been driven by the vendors who sell vulnerability scanning services or the vulnerability scanners themselves. The hard facts prove that vulnerability scanners can not protect your IT Infrastructure from malicious hackers. (My team penetrates "scanned" networks on a regular basis during customer engagements). That is not to say that vulnerability scanners are useless, but it is to say that people need to readjust their perception of what vulnerability scanning really is.
While there are various types of vulnerability scanners they suffer from the same disease that most security technologies suffer from. That disease is that they are reactive to hackers and will never be proactive. The fact is that vulnerability scanners can not detect vulnerabilities unless someone has first identified the vulnerability and created a signature for its detection. This process can take quite a while and is often not an ethical one. So here is how it works...
A hacker decides to perform research against a common technology like your firewall. That hacker might spend minutes, months or even years doing research just for the purpose of identifying an exploitable security vulnerability. Once that vulnerability is identified the hacker has an ethics based decision to make. Does he notify the vendor of his discovery and release a formal advisory or does he use his discovery to hack networks, steal information and profit.
If the hacker decides to notify the vendor and release an advisory then there is usually a wait period of 1-3 months before the vendor releases a patch. This lag time means that the vendor's customers will remain vulnerable for at least another 1-3 months, most probably longer. What's even more interesting is that this vulnerability may have been discovered previously by a different researcher that didn't notify the vendor. If that's the case then that probably means that the vulnerability has been in use as a tool to break into networks for a while. Who knows, it could have been discovered months or even years ago? That type of unpublished vulnerability is known as a 0day and is the favorite weapon of the malicious hacker.
At some point the vulnerability does become public knowledge. Its also at this point that the vendors who make the vulnerability scanning technology become aware of the new risk. When they do learn about the new risk they need to develop a signature, or script for their scanning technology so that it can detect the risk. That development process can take anywhere from a few days to a few weeks depending on the complexity risk. As a result, the customers that rely on vulnerability scanning are in the dark until the vendor can publish a working and tested signature... but the hackers don't need to wait at all. The hackers can use it almost immediately.
So in summary, there is a large risk window between the point of discovery of a vulnerability and the point at which a vulnerability scanner can detect the vulnerability. This risk and exposure window is usually never smaller than a few months, and can be as large as several years. During that time there is a very good chance that malicious hackers will be using your undiscovered risks to penetrate into your infrastructures. Whats worse is that you'll have no idea that you've been hacked because like vulnerability scanning technology, Intrusion Detection technology also can't identify threats if it doesn't know what to look for. Moreover most Intrusion Detection technologies aren't configured properly and as such don't work properly.
Unfortunately the story doesn't end there. Vulnerability scanners also suffer from significant issues with accuracy. In all cases where I've used (various) vulnerability scanners, the best results that I've ever achieved were about 30% accurate. This means that most of the vulnerabilities that were detected during my various scans weren't actually vulnerabilities but instead were false alarms, also called false positives. More frightening is the number of vulnerabilities that I discovered while performing Real Time Dynamic Testing (manual hacking) that were entirely missed by the vulnerability scanner. If you don't believe me then go download a free vulnerability scanner, test your network and verify the results yourself.
This inaccuracy is partially due to the architecture of the vulnerability scanners and the fact that no two networks are alike. Vulnerability scanners use static signatures or scripts that are only capable of checking a target for a vulnerability if their syntax is exactly accurate and if the target responds in a way that the scanner can understand. If however the target, lets say its a computer system, is configured in a custom way then it may not respond in a way that the scanner will understand (how many of you keep the default configuration?). This communication barrier is a large part of what causes false positives and false negatives.
An important note about false positives and false negatives. Some vendors claim that their vulnerability scanners have low rates of false positives. As with Intrusion Detection, if low false positive rates are true then its usually reasonable to say that the technology has high rates of false negatives. You can think of it as a sliding scale of 1 to 10 where 1 is 100% False Positives and 10 is 100% False Negatives. As you move up and down the scale you inevitably end up with more of one or the other, you can never eliminate them. With that said, its my opinion that more false positives are better than more false negatives.
If vulnerability scanners aren't the right way to protect yourself then what is? You should protect yourself by exposing your business to an accurate and controlled reproduction of the threat by using a quality security provider. It is important to remember that no single hacker, good or bad, has access to all of the 0-day's in the world. As such, it is entirely possible for a team of ethical hackers to accurately reproduce the threat that unethical hackers can create. Testing at that level enables you to identify weaknesses in your defenses that would not otherwise be detected by testing at lesser levels. What good would a penetration test or a vulnerability assessment do if the malicious hackers will test you harder?
One of the many advantages of using a team of talented hackers for security testing instead of relying on automated vulnerability scanners is that those hackers can and should perform research against unique technologies that they encounter during a security test. I practice what I preach by the way. When our team delivers an Advanced Penetration Test to a customer we always perform our own research against interesting targets. Those targets can be Web Applications, Web Services, or even custom daemons running on systems. In the end, if we find something new we'll write an exploit (proof of concept) for the customer and include that in the final deliverable.
In closing, I am not suggesting that network vulnerability scanners are bad because they do have their place and they do serve a purpose. They are particularly useful in the hands of a skilled security expert especially when performing reconnaissance against large networks. In that scenario the scanner enables the expert to save time and to rapidly collect intelligence about targets given that the engagement is non-stealth in nature. With that said, I wouldn't rely on scanners for anything more than just reconnaissance, at least not yet.
Note: (Thank you to minoo for pointing out a few mistakes in my previous revision of this entry. I hope that this entry is as clear as I intend it to be. There is no one team that is the best, but there are only a few good ones. If this isn't clear enough or if it needs more revision please comment.)
Wednesday, January 7, 2009
Monday, January 5, 2009
Finding The Quality Security Vendor (Penetration Testing, Vulnerability Assessments, Web Application Security, etc)
While I've written several detailed white-papers on the subject of identifying quality security vendors, I still feel compelled to write more about the subject. It is my opinion that choosing the right security vendor is critical to the health and safety of a business. Choosing the wrong vendor can leave you with a false sense of security that in the end might result in significant damages. Often times those damages can't be fully measured and appreciated, especially when they involve the tarnishing of a good name.
This problem of identifying quality isn't new but it does take on a new importance when it involves the safety of your trade secrets, source code, or otherwise critically sensitive information. When you trust a security provider to test your IT Infrastructure, your people, physical security, etc. you are relying on them to identify risks that malicious hackers might otherwise discover. If the provider does not test you at the same threat level as the malicious hackers then their service is almost useless.
If that doesn't compel you to want quality security services then go ahead and take the risk. I suppose the question really is, how much is your network (and its data) worth? If its worth more than $500,000.00 then its probably worth spending money on a quality security vendor to protect it right?
So how do you know which providers are quality and which ones are frauds?
The first rule of thumb is to watch out for the vendors that produce deliverables that are the product of vulnerability scanners. There are two reasons for this, the first being that you don't need to pay anyone to run an automated scan when you can do it yourself for much less, or for free. You can choose from a variety of free tools like nessus, or you go out and buy a license for a vulnerability scanner.
Don't be fooled though, vulnerability scanners do not produce accurate results. In fact most vulnerability scanners produce results that contain anywhere from 40-90% false positives with an unknown rate of false negatives. While these tools are useful for reconnaissance they should not be used as the primary method for security testing.
Watch out for the vendor that tells you that they will run a vulnerability scan against your network and then "vet" the results. Vetting doesn't mean that they are going to do additional discovery. Vetting only means that the vendor will check the results of the vulnerability scan and eliminate the false positives. The quality of the end product is then only as good as the accuracy of the vulnerability scanner. Would you bank on that?
When you are choosing the vendor make sure to ask them specific questions. Questions that I find helpful are realistic but based on theoretical architectures. For example you could ask a vendor the following question:
"Suppose you are confronted with an architecture that consisted of 10 desktops behind a single firewall. That firewall has properly configured IPS capabilities and there are no ports forwarded from the internet to any system behind that firewall. How would you [the vendor] penetrate into that network? Once you penetrate how would you perform Distributed Metastasis?" Email me for the answer if you don't know it already.
You can also ask the vendor how they would use a directory traversal vulnerability to penetrate into a network. This is a bit of a trick question but if they know what they are doing then they will be able to answer it properly. The short answer is that you need to inject code into the web-server's error log and then use the directory traversal vulnerability to render the code. (Again, if you need the complete answer email me and I'll get it to you.)
Another good rule is to only choose security vendors who also perform Vulnerability Research and Development ("R&D"). That is to say that the vendor must frequently perform security research against technology, identify vulnerabilities in that technology, create exploits for those vulnerabilities and must release formal security advisories. If they don't then chances are they don't know how to do it, but why is R&D important?
R&D enables the vendor to keep its penetration testing skills honed (so long as the research done by the penetration testers). Penetration Testers who do not perform this kind of research are literally Script Kids (sorry guys). Script Kids are people who download tools and use those tools to penetrate into networks. In almost all cases they don't have any understanding of how the tools work. If you think about it, thats like giving a loaded gun to a 3 year old.
You can also ask the vendor how they collect their threat intelligence. Threat intelligence is a critical aspect of delivering quality security services. If the vendor doesn't have current threat intelligence about the threat then how will they help you to defend yourself against the threat? While I won't tell you how my team collects this intel, I will tell you that its not from the news and most certainly not all public forum.
In closing, my recommendation to you is that you do your homework before you choose a vendor. Research the components required for delivering a quality service, then use your research to question the provider. As an example, if you were going to get a Web Application Penetration Test ask the vendor to define the term "Penetration Test". Ask the vendor what the difference is between a Penetration Test and a Vulnerability Assessment. Also ask them to explain RFI, LFI, XSS, SQL Injection, Blind SQL Injection, etc. Remember, you are going to spend money on security, might as well make it worth while. If you don't then you're just adding that money to the damages from the hack that you'll suffer in the end.
If you have any questions please feel free to leave me a comment or send me an email. You might also want to check out the white papers that I've linked at the upper right hand corner of this blog. Those papers go into more detail about how to choose a good security vendor and how to select the right service.
Sunday, January 4, 2009
Followup to my last Brian Chess - Fortify Software post.
Recently I published a post about Fortify Software's Brian Chess because of some outlandish claims that he made in an article about penetration testing being "Dead by 2009". The off-line and 0n-line comments that resulted from that post were mostly in favor of what I'd written and one of those comments really caught my eye. So here is a post dedicated to Rafal in response to his comment on my article about Brian Chess.
Comment By Rafal shown below, verbatim in pink:
"If I may call a sanity timeout here folks - while I don't agree with Brian's assertions necessarily - if you combine a few factors you could conceivably come to the conclusion that penetration testing will start to dwindle (just not as quickly as 2009)."
Its only conceivable for those who do not know what Penetration Testing is, and many self-proclaimed security guru's don't. So lets start with some (partial) definitions here:
Vulnerability Assessment:
(Assessment: the act of assessing; appraisal; evaluation.)
A Vulnerability Assessment is a service that evaluates a particular target, or set of targets for the purpose of identifying points of exposure that are open to assault. A Vulnerability Assessment does not attempt to compromise or penetrate into a target once a point of exposure is identified, it only aims at assessing the target for points of risk. Vulnerability Assessments by their very nature are prone to False Positives and False Negatives as the findings are never validated via Penetration or Exploitation.
Vulnerability Assessment Tools include:
Penetration Test:
(Penetration: the act or power of penetrating.)
A Penetration Test is a service that evaluates a particular target, or a set of targets for the purpose of identifying points of exposure that are open to assault. A Penetration Test differs from a Vulnerability Assessment in that it attempts to penetrate into the target by exploiting any discovered points of risk and exposure. A Penetration Test when done properly will result in an accurate deliverable that contains no false positives. This is possible because exploitation of a risk or point of exposure is either successful or not. Penetration Tests can include theoretical findings but they should not be reported on as positives.
Penetration Testing Tools include (I'd recommend these):
You can use a Vulnerability Assessment or a Penetration Test against any type of target not just technology based targets. At Netragard we perform physical penetration tests, wireless penetration tests, network penetration tests, social engineering based penetration tests, web application penetration tests, etc. Likewise we can deliver vulnerability assessments against the same set of targets if penetration testing is too aggressive.
(I get the feeling that both Rafal and Brian Chess think that Penetration Testing is a Web Application only service)
"Here's my logic - feel free to scrutinize. For the record I work for HP (the SPIDynamics acquisition) so you guys can feel free to rip on the fact that our marketing folks I'm sure make interesting claims as well... but I digress. Here are some things to consider:
Actually we've got quite a bit of interesting history with HP, but that's a different story. With respect to SPIDynamics and the Web Inspect tool, I'm sorry that HP ever acquired SPIDynamics. WebInspect was a reasonable tool for doing preliminary reconnaissance against Web Applications during non-covert services. Once HP acquired the technology its quality went down the tubes. Not only that but the process of acquiring a license from HP is excruciatingly painful at best. What ever happened to being able to purchase the product online? /end rant
"1. When you do penetration testing, what are you really testing? Are you testing the system or the intelligence and skill of the pen tester? This is a very tough question to answer."
Why is that a difficult question to answer? If you've built your penetration testing team properly then your team will be able to expose its targets to the same or greater threat level than that which they will likely face in the real world. The fact of the matter is the more secure the infrastructure the more challenging the test and yes, its impossible to know everything but its not impossible to do a great job.
"2. Pursuant to #1 above, and the business' (living in reality land here) need to do lowest-cost vendors... what value do you suppose that the 90%+ of companies that go lowest-cost (outsourced to India, China, Mexico, etc) are getting?"
Businesses do not "need to do the low-cost vendors", they choose to because they are making uneducated decisions in most cases. Mind you the lack of education on their part is not their fault, its the fault of the poor quality vendors. Poor quality vendors advertise their services as if they are the same quality as the high quality vendors thereby causing confusion. When a business compares the two services they don't see the difference and so they choose the less expensive one.
"3. With every point-and-click testing tool there is a double-edged sword... here's why 3a. Tools make you more efficient BUT"
I only partially agree. When the tool spits out over 2,000 false positives (like WebInspect did the last time we used it) with only 3 real positives its doing very little to increase the efficiency of a team. Other tools that produce less false positives and more accurate results are very useful for time savings but their results should not be used to create an end product. Automated tools are not dynamic by nature and as such can not identify the same risks as talented penetration testers.
"3b. Tools can make you less "hands-on" when it comes to writing low-level exploits or code..."
Tools are also the root cause of the the fraudulent security experts. I'm not saying that tools don't have their place because they certainly do. But they allow people to become lazy and as such breed "experts" that are for all intents and purposes no better than script kids (which might I add are very dangerous because they don't know what they are doing).
"4. Penetration testing is an after-the-fact requirement... which is too late. You have to use tools to augment and empower your developers to write better code at the grass-roots otherwise you're hosed."
You're partially wrong. The tools that you speak of are derived from the attacks that were created by Penetration Testers (aka: hackers). With respect to the world of Web Applications, do you think that a tool discovered the first SQL Injection vulnerabilitiy and created a method for exploitation? Ofcourse not! Tools will always be a few steps behind the capabilities of a real hacker, regardless of that hackers ethical bias. The fact of the matter is that as hackers, we perform research and identify new methods for penetration that were not previously discovered and your tools can not and will not ever be able to defend against that.
"So - to summarize, penetration testing isn't going to be "dead" in this year of 2009, but it may start to dwindle down some depending on how good the marketing machines of the tools vendors are. Brian's statement is a self-fulfilling prophecy... he is making a statement that he hopes will incite people to make that statement come true."
I disagree, and again, you are working for a vendor that makes these tools. Its in your best interest to suggest that some how Penetration Testing will be less of a requirement because of the tools that you create. The reality of it is that if people drink that kool aid they will become more vulnerable, not more secure.
When our military tests the armor of its M1A2 Abrams Tank they test it against the real threat. So why aren't we pushing our customers to do the same thing, it makes perfect sense? In our case the real threat is always going to be the malicious hacker, not the software vendor making pretty and easy to use tools. The tools do have a place but they will only ever identify the low hanging fruit. It takes a professional hacker/penetration tester to actually test an infrastructure properly. Lets see your tools perform Social Engineering or drop USB sticks in parking-lots.
Comment By Rafal shown below, verbatim in pink:
"If I may call a sanity timeout here folks - while I don't agree with Brian's assertions necessarily - if you combine a few factors you could conceivably come to the conclusion that penetration testing will start to dwindle (just not as quickly as 2009)."
Its only conceivable for those who do not know what Penetration Testing is, and many self-proclaimed security guru's don't. So lets start with some (partial) definitions here:
Vulnerability Assessment:
(Assessment: the act of assessing; appraisal; evaluation.)
A Vulnerability Assessment is a service that evaluates a particular target, or set of targets for the purpose of identifying points of exposure that are open to assault. A Vulnerability Assessment does not attempt to compromise or penetrate into a target once a point of exposure is identified, it only aims at assessing the target for points of risk. Vulnerability Assessments by their very nature are prone to False Positives and False Negatives as the findings are never validated via Penetration or Exploitation.
Vulnerability Assessment Tools include:
- WebInspect for Web Application Vulnerability Assessments
- Nessus for Network Vulnerability Scanning
- Fortify for Web Application Vulnerability Assessments
- Retina Network Vulnerability Scanning
- etc... you get the idea.
Penetration Test:
(Penetration: the act or power of penetrating.)
A Penetration Test is a service that evaluates a particular target, or a set of targets for the purpose of identifying points of exposure that are open to assault. A Penetration Test differs from a Vulnerability Assessment in that it attempts to penetrate into the target by exploiting any discovered points of risk and exposure. A Penetration Test when done properly will result in an accurate deliverable that contains no false positives. This is possible because exploitation of a risk or point of exposure is either successful or not. Penetration Tests can include theoretical findings but they should not be reported on as positives.
Penetration Testing Tools include (I'd recommend these):
You can use a Vulnerability Assessment or a Penetration Test against any type of target not just technology based targets. At Netragard we perform physical penetration tests, wireless penetration tests, network penetration tests, social engineering based penetration tests, web application penetration tests, etc. Likewise we can deliver vulnerability assessments against the same set of targets if penetration testing is too aggressive.
(I get the feeling that both Rafal and Brian Chess think that Penetration Testing is a Web Application only service)
"Here's my logic - feel free to scrutinize. For the record I work for HP (the SPIDynamics acquisition) so you guys can feel free to rip on the fact that our marketing folks I'm sure make interesting claims as well... but I digress. Here are some things to consider:
Actually we've got quite a bit of interesting history with HP, but that's a different story. With respect to SPIDynamics and the Web Inspect tool, I'm sorry that HP ever acquired SPIDynamics. WebInspect was a reasonable tool for doing preliminary reconnaissance against Web Applications during non-covert services. Once HP acquired the technology its quality went down the tubes. Not only that but the process of acquiring a license from HP is excruciatingly painful at best. What ever happened to being able to purchase the product online? /end rant
"1. When you do penetration testing, what are you really testing? Are you testing the system or the intelligence and skill of the pen tester? This is a very tough question to answer."
Why is that a difficult question to answer? If you've built your penetration testing team properly then your team will be able to expose its targets to the same or greater threat level than that which they will likely face in the real world. The fact of the matter is the more secure the infrastructure the more challenging the test and yes, its impossible to know everything but its not impossible to do a great job.
"2. Pursuant to #1 above, and the business' (living in reality land here) need to do lowest-cost vendors... what value do you suppose that the 90%+ of companies that go lowest-cost (outsourced to India, China, Mexico, etc) are getting?"
Businesses do not "need to do the low-cost vendors", they choose to because they are making uneducated decisions in most cases. Mind you the lack of education on their part is not their fault, its the fault of the poor quality vendors. Poor quality vendors advertise their services as if they are the same quality as the high quality vendors thereby causing confusion. When a business compares the two services they don't see the difference and so they choose the less expensive one.
"3. With every point-and-click testing tool there is a double-edged sword... here's why 3a. Tools make you more efficient BUT"
I only partially agree. When the tool spits out over 2,000 false positives (like WebInspect did the last time we used it) with only 3 real positives its doing very little to increase the efficiency of a team. Other tools that produce less false positives and more accurate results are very useful for time savings but their results should not be used to create an end product. Automated tools are not dynamic by nature and as such can not identify the same risks as talented penetration testers.
"3b. Tools can make you less "hands-on" when it comes to writing low-level exploits or code..."
Tools are also the root cause of the the fraudulent security experts. I'm not saying that tools don't have their place because they certainly do. But they allow people to become lazy and as such breed "experts" that are for all intents and purposes no better than script kids (which might I add are very dangerous because they don't know what they are doing).
"4. Penetration testing is an after-the-fact requirement... which is too late. You have to use tools to augment and empower your developers to write better code at the grass-roots otherwise you're hosed."
You're partially wrong. The tools that you speak of are derived from the attacks that were created by Penetration Testers (aka: hackers). With respect to the world of Web Applications, do you think that a tool discovered the first SQL Injection vulnerabilitiy and created a method for exploitation? Ofcourse not! Tools will always be a few steps behind the capabilities of a real hacker, regardless of that hackers ethical bias. The fact of the matter is that as hackers, we perform research and identify new methods for penetration that were not previously discovered and your tools can not and will not ever be able to defend against that.
"So - to summarize, penetration testing isn't going to be "dead" in this year of 2009, but it may start to dwindle down some depending on how good the marketing machines of the tools vendors are. Brian's statement is a self-fulfilling prophecy... he is making a statement that he hopes will incite people to make that statement come true."
I disagree, and again, you are working for a vendor that makes these tools. Its in your best interest to suggest that some how Penetration Testing will be less of a requirement because of the tools that you create. The reality of it is that if people drink that kool aid they will become more vulnerable, not more secure.
When our military tests the armor of its M1A2 Abrams Tank they test it against the real threat. So why aren't we pushing our customers to do the same thing, it makes perfect sense? In our case the real threat is always going to be the malicious hacker, not the software vendor making pretty and easy to use tools. The tools do have a place but they will only ever identify the low hanging fruit. It takes a professional hacker/penetration tester to actually test an infrastructure properly. Lets see your tools perform Social Engineering or drop USB sticks in parking-lots.
Friday, January 2, 2009
ROI of good security.
The cost of good security is a fraction of the cost of damages that usually result from a single successful compromise. When you choose the inexpensive security vendor, you are getting what you pay for. If you are looking for a check in the box instead of good security services, then maybe you should re-evaluate your thinking because you might be creating a negative Return on Investment.
Usually a check in the box means that you comply with some sort of regulation, but that doesn't mean that you are actually secure. As a matter of fact, almost all networks that contain credit card information and are successfully hacked are PCI compliant (a real example). That goes to show that compliance doesn't protect you from hackers, it only protects you from auditors and the fines that they can impose. Whats more is that those fines are only a small fraction of the cost of the damages that can be caused by a single successful hack.
When a computer system is hacked, the hacker doesn't stop at one computer. Standard hacker practice is to perform Distributed Metastasis and propagate the penetration throughout the rest of the network. This means that within a matter of minutes the hacker will likely have control over the most or all of the critical aspects of your IT infrastructure and will also have access to your sensitive data. At that point you've lost the battle... but you were compliant, you paid for the scan and now you've got a negative Return on that Investment ("ROI").
So what are the damages? Its actually impossible to determine the exact cost in damages that result from a single successful hack because its impossible to be certain of the full extent of the compromise. Never the less, here are some of the areas to consider when attempting to calculate damages:
Now lets consider the Return on Investment of *good* security. An Advanced Penetration Test against a small IT Infrastructure (~50 computers in total) might cost something around $16,000.00-$25,000 for an 80 hour project. If that service is delivered by a quality vendor then it will enable you to identify and eliminate your risks before they are exploited by a malicious hacker. The ROI of the quality service would be equal to the cost in damages of a single successful compromise minus the cost of the services. Not to mention you'd be complaint too...
(Note: the actual cost of services varies quite a bit depending on what needs to be done, etc.)
So why is it that some vendors will do this work for $500.00 or $2,000.00, etc? Its simple, they are not delivering the same quality service as the quality vendor. When you pay $500.00 for a vulnerability scan you are paying for something that you could do yourself for free (go download nessus). Never the less, when you pay $500.00 you are really only paying for about 5 minutes of manual labor, the rest of the work is automated and done by the tools. (If you broke that down to an hourly rate you'd be paying something like $6000.00 an hour since you're paying $500.00 per 5 minutes). In the end you might end up with a check in your compliance box but you'll still just as vulnerable as you were in the beginning.
Usually a check in the box means that you comply with some sort of regulation, but that doesn't mean that you are actually secure. As a matter of fact, almost all networks that contain credit card information and are successfully hacked are PCI compliant (a real example). That goes to show that compliance doesn't protect you from hackers, it only protects you from auditors and the fines that they can impose. Whats more is that those fines are only a small fraction of the cost of the damages that can be caused by a single successful hack.
When a computer system is hacked, the hacker doesn't stop at one computer. Standard hacker practice is to perform Distributed Metastasis and propagate the penetration throughout the rest of the network. This means that within a matter of minutes the hacker will likely have control over the most or all of the critical aspects of your IT infrastructure and will also have access to your sensitive data. At that point you've lost the battle... but you were compliant, you paid for the scan and now you've got a negative Return on that Investment ("ROI").
So what are the damages? Its actually impossible to determine the exact cost in damages that result from a single successful hack because its impossible to be certain of the full extent of the compromise. Never the less, here are some of the areas to consider when attempting to calculate damages:
- Man hours to identify every compromised device
- Man hours to reinstall and configure every device
- Man hours required to check source code for malicious alterations
- Man hours to monitor network traffic for hits of malicious traffic or access
- Man hours to educate customers
- Penalties and fines.
- The cost of downtime
- The cost of lost customers
- The cost of a damaged reputation
- etc.
Now lets consider the Return on Investment of *good* security. An Advanced Penetration Test against a small IT Infrastructure (~50 computers in total) might cost something around $16,000.00-$25,000 for an 80 hour project. If that service is delivered by a quality vendor then it will enable you to identify and eliminate your risks before they are exploited by a malicious hacker. The ROI of the quality service would be equal to the cost in damages of a single successful compromise minus the cost of the services. Not to mention you'd be complaint too...
(Note: the actual cost of services varies quite a bit depending on what needs to be done, etc.)
So why is it that some vendors will do this work for $500.00 or $2,000.00, etc? Its simple, they are not delivering the same quality service as the quality vendor. When you pay $500.00 for a vulnerability scan you are paying for something that you could do yourself for free (go download nessus). Never the less, when you pay $500.00 you are really only paying for about 5 minutes of manual labor, the rest of the work is automated and done by the tools. (If you broke that down to an hourly rate you'd be paying something like $6000.00 an hour since you're paying $500.00 per 5 minutes). In the end you might end up with a check in your compliance box but you'll still just as vulnerable as you were in the beginning.
Monday, December 29, 2008
Brian Chess, CTO of Fortify Software - Creating Confusion
So this entry goes to support my previous post about Insecure Security Technologies and some of the confusion that these vendors can cause. Recently Networkworld published an article named "Penetration Testing: Dead in 2009" and cited Brian Chess, the CTO of Fortify Software as the expert source.
The first thing that I want to point out is that Brian Chess is creating confusion amongst the non-expert people who read the article linked above. The laymen might actually think that Penetration Testing is going to be dead in 2009 and as a result might decide to buy technology as a replacement for the service. Well, before you make that mistake read this entire entry. I'll give you facts (not dreamy opinions) about why Penetration Testing is required and why its here to stay.
As a side note, Brian Chess has a vested interest in perpetrating this fantasy because his objective is first and foremost to sell you his technology.
Technology, like Brian Chess's technology is a solution to a problem, which by definition means that the problem came first and the technology was always a few steps behind. With respect to IT Security, hackers are always creating new methods for penetrating into networks (the problem). Because those methods of attack are new, the technology is not able to defeat them (because the solution doesn't yet exist). So if technology can't protect you, then how do you protect yourself?
The best way to protect yourself is to use a combination of technology (to solve known problems) and Penetration Testing (to identify the unknown). A properly executed penetration test will reproduce the same or greater threat levels that your infrastructure will likely face in the real world. This is akin to testing the armor of the M1A2 tank. You shoot the armor with RPG's and armor piercing rounds so that you can study the impact and improve the armor to the point where it defeats the threat. As a result Penetration Testing can move your security posture well past the limits of what technological solutions have to offer. My professional recommendation is that both Technology and Penetration Testing should be used. Sorry Mr Chess, but telling people that Penetration Testing will be dead by 2009 is just fiction.
Moving on...
As a general rule of thumb I try to avoid saying that anything is 100% secure or invulnerable to attack because that sort of claim is impossible. But while reviewing the Fortify website I found the following text and thought it was worthy of note: "Fortify 360 renders software invulnerable to attacks from cyber predators." This sort of marketing fluff falls under the same class of confusing noise as Brian Chess's claim that Penetration Testing will be dead by 2009, total fiction. It is mathematically impossible for Fortify 360 to render software "invulnerable to attacks from cyber predators." unless the software is mathematically proven to be secure, and it hasn't.
If anyone disagrees with what I've said here by all means leave me a comment. If you can prove me wrong then I'll happily make corrections, but I'm pretty sure I'm on the ball with this one. And Mr. Chess, if you think that your technology renders your customers "invulnerable to attacks from cyber predators" then I challenge you to let my research team test an evaluation copy of your technology, after all the skills that we posses according to you are outdated and shouldn't pose a threat to your software. ;]
Tuesday, December 23, 2008
Insecure *Security* Technologies
There is not a single piece of software that exists today that is free from flaws and many of those flaws are security risks. Every time a new security technology is added to an Infrastructure, a host of flaws are also introduced. The majority of these flaws are undiscovered but in some cases the vendor already knows about them.
As an example, we encountered a Secure Email Gateway during an Advanced External Penetration Test for a customer. When a user sends an email, the email can either be sent from the gateway's webmail gui, or from outlook. If it is sent from outlook then the gateway will intercept the email and store the message contents locally. Then instead of actually sending the sensitive email message to the recipient, the gateway sends a link to the recipient. When the recipient clicks on the link their browser launches and they are able to access the original message content.
While this all looked fine, there was something about that gateway that made me want to learn more (a strange jboss version response), so I did... I called the vendor and ask to speak to a local sales rep. When the rep got on the phone I told him that I had an immediate need for 50 gateways but wouldn't make any purchases until I knew that his technology was compatible with my infrastructure. He got really excited and asked me what I needed in order to verify compatibility. I told the rep that I needed a list of all Open Source libraries and software that had been built into the gateway along with version information. The rep said that he didn't really understand what I was asking him but that he'd go to someone in development and figure it out. Within about fifteen minutes I received an email with a .xls attachment. Shortly after that I received an email from the rep asking me to delete the .xls attachment because he wasn't supposed to share that particular one.... go figure...
(I deleted it after I read it)
When I studied the document I realized that the gateway was nothing more than a common bloated linux box with a bunch of very, very old Open Source software installed on it. In fact, based on the version information provided, the newest package that was installed was OpenSSL and that was 3 years old! The JBoss application sever was even older than that and was also vulnerable as hell (but it was hacked and reported incorrect version information). Needless to say we managed to penetrate the secure email gateway by using a published exploit that was also about 3 years old. Once we got in our client decided that their secure gateway wasn't so secure any more and did away with it. We did contact the vendor by the way and they weren't receptive or willing to commit to any sort of fix.
The fact of the matter is that we run into technology like this all the time, especially with appliances. We've seen this same sort of issue with patch management technologies, distributed policy enforcement technologies, anti-virus technologies, HIDS technologies, etc. In almost every case we are able to use these technologies to penetrate or at least to assist in the penetration of our target. While most of these technologies introduce more risk than the risk that they resolve, there are a few good ones. My recommendation is to have a third party assess the technology before you decide to use it, just make sure that they are actually qualified and not Fraudulent Security Experts.
Friday, December 19, 2008
Raising Infrastructural Awareness in 2008
Before 2008 nobody had done any high visibility vulnerability research and exploit development against critical systems used to maintain our critical infrastructure. In early to mid 2008 that all changed. Initially Core Security released a security vulnerability for Citect SCADA. That security vulnerability got media attention because it was one that could be used to penetrate into important control systems that are used to control our infrastructure. (Electricity, Water, Gas, Oil, etc).
When the vendor released their statement about the vulnerability they downplayed the criticality of the issue in a very significant way. In our opinion that downplay was borderline unethical and was an attempt to save face. Fortunately for all of you who rely on electricity, running water, etc, we weren't going to stand for that. More specifically, Kevin Finisterre our lead researcher wasn't going to stand for it.
At first Kevin and I tried talking to the engineers about the criticality of the vulnerability. That discussion got us nowhere fast, the engineers simply didn't want to hear it and didn't want to assume responsibility for the problem. At that point Kevin decided to take the game to the next level, and this time the actual risk for the vulnerability would be proved.
Kevin decided that he would write an exploit for the Citect SCADA vulnerability, after all the vendor said that it was a low risk issue right? So Kevin did just that, he wrote an exploit and published it to the Metasploit Framework. Once word of that got out, the attitudes at Citect and those of the engineers changed so fast that heads spun. All of the sudden this non-critical issue was a critical issue and something had to be done.
So why was it so important for us to do that? Why did we feel that it was the ethical thing to do? Here's why....
An exploit had already been created by a few other people and was in circulation. So the bad guys had it and the good guys didn't. When Kevin published the exploit he evened out the playing field and gave the good guys the same caliber guns. When the good guys fired the gun the reality of their vulnerability was very apparent and only then did they jump to work on the issues. That said, some of them are still vulnerable.
Through out 2008 we kept on researching SCADA vulnerabilities and other security issues related to Infrastructural systems. As it turns out we caught a lot more interest than we thought we would have, and we had a much bigger impact on the industry than expected. Today Citect is taking security very seriously and many government agencies have become very aware of these risks.
Here is a podcast where people reference the work that we've done with vulnerability research and exploit releases. They never directly mention our names (go figure) but we all know who they are talking about.
Thursday, December 18, 2008
Utility Companies and Food for Thought
Something that I keep on hearing from engineers (power, water, etc) on the SCADASEC mailing list is that they are more concerned about human error causing an outage than an attack over the internet. Most of the incidents that I hear about are operator error and they involve accidentally shutting down a computer system or perhaps configuring one improperly (The utility guys like to call these "cyber" incidents). When that happens things "go to hell in a hand basket" fast and people can and do die. They seem to be more concerned about those types of "cyber" incidents than they are the hacker threat... but they're not getting it right?
The fact of the matter is that a malicious hacker could trigger any number of these "cyber" incidents either deliberatley or accidently, and the end result is the same. How do we get these guys to take the threat more seriously? I think its happening, but I don't feel like its happening fast enough.
The fact of the matter is that a malicious hacker could trigger any number of these "cyber" incidents either deliberatley or accidently, and the end result is the same. How do we get these guys to take the threat more seriously? I think its happening, but I don't feel like its happening fast enough.
Wednesday, December 17, 2008
Fradulent Security Experts
So I've been participating in the penetration testing mailing list that is hosted by securityfocus and I can't say that I am impressed. In fact, I might even go so far as to say that I am concerned about the caliber of the people that are offering paid services, here's why.
When a customer hires a security professional to perform a Penetration Test, Web Application Security Assessment, or any other service that customer should be getting a real expert. That expert should be able to assess the customers target infrastructure, application, or whatever and should be able to determine points of vulnerability and their respective risks. But that is not what I am seeing.
The other day a self proclaimed "expert" asked how dangerous a SQL Injection vulnerability was. They apparently identified a SQL Injection vulnerability in their customer's website but didn't know what to do with it!!! They also asked about how to exploit the vulnerability and what successful exploitation might do.
Well the first thing that came to mind was "Why the hell are you offering services if you don't know what you are doing?". I actually asked that but I didn't get any response back from the original author. When someone hires a security professional to deliver security services they expect those professionals to be subject matter experts. The unfortunate thing is that in most cases the customer has no way of verifying the professional's expertise and the customer gets taken for a ride. (Take a look at our white papers!!!)
Another example is in a recent vulnerability that one of my team members found. He was researching a product's webservice and found that it was just chalk full of holes. When he contacted the vendor, they responded with "but we just had a very extensive security assessment done against our product". We certainly couldn't tell... looks like they got taken for a ride like so many others.
Why is this a problem, why do I care? Its a problem because the providers who offer these low quality services advertise the same way as the high quality providers. The difference is that their service doesn't do anything to protect the customer, and ours does. We're not the only good security company out there, but we are one of very, very few.
When a customer hires a security professional to perform a Penetration Test, Web Application Security Assessment, or any other service that customer should be getting a real expert. That expert should be able to assess the customers target infrastructure, application, or whatever and should be able to determine points of vulnerability and their respective risks. But that is not what I am seeing.
The other day a self proclaimed "expert" asked how dangerous a SQL Injection vulnerability was. They apparently identified a SQL Injection vulnerability in their customer's website but didn't know what to do with it!!! They also asked about how to exploit the vulnerability and what successful exploitation might do.
Well the first thing that came to mind was "Why the hell are you offering services if you don't know what you are doing?". I actually asked that but I didn't get any response back from the original author. When someone hires a security professional to deliver security services they expect those professionals to be subject matter experts. The unfortunate thing is that in most cases the customer has no way of verifying the professional's expertise and the customer gets taken for a ride. (Take a look at our white papers!!!)
Another example is in a recent vulnerability that one of my team members found. He was researching a product's webservice and found that it was just chalk full of holes. When he contacted the vendor, they responded with "but we just had a very extensive security assessment done against our product". We certainly couldn't tell... looks like they got taken for a ride like so many others.
Why is this a problem, why do I care? Its a problem because the providers who offer these low quality services advertise the same way as the high quality providers. The difference is that their service doesn't do anything to protect the customer, and ours does. We're not the only good security company out there, but we are one of very, very few.
Wednesday, December 10, 2008
Conference with Green Hills Software
I recently gave a speech with Green Hills Software, Inc. in California. The presentation covered the real threat that businesses face as opposed to the theoretical threat that most people seem to worry more about. I also made it a point to uncover some of the more unorthodox attack methods that hackers use like the spreading of infected USB Sticks in parking lots or the use of rapid Distributed Metastasis.
Here are some articles that were written as a result of the conference:
EEtimes
EEtimes.eu
Astalavista
ISA
Here are some articles that were written as a result of the conference:
EEtimes
EEtimes.eu
Astalavista
ISA
Subscribe to:
Posts (Atom)