Network Vulnerability Scanning Doesn't Protect You

Vulnerability scanning can have a detrimental negative impact on the security posture of your IT infrastructure if used improperly. This negative impact is due to a perceptional issue that has been driven by the vendors who sell vulnerability scanning services or the vulnerability scanners themselves. The hard facts prove that vulnerability scanners can not protect your IT Infrastructure from malicious hackers. (My team penetrates "scanned" networks on a regular basis during customer engagements). That is not to say that vulnerability scanners are useless, but it is to say that people need to readjust their perception of what vulnerability scanning really is.

While there are various types of vulnerability scanners they suffer from the same disease that most security technologies suffer from. That disease is that they are reactive to hackers and will never be proactive. The fact is that vulnerability scanners can not detect vulnerabilities unless someone has first identified the vulnerability and created a signature for its detection. This process can take quite a while and is often not an ethical one. So here is how it works...

A hacker decides to perform research against a common technology like your firewall. That hacker might spend minutes, months or even years doing research just for the purpose of identifying an exploitable security vulnerability. Once that vulnerability is identified the hacker has an ethics based decision to make. Does he notify the vendor of his discovery and release a formal advisory or does he use his discovery to hack networks, steal information and profit.

If the hacker decides to notify the vendor and release an advisory then there is usually a wait period of 1-3 months before the vendor releases a patch. This lag time means that the vendor's customers will remain vulnerable for at least another 1-3 months, most probably longer. What's even more interesting is that this vulnerability may have been discovered previously by a different researcher that didn't notify the vendor. If that's the case then that probably means that the vulnerability has been in use as a tool to break into networks for a while. Who knows, it could have been discovered months or even years ago? That type of unpublished vulnerability is known as a 0day and is the favorite weapon of the malicious hacker.

At some point the vulnerability does become public knowledge. Its also at this point that the vendors who make the vulnerability scanning technology become aware of the new risk. When they do learn about the new risk they need to develop a signature, or script for their scanning technology so that it can detect the risk. That development process can take anywhere from a few days to a few weeks depending on the complexity risk. As a result, the customers that rely on vulnerability scanning are in the dark until the vendor can publish a working and tested signature... but the hackers don't need to wait at all. The hackers can use it almost immediately.

So in summary, there is a large risk window between the point of discovery of a vulnerability and the point at which a vulnerability scanner can detect the vulnerability. This risk and exposure window is usually never smaller than a few months, and can be as large as several years. During that time there is a very good chance that malicious hackers will be using your undiscovered risks to penetrate into your infrastructures. Whats worse is that you'll have no idea that you've been hacked because like vulnerability scanning technology, Intrusion Detection technology also can't identify threats if it doesn't know what to look for. Moreover most Intrusion Detection technologies aren't configured properly and as such don't work properly.

Unfortunately the story doesn't end there. Vulnerability scanners also suffer from significant issues with accuracy. In all cases where I've used (various) vulnerability scanners, the best results that I've ever achieved were about 30% accurate. This means that most of the vulnerabilities that were detected during my various scans weren't actually vulnerabilities but instead were false alarms, also called false positives. More frightening is the number of vulnerabilities that I discovered while performing Real Time Dynamic Testing (manual hacking) that were entirely missed by the vulnerability scanner. If you don't believe me then go download a free vulnerability scanner, test your network and verify the results yourself.

This inaccuracy is partially due to the architecture of the vulnerability scanners and the fact that no two networks are alike. Vulnerability scanners use static signatures or scripts that are only capable of checking a target for a vulnerability if their syntax is exactly accurate and if the target responds in a way that the scanner can understand. If however the target, lets say its a computer system, is configured in a custom way then it may not respond in a way that the scanner will understand (how many of you keep the default configuration?). This communication barrier is a large part of what causes false positives and false negatives.

An important note about false positives and false negatives. Some vendors claim that their vulnerability scanners have low rates of false positives. As with Intrusion Detection, if low false positive rates are true then its usually reasonable to say that the technology has high rates of false negatives. You can think of it as a sliding scale of 1 to 10 where 1 is 100% False Positives and 10 is 100% False Negatives. As you move up and down the scale you inevitably end up with more of one or the other, you can never eliminate them. With that said, its my opinion that more false positives are better than more false negatives.

If vulnerability scanners aren't the right way to protect yourself then what is? You should protect yourself by exposing your business to an accurate and controlled reproduction of the threat by using a quality security provider. It is important to remember that no single hacker, good or bad, has access to all of the 0-day's in the world. As such, it is entirely possible for a team of ethical hackers to accurately reproduce the threat that unethical hackers can create. Testing at that level enables you to identify weaknesses in your defenses that would not otherwise be detected by testing at lesser levels. What good would a penetration test or a vulnerability assessment do if the malicious hackers will test you harder?

One of the many advantages of using a team of talented hackers for security testing instead of relying on automated vulnerability scanners is that those hackers can and should perform research against unique technologies that they encounter during a security test. I practice what I preach by the way. When our team delivers an Advanced Penetration Test to a customer we always perform our own research against interesting targets. Those targets can be Web Applications, Web Services, or even custom daemons running on systems. In the end, if we find something new we'll write an exploit (proof of concept) for the customer and include that in the final deliverable.

In closing, I am not suggesting that network vulnerability scanners are bad because they do have their place and they do serve a purpose. They are particularly useful in the hands of a skilled security expert especially when performing reconnaissance against large networks. In that scenario the scanner enables the expert to save time and to rapidly collect intelligence about targets given that the engagement is non-stealth in nature. With that said, I wouldn't rely on scanners for anything more than just reconnaissance, at least not yet.



Note: (Thank you to minoo for pointing out a few mistakes in my previous revision of this entry. I hope that this entry is as clear as I intend it to be. There is no one team that is the best, but there are only a few good ones. If this isn't clear enough or if it needs more revision please comment.)