Finding The Quality Security Vendor (Penetration Testing, Vulnerability Assessments, Web Application Security, etc)

While I've written several detailed white-papers on the subject of identifying quality security vendors, I still feel compelled to write more about the subject. It is my opinion that choosing the right security vendor is critical to the health and safety of a business.  Choosing the wrong vendor can leave you with a false sense of security that in the end might result in significant damages. Often times those damages can't be fully measured and appreciated, especially when they involve the tarnishing of a good name.

This problem of identifying quality isn't new but it does take on a new importance when it involves the safety of your trade secrets, source code, or otherwise critically sensitive information.  When you trust a security provider to test your IT Infrastructure, your people, physical security, etc. you are relying on them to identify risks that malicious hackers might otherwise discover.  If the provider does not test you at the same threat level as the malicious hackers then their service is almost useless. 

If that doesn't compel you to want quality security services then go ahead and take the risk.  I suppose the question really is, how much is your network (and its data) worth? If its worth more than $500,000.00 then its probably worth spending money on a quality security vendor to protect it right?

So how do you know which providers are quality and which ones are frauds?

The first rule of thumb is to watch out for the vendors that produce deliverables that are the product of vulnerability scanners.  There are two reasons for this, the first being that you don't need to pay anyone to run an automated scan when you can do it yourself for much less, or for free. You can choose from a variety of free tools like nessus, or you go out and buy a license for a vulnerability scanner.  

Don't be fooled though, vulnerability scanners do not produce accurate results. In fact most vulnerability scanners produce results that contain anywhere from 40-90% false positives with an unknown rate of false negatives.  While these tools are useful for reconnaissance they should not be used as the primary method for security testing. 

Watch out for the vendor that tells you that they will run a vulnerability scan against your network and then "vet" the results.  Vetting doesn't mean that they are going to do additional discovery. Vetting only means that the vendor will check the results of the vulnerability scan and eliminate the false positives. The quality of the end product is then only as good as the accuracy of the vulnerability scanner. Would you bank on that?

When you are choosing the vendor make sure to ask them specific questions.  Questions that I find helpful are realistic but based on theoretical architectures.  For example you could ask a vendor the following question:

"Suppose you are confronted with an architecture that consisted of 10 desktops behind a single firewall.  That firewall has properly configured IPS capabilities and there are no ports forwarded from the internet to any system behind that firewall. How would you [the vendor] penetrate into that network? Once you penetrate how would you perform Distributed Metastasis?" Email me for the answer if you don't know it already. 

You can also ask the vendor how they would use a directory traversal vulnerability to penetrate into a network.  This is a bit of a trick question but if they know what they are doing then they will be able to answer it properly.  The short answer is that you need to inject code into the web-server's error log and then use the directory traversal vulnerability to render the code. (Again, if you need the complete answer email me and I'll get it to you.)

 

Another good rule is to only choose security vendors who also perform Vulnerability Research and Development ("R&D").  That is to say that the vendor must frequently perform security research against technology, identify vulnerabilities in that technology, create exploits for those vulnerabilities and must release formal security advisories. If they don't then chances are they don't know how to do it, but why is R&D important? 

R&D enables the vendor to keep its penetration testing skills honed (so long as the research done by the penetration testers).  Penetration Testers who do not perform this kind of research are literally Script Kids (sorry guys).  Script Kids are people who download tools and use those tools to penetrate into networks. In almost all cases they don't have any understanding of how the tools work.  If you think about it, thats like giving a loaded gun to a 3 year old. 

You can also ask the vendor how they collect their threat intelligence.  Threat intelligence is a critical aspect of delivering quality security services.  If the vendor doesn't have current threat intelligence about the threat then how will they help you to defend yourself against the threat? While I won't tell you how my team collects this intel, I will tell you that its not from the news and most certainly not all public forum. 

In closing, my recommendation to you is that you do your homework before you choose a vendor. Research the components required for delivering a quality service, then use your research to question the provider.  As an example, if you were going to get a Web Application Penetration Test ask the vendor to define the term "Penetration Test".  Ask the vendor what the difference is between a Penetration Test and a Vulnerability Assessment.  Also ask them to explain RFI, LFI, XSS, SQL Injection, Blind SQL Injection, etc.  Remember, you are going to spend money on security, might as well make it worth while.  If you don't then you're just adding that money to the damages from the hack that you'll suffer in the end. 

If you have any questions please feel free to leave me a comment or send me an email.  You might also want to check out the white papers that I've linked at the upper right hand corner of this blog.  Those papers go into more detail about how to choose a good security vendor and how to select the right service.