So this entry goes to support my previous post about Insecure Security Technologies and some of the confusion that these vendors can cause. Recently Networkworld published an article named "Penetration Testing: Dead in 2009" and cited Brian Chess, the CTO of Fortify Software as the expert source.
The first thing that I want to point out is that Brian Chess is creating confusion amongst the non-expert people who read the article linked above. The laymen might actually think that Penetration Testing is going to be dead in 2009 and as a result might decide to buy technology as a replacement for the service. Well, before you make that mistake read this entire entry. I'll give you facts (not dreamy opinions) about why Penetration Testing is required and why its here to stay.
As a side note, Brian Chess has a vested interest in perpetrating this fantasy because his objective is first and foremost to sell you his technology.
Technology, like Brian Chess's technology is a solution to a problem, which by definition means that the problem came first and the technology was always a few steps behind. With respect to IT Security, hackers are always creating new methods for penetrating into networks (the problem). Because those methods of attack are new, the technology is not able to defeat them (because the solution doesn't yet exist). So if technology can't protect you, then how do you protect yourself?
The best way to protect yourself is to use a combination of technology (to solve known problems) and Penetration Testing (to identify the unknown). A properly executed penetration test will reproduce the same or greater threat levels that your infrastructure will likely face in the real world. This is akin to testing the armor of the M1A2 tank. You shoot the armor with RPG's and armor piercing rounds so that you can study the impact and improve the armor to the point where it defeats the threat. As a result Penetration Testing can move your security posture well past the limits of what technological solutions have to offer. My professional recommendation is that both Technology and Penetration Testing should be used. Sorry Mr Chess, but telling people that Penetration Testing will be dead by 2009 is just fiction.
Moving on...
As a general rule of thumb I try to avoid saying that anything is 100% secure or invulnerable to attack because that sort of claim is impossible. But while reviewing the Fortify website I found the following text and thought it was worthy of note: "Fortify 360 renders software invulnerable to attacks from cyber predators." This sort of marketing fluff falls under the same class of confusing noise as Brian Chess's claim that Penetration Testing will be dead by 2009, total fiction. It is mathematically impossible for Fortify 360 to render software "invulnerable to attacks from cyber predators." unless the software is mathematically proven to be secure, and it hasn't.
If anyone disagrees with what I've said here by all means leave me a comment. If you can prove me wrong then I'll happily make corrections, but I'm pretty sure I'm on the ball with this one. And Mr. Chess, if you think that your technology renders your customers "invulnerable to attacks from cyber predators" then I challenge you to let my research team test an evaluation copy of your technology, after all the skills that we posses according to you are outdated and shouldn't pose a threat to your software. ;]
lawl, pwnt.
ReplyDeletePCI DSS requires quarterly penetration testing (sections 11.2-11.3). Guess it will be still in use for companies wanting to be in compliance ;)
ReplyDeletenice article simon i would love to put some of his software to "peer review" muhahaha
ReplyDeleteNo disagreement here. I love it when people make ridiculous claims about their products; it always takes me back to the days of the "Unbreakable Oracle." Mr. Fortify CTO should understand security well enough not to make such absolutist claims, but hey, it's his lame ass funeral. I wonder if the Fortify folks have stopped doing assessments of their www. ;)
ReplyDeleteHowever demonstrably erroneous I believe Brian's comments regarding pen testing to be, I can say with a reasonable level of certainty that Brian probably had nothing to do with that ridiculous claim about Fortify 360.
ReplyDeleteHe is the CTO, not Director of Marketing. While putting together a little piece myself I went to many of the static analysis tool websites and pulled out a number of outright lies, misleading statements, and cute but unsubstantiated jabs at other industry sectors.
While I'm disappointed, I can't really say I'm surprised. We all hate the assholes in marketing, and for good reason, but all we can do is let them fight it out, and in the meantime let the customers figure out what's really valuable.
You are right, he is the CTO and not the marketing director, I'm a CTO too. A part of my responsibility is to make sure that the message that we portray to our clients is truthful, accurate, and realistic. I do not allow anything to be posted on our website that violates our standards or warps reality. Security 101 mandates that absolutes are a big no-no doesn't it? But anyway, you're right. What Brian Chess did say was that Penetration Testing would be dead by 2009 and that was based on nothing more than his wishful opinion.
ReplyDeleteThe art of penetration testing and 'quality assurance' isn't nailed down to computer security only, at all, so I wouldn't doubt this useful field staying around for much longer than.. tomorrow.
ReplyDeleteIf I may call a sanity timeout here folks - while I don't agree with Brian's assertions necessarily - if you combine a few factors you could conceivably come to the conclusion that penetration testing will start to dwindle (just not as quickly as 2009). Here's my logic - feel free to scrutinize. For the record I work for HP (the SPIDynamics acquisition) so you guys can feel free to rip on the fact that our marketing folks I'm sure make interesting claims as well... but I digress. Here are some things to consider:
ReplyDelete1. When you do penetration testing, what are you really testing? Are you testing the system or the intelligence and skill of the pen tester? This is a very tough question to answer.
2. Pursuant to #1 above, and the business' (living in reality land here) need to do lowest-cost vendors... what value do you suppose that the 90%+ of companies that go lowest-cost (outsourced to India, China, Mexico, etc) are getting?
3. With every point-and-click testing tool there is a double-edged sword... here's why
3a. Tools make you more efficient BUT
3b. Tools can make you less "hands-on" when it comes to writing low-level exploits or code...
4. Penetration testing is an after-the-fact requirement... which is too late. You have to use tools to augment and empower your developers to write better code at the grass-roots otherwise you're hosed
So - to summarize, penetration testing isn't going to be "dead" in this year of 2009, but it may start to dwindle down some depending on how good the marketing machines of the tools vendors are. Brian's statement is a self-fulfilling prophecy... he is making a statement that he hopes will incite people to make that statement come true.
Cheers all.
Ehem, the only reason that someone would have to stop using pentest to ensure that his assets are secure, is knowing this in another way. That will not happen in 2009 for sure.
ReplyDelete