I ran a vulnerability scan the other day on one of my test hosts with an unnamed scanning product from one of Tripwire’s competitors. ONE host. The scan was super easy to setup and run. Nice web interface. I got a link to a pretty PDF report. Great experience. The length of the report for this 1 host? 153 pages.
To be fair, this was not the most secure, locked down, impenetrable host in the world. But 153 pages! It made me painfully aware of the impossible tasks that we often ask IT security pros to deal with to provide basic must-have security controls.
I am not writing this post to bash a competitor, and I won’t even tell you who it is – because this is not just a problem that one vendor has or that I have not been a guilty contributor to in the last 17 years I’ve been building multiple vulnerability scanning products at different companies. It is a problem that the whole vulnerability management world has, and we ALL need to do a better job at this for our users.
I started to dig into the information in these 153 pages to figure out where we all went so wrong. The first thing that hit me as I read through the 218 items in this report was just how many of them were completely and utterly worthless and uninteresting to me in assessing the vulnerability of this system. Page 128 contained about half a page on traceroute.
Traceroute is a utility installed on most systems that sends a series of packets to a target host with increasing time to live values so that you can attempt to see the intermediary hops on the route from your system to the target.
While many client-side traceroute vulnerabilities have been published over the years, this system was not vulnerable to any of them. My vulnerability report included a traceroute to the scanned host. Now do security pros ever use traceroute to do recon on a host and find out useful information?
Absolutely. The hostnames of each hop on the path to my target host reveal things like what city this host is probably in, who the hosting provider for this host is, and intermediary targets to attack. Some of that is a little difficult to ascertain in this particular report since the DNS names are not resolved and you just get IP addresses.
But my problem is not the lack of DNS resolution. My problem is: TRACEROUTE IS NOT A VULNERABILITY!
To play the role of vendor’s advocate for a moment, I might say “yes, but I didn’t say it was a vulnerability – it is just information that we were able to pick up and report for you about the host.” But, thinking as a user, I ran a vulnerability scan. I didn’t run a “tell me everything possible about this host regardless of its value scan!”
Unfortunately, all too often vulnerability scans have turned into exactly that. This was not an isolated issue – 40% of the report was full of non-vulnerabilities exactly like this. Another 15% were issues that may or may not have been actual vulnerabilities on the system. Thank you for telling me that my system may or may not be vulnerable – that just about covers all the possibilities.
Now, is every company that builds a vulnerability management product stupid? Actually, no. Super smart engineers are at every one of them. I can pinpoint exactly why these reports have turned into total bloatware, and it is perfectly logical how it happened. Every sales person, for every company that has ever sold a vulnerability management product in the history of the world, has heard a prospect utter this line to them at one point in their life:
“I just ran a comparative set of scans using both your product and a competitor’s product and the other product is finding more vulnerabilities than yours”
Of course, they know that the competition may have produced a report with more in it, but that does not mean that there is more useful, relevant, actionable, accurate information in that report. Surely the customer wants the most concise, useful information possible.
But, it is a long laborious uphill battle to start going through what is now thousands of pages of reports across numerous hosts and products and trying to extract and effectively communicate that comparison to a prospect. The reality is there often are real differences in the vulnerabilities found and those are extremely legitimate questions to ask – but in the sea of worthless information it becomes a lot harder to truly compare the real, relevant differences that should be evaluated.
So while they may actually do that, the reaction back at the company is also usually asking “how do I stop getting this objection raised in the first place?” The result of this has been a decades long race to find MORE information about a host and shove it all into the first report a prospect sees. Regardless of how useful it is, there is a perception that somehow the longest report wins.
And that is unfortunate — because traceroute is not a vulnerability.
I faced this problem again when Tripwire started offering its PureCloud vulnerability scanning service. This was one of the most fun projects I have worked on because I had the chance to redesign and reimagine what makes up a vulnerability management system from the ground up.
We drew a line in the sand in the design of the initial report out of the system and declared NO MORE USELESS INFORMATION IN VULNERABILITY REPORTS! The length of the report against that same test host? 8 pages. I was so proud of what we had delivered.
How long do you think that service lasted before my sales guys called me up and said, “Dave, we’re getting killed here the competition is finding so many more vulnerabilities than us!” I did comparisons. I explained the differences. I pointed out the fallacy of always believing bigger is better. And at the end of the day, while everyone acknowledged that I was right, it came right back to “But it is still an objection we have to continually overcome.”
I hemmed and hawed and dug in my heels for what I knew to be right. Eventually, though, I said if this is what people keep saying, then let’s go ahead and give them what they want – a long report. So we did. The objections went away; my sales team was happy, and life was good.
But our engineering team sneaked in a little option – the ‘Actionable Report’, right next to the big long one, so that no user looking for vulnerabilities need ever be subjected to traceroute output.
I hope for the day where all vulnerability scanners produce short, useful reports by default, and we all agree to make the fight over the most accurate, relevant, and useful data — not the longest report.
- Heartbleed and Your SOHO Wireless Systems
- Stopping the Heartbleed
- Detecting Heartbleed Exploits in Real-Time
- How to Detect the Heartbleed OpenSSL Vulnerability in Your Environment
The Executive’s Guide to the Top 20 Critical Security Controls
Tripwire has compiled an e-book, titled The Executive’s Guide to the Top 20 Critical Security Controls: Key Takeaways and Improvement Opportunities, which is available for download [registration form required].
Title image courtesy of ShutterStock