David Spark here reporting for Tripwire at the 2010 RSA Conference in San Francisco.
Your organization is measuring an endless stream of data. You could get buried trying to look at it all. The question is, “Why are you looking at it all?” Shouldn’t you just be looking at the good stuff? The stuff that predicts incident rates? That would be great if you knew what to actually be tracking. And are you finding enough incident/outcome data so you can pinpoint the key factors that improve prediction within your enterprise security environment?
Those were the questions and the topic of discussion at “Proving the Worth of Security Metrics with Real-World Data.”
On the panel were:
Andrew Jaquith, Senior Analyst, Forrester Research, Inc.
Gene Kim, CTO, Tripwire, Inc.
Marcus Sachs, Executive Director, National Security and Cyber Policy, Verizon Business
Moderator: James Cowie, CTO, Renesys
To a standing room only crowd, here’s a summary of some of the issues and answers that came up in the discussion of “Proving the Worth of Security Metrics with Real-World Data:”
- Your compliance metric determines whether you are walking the walk that you are talking.
- When it comes to security metrics, don’t just measure inputs, measure outputs.
- Can metrics demonstrate that the cost of collecting and tracking?
- Will this metric will pay off with measurable improvements in organizational capabilities and/or reduced vulnerabilities?
Traditionally, when you examine data metrics and security, there’s a focus to look at the losers that succumb to data breaches. Gene Kim, Tripwire’s CTO, chose to study the winners, those companies that are high performers. He reviewed the statistics of these high performing companies and how they correlate with security performance. By Kim’s estimation, high performing companies comprise only five percent of all organizations with the rest evenly split between medium and poor performers. Here are some statistics from a report by IT Process Institute (May, 2006).
- High performers maintain a posture of compliance. They have the fewest number of repeat audit findings and they extend one-third the effort for audit preparation.
- High performers find and fix security breaches faster. They’re five times more likely to detect breaches by automatic control. At the same time they’re five times less likely to have breaches result in a loss event.
- When high performers implement changes, they have 14 times as many changes. They have one-half the change failure rate and one-quarter the first fix failure rate.
- When high performers manage IT resources, they deal with one-third the amount of unplanned work. They have eight times more projects and IT services and six times more applications.
On the flip side, Verizon publishes an annual report on the losers. A data breach investigation report that shows how the bad guys get your data. They operate under the theory that we learn a lot from failures. Here are some findings from their most recent study:
- 99.6% of records were compromised from servers and applications. Data breaches are not coming from client computers. It’s like asking a bank robber why do you rob the bank? That’s where the money is.
- 91% of all compromised records were attributed to organized criminal groups. People trying to break into your networks are doing it for monetary gain.
- 74% data breaches resulted from external sources. Often, people are quick to assume that a data breach is an inside job. More often than not, it’s from the outside.
- 69% data breaches were discovered by a third party. This is depressing. The company that was broken into didn’t know they were broken into until someone else told them. Why are you logging all of your activity if you don’t know what’s going on?
- 67% of data breaches were aided by significant errors.
- 32% of data breaches implicated business partners
If you do common business practices, such as good passwords and checking your logs, you won’t be broken into. It’s the companies that say one thing but do another are the ones that get broken into.
Check out more of Tripwire’s coverage from the 2010 RSA Conference in San Francisco.