Skip to content ↓ | Skip to navigation ↓

This week I attended the Gartner Security & Risk Management Summit in Washington, D.C.  I attended a lot of very good sessions, but the one that left the biggest mark on me was a session called “Metrics That Matter,” delivered by Jeffrey Wheatman.

I went to this session because I’ve had a lot of conversations with information security executives this year, and a common question is “What should I really be measuring?,” or they make comments like “I report on a lot of things, but I am not sure what the top security indicators are that I should roll up to my executive team.”

Wheatman shared a really good list of “Five characteristics of effective metrics,” and I think it is a good litmus test for our metrics (security or otherwise).  I’ll paraphrase some of my session notes so you can get a feel for this:

  1. Effective metrics must support the business’s goals, and the connection to those goals should be clear.
  2. Effective metrics must be controllable. (In other words, don’t report on the number of vulnerabilities in your environment, since you can’t control that.  Instead, report on the % of “Critical” systems patched within 72 hours, which you can control)
  3. Effective metrics must be quantitative.
  4. Effective metrics must be easy to collect and analyze. (Wheatman says “If it takes 3 weeks to gather data that you report on monthly, you should find an easier metric to track.”)
  5. Effective metrics are subject to trending.  (Tracking progress and setting targets is vital to get people to pay attention)

This set of guidelines really resonated with me, and I am going to run my metrics through this regimen to make my own metrics better.  If you’re a Gartner client, there is a detailed research report from Wheatman on this topic, and I suggest you grab a copy.

The other thing I’ve noticed is that there seems to be a gap out here in the real world in terms of effective security metrics that are “universal” and also meet these criteria.  So, I’m on a quest to find and/or establish some good ones that transcend company boundaries.

If you’ve been reading my posts here you know I’d like your help.  If you have either a) good metrics that are working; b) vexing metrics problems you’d like to collaborate on; I would love to hear from you.  Drop me a line at “dm at” and let me know what’s on your mind.

10 Ways Tripwire Outperforms Other Cybersecurity Solutions
  • I saw that session material as well. It would have been nice if there were more use cases to explain the way security metrics could be connected to business goals. The way things stand today with BYOD, cloud, federation, and other trends the business is jumping into the metrics may be about how many security events their latest toys are popping up onto the board. 

    Since Quest & Tripwire have been used in conjunction in many places to help deliver on #3, quantitative metrics, and #4, making metrics easy to gather, I thought I'd chime in about how sometimes it's about the tools you choose in this area. Quest's part is making data that you simply can't get other ways appear. Whether its translating native log gibberish into human readable events that can become real metrics or allowing you to wrestle complicated processes into automated workflows which can then be tracked, Quest tries to make everything more actionable and understandable. And, of course, when you add the Tripwire layer above that to make sure that good data fits into the right contexts, you're most of the way done. 

    • Thanks, Jonathan – great commentary.  I'm also looking for good examples of how to connect the metrics to goals.  I have spoken with a few folks who've made the connection, but it is definitely an area of pent-up demand from what I see.  One of the other challenges is that the "hands on" practitioners have a very different perspective from their hands-off execs who are more driven by news articles and more indirect influences.

<!-- -->