"I don't know if anyone in risk reads the PDF we send them. I mean, even we don't understand some of what we're reporting, so why should they?" "The CFO hates our risk management meetings. They look at these numbers we give them and have no idea if it means we're better or worse." “We have 217 metrics, but those metrics are changing week to week as we try to figure out what value a measurement has.” These are just some of the comments we’ve heard over the past year when talking to CISOs and data analysts about their attempts to use metrics to help quantify risk and prioritize resources. Metrics in security aren't easy – and not only because of "the math." Politically, metrics often end up orange because red is bad news, and green means the CFO may think we don’t need any more budget. When you try to move beyond Red-Amber-Green, you can easily find yourself playing "insight roulette," hoping to win big with the next dashboard you produce. So, how can we move beyond the current state of confusion and angst towards effective metrics for security? The first guiding principle is that if we're not reporting information that people can use to make decisions and drive significant change in risk exposure or security performance efficacy, then all we're doing is measuring things for the sake of it. The key here is to put security metrics in the context of business, IT and security process, as well as break big, broad numbers down into manageable projects that give teams the best cost action to solve a problem. The high-level metrics that go to the board must be grounded in the day-to-day reality of the team actually fixing stuff. It’s all too easy to focus on the “how” of security metrics:
- How do I get hold of data sources that seem relevant?
- How do I make sense of them?
- How do I correlate them?
It’s true there’s a lot of work to do here, but teams often get way ahead of themselves when they dive straight into the data. This leads us to the second guiding principle of security metrics – starting with the "what":
- What is the purpose of the metrics?
- Are we trying to get visibility into risk or measure compliance?
- What result will we enable with a metric?
- For whom is it actionable?
- In what timeframe?
- Is there budget to address the metric if it’s “red”?
- And what is the impact of the metric?
- Does it tick the box of "what matters most?" or not?
If you don’t have the answers to these questions, then put down the laptop/spreadsheet/risk report and back away slowly! Without devoting considerable thought to the ‘whats,’ investing resources in sorting the ‘hows’ will likely turn out to be a waste of time. And when everyone is saying there's a skills gap, that just doesn't make sense. I’m going to be talking at BSides Las Vegas about our experiences creating metrics and what we've found works when it comes to turning data from security tools into actionable insights that can significantly shift the security posture of organizations for the better. To find out the details and discover “How to make metrics and influence people,” come listen to my talk on Wednesday, July 26 at 6:00 PM.
About the Author: Leila Powell is a Data Scientist working in security. Leila used to use supercomputers to study the evolution of galaxies as an astrophysicist. Now she tackles more down-to-earth challenges, (yes, the puns get that bad), helping companies use different data sets to understand and address security risk. As part of the team at Panaseer (a London based security start up), Leila works with security functions in global financial firms, applying data science to help solve strategic and operational challenges. You can follow Panaseer on Twitter here. Editor’s Note: The opinions expressed in this guest author article are solely those of the contributor, and do not necessarily reflect those of Tripwire, Inc.
