Skip to content ↓ | Skip to navigation ↓

Last week, I attended the ISACA-Silicon Valley‘s Foundation2Innovation Conference in San Jose, and led a session on  “Covering the Enterprise End-to-End.”  We had a lively discussion around this topic, particularly regarding how to use metrics and other data to govern information security and make the metrics matter.

One of the questions following this session was, “What does it take to get people to actually pay attention to our metrics and do something about it?”  I thought I’d share some of the critical elements, from my perspective:

  1. Make sure your business recognizes the linkage between your metrics and business success.  Don’t talk about things you care about – talk about things they care about, and use terms they understand (i.e. business services, business units, products, revenue, profit, cost, loss, etc.).
  2. Make sure your metrics drive action and decisions.  It is fine to present status and tracking indicators, but identify and focus on a small number of key indicators that drive action.  Ask yourself things like the questions below, to determine if your metrics are adding value:
    • “If this metric goes up, does it increase my confidence and make it easier to sustain my projects?”
    • “If this metric goes down, do I take action immediately to correct it?  Does anyone in the business care?”
    • If I want to change this metric, can I?”
  3. Ensure you have “tone at the top.”  If you want to see results, having top-level management commitment to monitor the metrics and take action when there is a problem is key.  If there are no consequences for poor performance, then poor performance is acceptable and will not improve.
  4. Make sure your metrics are reasonable and achievable, and support the goals of the organization.  I have seen organizations get stuck (or even give up) because their metrics were not attainable.  Whether it is due to skills, resources, practicality, or some other reason, make sure you are not setting your organization up to fail.
    Also, if you’re dealing with a top-level, core metric and you don’t have the resources to succeed then something may be wrong.  Perhaps what your measuring isn’t truly important to the organization, or the linkage (see #1) is not clearly understood.  If what you’re doing matters to the organization, alignment of resources, targets, execution, and metrics is critical.
  5. Ensure your metrics drive the right sort of improvement.  One of the risks in metrics is creating a framework that either drives unintended consequences or allows people to game the system.  To guard against this, periodically review the behaviors your metrics are driving, and adjust them as necessary.  Also spend some time looking at where the metric “pushes” people to do something different.  For example, I’ve been working with an organization that tracks the percentage of incidents detected by an automated control.  When I asked them why, they had a great rationale:
    • If we focus on driving up the percentage of incidents detected by an automated control, a few things happen:
      • We improve our ROI / cost profile by automating things and taking human inspection out of the loop for more incidents.
      • We evaluate our incidents to determine what processes, monitoring, or additional controls could help us catch things early without human intervention.
      • We take a lot of the “grunt work” off of our analysts plates, which makes them happier.
      • We gain a clearer understanding of which of our controls are actually contributing the value we expected them to contribute.
      • We are pushed to codify the thinking we use to identify suspicious activities in our environment – rather than that knowledge living inside a smart person’s head, we have brought a great deal of it out into scripts, rules, documents, and other re-usable artifacts.

Hopefully this will give you some food for thought in these areas.  If you have other ideas (or vexing problems) I’d love to hear about them.