Skip to content ↓ | Skip to navigation ↓

Recently, Tripwire donated its Benchmark service to the Center for Cyber Security, Information Privacy and Trust at Penn State’s College of Information Sciences and Technology. Last week, Penn State released its first round of research on how the top 25 percent of vulnerability management participants lowered their security risks.

But what exactly is Benchmark and how can you use it to improve your organization’s security? In this blog post, I’ll outline the Benchmark program and how metrics can help connect security to your business.

What is Benchmark?

A benchmark is a standard measure created to compare performance and organizational goals. The term “benchmark” often relates to broadly understood metrics that have become consistent and clear enough for industry use.

We can easily see benchmarks in the financial services world: the performance of stocks, bonds, mutual funds, ETFs, annuities, insurance (and so forth) each have benchmarks to give the context of consistent comparison.

For example, how often have you heard a stock’s performance compared to a standard “benchmark” such as the DOW or S&P 500? Those indexes have become typical, standard and important benchmarks to compare the success of stocks.

In the cyber security world, the new analytics tool “Benchmark” has been in play for a few years, allowing organizations to track, trend and compare their security performance. Benchmark lets organizations set their own internal goals and analyze how well they’re doing on their overall security posture. In addition (or alternatively), Benchmark allows organizations to contribute anonymized data to a community of contributors in order to see how they compare against a broader group.

A simplified C-suite narrative by the CISO might go like this: “Our overall security performance this year is significantly improved by approximately 27% over last year, but compared to the Benchmark Community of those contributing vulnerability assessment data; we’re performing only 50% as well as our peers. In other words, we’re falling behind our competition and need to improve more in the coming year.”

Why are metrics and Benchmarking important?

The security industry is maturing and just like any industry, it’s difficult to improve what you’re not tracking and measuring. Simply put, security professionals and the industry at large need benchmarks–defined standards against which we can accurately measure security performance.

It’s common knowledge that organizations need multiple layers of technology, processes and practices to improve safety and/or minimize developing security issues. There are a few industry standards, like the Consensus Information Security Metrics (CIS), that offer their own performance goals, but few groups share metrics (and details for how to improve them) with others. We need metrics that rapidly evolve with the new as well as past threats organizations face.

What are the key characteristics of good metrics?

Regardless of the industry, good metrics will tend to be based on the following:

  • Factual and objective (not “our firewall stopped 1.6M ‘suspected’ attackers”)
  • Measured consistently and regularly (daily, weekly, monthly, quarterly)
  • Created with numeric data representing relationships (such as ratios, percentages, numeric contrasts)
  • Normalized rationally and consistently across multiple controls or different technologies (e.g. three flavors of anti-virus, each with different scan cycles, unique whitelists, and updated on distinctive schedules.)
  • Trending and comparative (how are we doing month/month, quarter/quarter, year/year compared to our competitors?)
  • Issued on normal business intervals (monthly, quarterly, annually, etc.)

Overall, when you present good metrics, you can tell a story that’s based in numbers, timelines, and facts. These stories will resonate more strongly with the C-suite and boards of directors because they’re very similar to how other business metrics are presented. It’s a style they’re used to and can more easily understand.


Related Articles:



picThe Executive’s Guide to the Top 20 Critical Security Controls

Tripwire has compiled an e-book, titled The Executive’s Guide to the Top 20 Critical Security Controls: Key Takeaways and Improvement Opportunities, which is available for download [registration form required].


Images courtesy of ShutterStock