Skip to content ↓ | Skip to navigation ↓

Are you ready for the Security BSides San Francisco event? It’s just a few short weeks away, and we are taking the opportunity to highlight a few of the informative sessions scheduled, with the first three articles in the series looking at Craig Young’s A Day in the Life of a Security Researcher, Ken Westin’s Telmex Email Security Hole – My Email was Indexed by Google, and Lance Cottrell’s Using System Fingerprints to Track Attackers.

picNext up is a session being presented by Jared Pfost (@JaredPfost) titled Building an SDL Metrics Program, which will examine how a robust metrics program can offer an organization’s leadership unparalleled visibility into on-going security control performance.

Pfost is the VP of Strategic Services & Products at Caliber Security Partners, and has worked in information security for 19 years, and includes a combination of working in IT Security teams and consulting with designing and shipping security software in start-ups.

Pfost is a self-proclaimed process nut who has demonstrated that we don’t need unlimited resources to run a measurable, accountable, and effective security shop. He previously founded Third Defense which was acquired by Caliber Security Partners in 2013, worked at Washington Mutual managing the security architecture, business security officers, and risk management teams.

Pfost was also the VP of Product Development & Strategy at Admit One Security, and spent six years at Microsoft conducting audits, general corporate information security, and working in software program management roles.

“As a practitioner, I helped build metric programs at two F100 companies, and many more now as a consultant. I’m seeing more executives interested in understanding their security posture and how incidents may affect business objectives,” Pfost said.

“Compliance driven audits serve their purpose. However the best approach to answer the question of how much security do we need requires visibility into on-going control performance and a commitment to operate at acceptable levels.”

Pfost says that measuring a process is often perceived as being expensive, and business leaders and development teams have little patience for anything that requires time or resources for their delivery.

“It’s rewarding to me to demonstrate the benefits of a metrics program outweigh the costs. If an executive decides not to measure, I still fully support them because an explicit risk acceptance decision was made. Most teams make implicit acceptance decisions that may or may not align with their organization,” Pfost said.

“I really enjoy helping development and operations teams decide how much measurement is right for them. We often start off with basic measurements to demonstrate value and justify additional investment.”

Pfost says the BSidesSF session is geared towards several audiences, including executives who are curious if their security posture aligns with risk their organization’s tolerance, development and operations managers who want to deliver and receive credit for building resilient solutions, and for developers and engineers who want specific examples of how to measure performance, not just be told what they should do.

“Our goal is to motivate teams to improve or begin a metrics program. The audience will come away with actionable tasks, specific metrics, and real-world stories how to avoid some of the pitfalls we’ve encountered,” Pfost said. “My target-based metric is to provide 100% of attendees with actionable experience.”

Pfost says the most common pitfall occurs when the security team drives the mandate to measure security, as accountability and support for measuring performance must come from the business owner, and the security team is responsible to help facilitate, build, and support the process.

“The second landmine is a failure to define targets for metrics e.g. P1 security bugs fixed within X days,” Pfost noted. “Targets should be negotiated with the development and security teams, and they should be approved by the business owner. If leaders aren’t included in target definitions, the escalation path will likely fail when targets are missed i.e. a state of unacceptable security posture.”

He also points out that measurement programs are typically a reaction to a security incident, but says he is starting to see project teams add metrics proactively, before a project is approved or early in the release cycle, as bolting a metrics program on to a production system is always an uphill battle.

“For a hairy prediction, I think metrics will eventually be required as evidence that due care was taken to build a product or service,” Pfost said. “Legal liability may someday propel metrics to a top business requirement, and eventually the compliance regimes will also require target-based performance measurements.”

Metrics programs can be scary and threatening to some folks since they hold everyone accountable for performance. Some people fear the added visibility may slow them down or threaten their delivery.

“Executives have to ask themselves if they want to know, and managers have to ask themselves if they want to show and tell,” Pfost said. “The security team has a great opportunity to ask the questions and help drive the answers.”


Related Articles:



picThe Executive’s Guide to the Top 20 Critical Security Controls

Tripwire has compiled an e-book, titled The Executive’s Guide to the Top 20 Critical Security Controls: Key Takeaways and Improvement Opportunities, which is available for download [registration form required].


picDefinitive Guide to Attack Surface Analytics

Also: Pre-register today for a complimentary hardcopy or e-copy of the forthcoming Definitive Guide™ to Attack Surface Analytics. You will also gain access to exclusive, unpublished content as it becomes available.


Title image courtesy of ShutterStock