Software Monitoring for NERC CIP Compliance: Part 1

Image

As organizations grappled with NERC CIP version 5, Tripwire learned along the way. In this series, I’ll cover the aspect of CIP that has come up the most in the last year: how to meet the software monitoring requirements.

Software Inventory as a Security Control

It is a simple question at first, but the more we peel back the layers, the more we see what a complicated topic it really is. This discussion may help some electric utilities with their ongoing efforts and provide ideas for companies not regulated by NERC about monitoring their software inventory as a security control. In the first of this two-part series, we will walk through the background of this topic to illuminate the software inventory challenges you’re facing. Part two takes what we’ve learned and presents practical and auditable approaches for success. For readers who do not focus on NERC CIP, I still encourage you to keep reading. Being able to identify the possible actors (software) on your system and their level of patching has definite security value. Indeed, software inventory is the CIS #2 control. You have the luxury of implementing controls with a lighter-weight process, no burden of audit and the lessons learned from our electric utility colleagues. Take the fire and leave the ashes!

A Quick NERC CIP Standards Summary

The North American Electric Reliability Council (NERC) is responsible for setting various standards that apply to companies generating or transmitting electricity above certain levels. The standards are intended to help ensure reliability of the electric grid. Only one of the standards, Critical Infrastructure Protection (CIP), relates to cyber assets. The CIP standard is further broken down into sub-areas also referred to as standards. Standard 10 (CIP-010) contains requirements for configuration management around software, specifically those found in its first requirement. Thus: NERC -> CIP -> CIP-010 -> Requirement 1 > Part 1.1.1 through 1.1.5. Also, keep in mind that the CIP standards were written by lawyers, for lawyers, with engineers caught in the middle. The data generated for showing compliance could literally end up in civil court if evidence of compliance seems inadequate. The country is divided into geographic regions for audit, a reality which can lead to some region-by-region variance in how the requirements are interpreted.

What Is “Software,” and What’s CIP Requiring, Exactly?

Within CIP-010 R1, there are sub-requirements listing what is needed regarding software monitoring. Additional information is provided in the Guidelines and Technical basis section at the end of the standard. “Software” boils down to understanding these terms:

• OS and firmware version
• Intentionally installed software
• Custom software
• Security patches

Next, I’ll define these software categories briefly. If you know the NERC CIP-010 standard pretty well, feel free to skip ahead. The section on custom software is probably worth a look, though.

What Is a Baseline?

A “baseline” is a Tripwire Enterprise concept as well as a term in the CIP standard, but they do not mean the same thing! In a Tripwire Enterprise context, “baseline” refers to a particular version (out of all observed versions of a file) that is considered the trusted or preferred version. Any new changes are compared to this baseline version, and a new version of a file can be identified as the new trusted version (aka, baseline) through a built-in workflow. In a NERC CIP context, baseline refers to a standard of configuration. It's more like a platonic form or ideal configuration. This concept of baseline maps best to Tripwire Enterprise policy and test features, not the “change and promote” concept described above. Most customers initially assume the “review change and promote” process will help with the CIP change management requirements. Both approaches are typically used by entities to one degree or another. The “change and promote” approach is much more granular and not as scalable, whereas the approach with policy tests allows for entities to manage compliance in a much more streamlined and scalable fashion.

Interpreting NERC CIP’s Firmware and OS Requirements

Firmware

The OS and firmware version requirements are the most straightforward to understand. “OS version” refers literally to the version of the operating system. “Firmware version” is usually read as pertaining to switches, routers and other networked devices lacking a traditional OS. Proprietary systems that run a traditional OS (like Linux) but which cannot have a Tripwire Enterprise agent installed are usually treated like a network device — i.e. we connect by SSH or telnet instead of installing an agent. BIOS versions can be included in scope, but I have not seen any entities actually implement such monitoring for NERC CIP.

Intentionally-Installed Software

Intentionally installed software is software that’s needed on the asset to support its bulk electric system critical function and that has been installed onto the OS. This software is part of the declared (CIP) baseline, and it’s necessary for the role of the asset. Presumably, unintentionally-installed software needs to be reviewed and then either approved or removed.

Confusion Over Custom Software

Custom software is the most interesting. Over the past year, I’ve seen entities across audit regions struggle with interpreting not only the language of the standard but also the Guidelines and Technical Basis for this standard. The Western Electricity Coordination Council, or WECC (the geographic audit region in the U.S. West) made a public presentation wherein the advisement was to stick to the strict language of the standard (which does not mention scripts) and ignore the Guidelines and Technical Basis (which does mention the scope “may include scripts”). WECC can only speak for their region, of course, but it is a helpful data point in how that interpretation may have consideration in other regions.

What Qualifies as a Script to NERC CIP Auditors?

Prior to this advisement from WECC, there was considerable angst among entities as they struggled with determining what qualifies as a script and how to monitor systems for scripts in a scalable and repeatable manner. Some common example questions that arose included:

• Should we only monitor files with executable permissions on Linux?
• Do Excel Spreadsheets with macros count as scripts?
• Do we need to scan whole systems for executables, or can we meet the requirement with a partial scan?

The rabbit hole quickly became the Grand Canyon! The Technical Basis and Guidelines section of CIP-010-2 says “custom” software may include scripts which every entity interprets as “definitely include scripts.” But then what? I have seen an entity or two define the scope of custom software as any executable on the system. I have yet to see specific feedback from an audit on that point, so it is questionable in my mind whether the scope has to be so broad. The review process would certainly be quite burdensome and heavy on staff time to maintain! What I have seen more widely adopted is along the lines of “crawl, walk, run” — do something, perhaps modest, now and then adjust up or down later as audit feedback becomes available.

Where Do Patches Fit In?

Finally: Patches. Patches refer to the hotfixes on Windows or analogous updates released for other traditional OSes. Tracking the firmware version of network devices is the security equivalent of tracking patches from a CIP perspective. This facet of the CIP software scope is straightforward to define but trickier to report on, as we shall see. The fun part is that not all patches are security related, and they may not be distinguished as such by the OS vendor. “Here’s a bunch of patches, go have fun!”

NERC CIP Lessons Learned

You don’t want to be the best or the worst at CIP audit. If you are the best, then perhaps scarce resources were over-dedicated. If you are the worst, your organization will stick out in the mind of the auditor as they travel from site to site and there could be a higher risk of a finding. In Part 2, I will describe approaches that have emerged over the years in collaboration with our NERC customers — NERC customers telling Tripwire what they need, Tripwire filling in with best practices and sometimes even product enhancements arising from discussions of the art of what is possible. Learn more about how Tripwire technology can help you achieve ongoing, audit-ready NERC CIP compliance by reading our case study on how Western Farmers Electric Cooperative guards their systems with Tripwire.