The number of missing security patches in an OT system is typically very large—measured in the thousands, at least. It would be difficult and expensive for an asset owner to evaluate each missing security patch / cyber asset pair. This may be one reason we see a patch everything approach, but this is also difficult and expensive. In fact, assessments show this is rarely done even where required by policy.
A vulnerability management system can identify the missing security patches for each cyber asset. Equally or even more importantly, a vulnerability management system can help an asset owner automate the decision of what to patch when. While I’m partial to a decision tree approach, see ICS-Patch: What To Patch When In ICS, there are a number of approaches. The key is for the process to meet two criteria:
- A prioritized patching process is developed where patches are deployed at a time interval that's related to risk reduction of applying each patch.
- The decision is automated. Information is fed into a process, which helps to decide whether to patch ASAP, schedule the fix on the regular patching/maintenance interval, or defer it to when the software will be upgraded for other reasons.
In a perfect world, software and firmware would never have vulnerabilities. In a near perfect world, all software and firmware vulnerabilities would be patched as soon as possible. There is no doubt that applying security patches is a good security practice and does reduce risk.
The OT reality, though, is a very small number of security patches result in significant risk reduction, and the majority of security patches that could be applied in OT result in a tiny, near-zero risk reduction. Therefore ,the key to effective OT vulnerability management is to match the speed and frequency of security patching to the risk reduction achieved with the patch.
A risk-based security patching approach as part of vulnerability management is still rare. Many asset owners determine a security patching cadence, monthly, quarterly, semi-annually, etc., and they try to apply all needed security patches in this interval. Again, this is not wrong in the sense that it is a bad practice. It is wrong from a risk management perspective because resources are not being applied to maximize risk reduction.
High Priority / Patch ASAP
One of the first steps of any OT security program is to deploy a security perimeter between OT and IT, typically in the form of firewalls and secure remote access appliances. If there is a vulnerability in the firewall or remote access solution, it needs to be patched immediately because the vulnerability can eliminate these perimeters and allow attackers on IT unfettered access to OT. In many cases, you will want to disconnect these security devices and break IT to OT communications until they are patched.
This is not a hypothetical issue. Security products are prime targets for attacks because of the value of a vulnerability to an attacker. For example, in the past two years, there have been multiple vulnerabilities in Pulse Connect Secure and Fortinet Firewalls, both widely used in OT.
Similarly, any security controls related to scanning removable media and transient laptops prior to use in OT would be in the High Priority / Patch ASAP category. Cyber assets such as USB scanning stations are actually another form of security perimeter – the perimeter to address physical devices walked around the cyber security perimeter.
Finally, any cyber asset directly accessible from the IT network should be in the High Priority / Patch ASAP category. If your architecture follows good security practice, this would be primarily cyber assets in an OT DMZ. Many asset owners do allow some direct IT-to-OT communication, and this needs to be patched ASAP because an attacker on IT is only one step away from achieving a foothold in OT.
Low Priority / Defer / Patch To Maintain On A Supported Version
Most items in this category are “insecure by design.” This means almost everything an attacker would want to do is possible without needing to find and exploit a vulnerability. If the attacker can access the cyber asset, the attacker can do what they please using documented features that lack authentication. (This is another reason why the OT security perimeter is so important.)
The most common OT cyber assets in this category are PLCs, Controllers, and other Purdue Model Level 1 devices. While there are some newer models with security features, 99%+ of these devices are insecure by design. If an attacker wants to turn up the heat, make something spin faster, or issue another control command, all they need to do is send the appropriate write request.
If an attacker wants to change the logic or program, they just need to upload it because it usually does not require authentication. What an attacker can do is limited by the engineering and automation skills, not their hacking ability. Many PLCs even do not require authentication to upload of new firmware.
Very little risk reduction is achieved in patching a vulnerability in an insecure by design device. Imagine a PLC has a vulnerability in its web interface or a vulnerability in its ftp service. Applying the patch would not slow down or distract the attacker. In fact, it is more work – and unnecessary at that – for an attacker to identify the device that has the vulnerability, exploit the flaw, and then leverage the bug.
Everything In Between
High- and low-priority cyber assets for vulnerability management are easily identified. It’s a harder decision on what to do with the rest of the cyber assets in OT. On one hand, if the attacker is on the OT network, they can leverage the insecure by design devices without exploiting any other systems. On the other hand, many OT computers and devices can be hardened to be made more difficult to compromise.
There are a number of factors to consider beyond the Exposure factor (leading to the ASAP cadence) and Insecure By Design (a rated Security Posture) factor (leading to the Defer cadence). I recommend considering:
- Safety Impact of a loss of availability or integrity of the cyber asset
- Process Impact of a loss of availability or integrity of the cyber asset
- Technical Impact of the vulnerability (which is typically a CVSS or other score)
Automating The Decision
All of these factors, and many others that you might consider, are typically static, unchanging over time. For example, a cyber assets safety impact, process impact, and exposure rarely change. Adding these static factors to an asset inventory that supports the fields or the addition of custom fields is a one time event with a periodic review. If you are just starting your asset inventory, they can be additional fields entered with each cyber asset.
The vulnerability management module in your asset inventory, and some scripting or other simple decision tree can then take as input the thousands of cyber asset/missing patch pairs and output them in the categories you have assigned. You then can get to patching each category in the agreed upon cadence.
Security patching moves from a process that's time intensive and separate from risk management to a process that applies time and other resources in line with a risk management approach. You will apply the security patches that provide significant risk reduction much faster and those that don’t much slower than you would with a one-size fits all security patching strategy.
About the Author: For over 20 years, Dale Peterson has been on the leading/bleeding edge, helping security-conscious asset owners effectively and efficiently manage risk to their critical assets. He has pioneered numerous ICS security tools and techniques such as the first intrusion detection signatures for ICS that are now in every commercial product. In 2007, Dale created the S4 Events to showcase the best offensive and defensive work in ICS security and to build a community. S4 is now the largest and most advanced ICS event in the world. Dale is constantly pushing and prodding the ICS community to move faster and get better.
Editor’s Note: The opinions expressed in this guest author article are solely those of the contributor, and do not necessarily reflect those of Tripwire, Inc.