Skip to content ↓ | Skip to navigation ↓

There are many efforts to create meaningful security metrics, which is a worthy goal. After benchmarking over 1000 IT operations and security organizations in the past four years, I’ve formed some very strong conclusions and opinions, some of which goes against security common wisdom.Wikipedia monoculture - potato field

I’ve come to believe that in order to safeguard the production IT environment, information security requires standardization and documentation. It requires controls such as checklists, and continual control (and where possible, the reduction) of production variance.

This is a very good things, as it aligns information security very closely with some of the key objectives of release management, as defined by ITIL. (This means that information security should be viewed as value-adding, instead of shrill, hysterical paranoids, always in the way of getting real work done.)

I’ve found that effective information security should strive to reduce variation in the production environment as much as possible. This may contradict what some information security theorists recommend that voice concerns about monocultures, advocating the supposedly inherent safety that diversity provides.

In the real world, it is difficult to achieve sustainable information security through random diversity. If we use a standardization strategy, we can rely on monitoring and reduction of configuration variance. If we rely on a random diversity strategy, we must rely on luck and obscurity.

A Thought Experiment

To test this conjecture, consider the following thought experiment. Suppose we had to inherit one of two undesirable scenarios. In Scenario A, we would inherit 1,000 servers supporting a given business process, configured identically but insecurely. In Scenario B, we have 1,000 servers supporting that same business process, but each server is configured randomly, but 50 percent are configured securely. Which scenario should we choose?

Some information security practitioners will choose Scenario B. They may give many reasons, including that of monocultures. For example, in biological systems, increased homogeneity in crops results in increased risk of catastrophic crop failures[2]. These information security practitioners may conclude from the biology analogy that the risk of disease is similar to the risk of unpatched and insecure infrastructure, making randomness better than consistency.

Benefits Of Standardization: Information Security And IT Operations

On the other hand, let’s explore Scenario A, which every high-performing IT organization would choose instead. High-performers emphatically point out that when every configuration is identical, then:

• Our mean time to generate security fixes is lower because we have only one fix to generate.

• We have higher confidence in our fix because we have high configuration mastery of our approved configuration.

• Our mean time to test fixes is lower because we can build one testing environment that faithfully matches the configuration of the production environment.

• Our change success rate is significantly higher because our changes are tested in an environment where we have high configuration mastery.

• Our mean time to deploy fixes is likely much lower because we have a uniform production environment, even allowing us to use automated software distribution tools with high confidence.

The systems in Scenario A also have one very interesting attribute from an information security perspective. The fact that all servers are identical shows that the organization can keep systems in a defined state, as opposed to letting them drift apart over time.

Scenario A Is Orders Of Magnitude Better

Just how much more expensive in terms of time, effort, and cost is Scenario B over Scenario A? In this thought experiment, we may conclude that if the effort scales linearly with the number of configurations, then Scenario A would require 500 times less effort than Scenario B. That is an astonishing difference in effort (whether it is planned or unplanned), and shows how much more desirable Scenario A is when you have one well-understood configuration.

To achieve configuration standardization, we must integrate into the release management process. Production and information security checklists may exist, but manual performance of the tasks they specify can introduce configuration errors, either in the implementation or the verification phase. What’s more, releases may be deployed without error into the production environment, but undocumented or unauthorized changes implemented after release may cause configurations to drift from the approved builds. These configuration problems can cause subsequent problems in applying patches and changes.

(Some of this text adapted from the book, Visible Ops Security: Achieving Common Security and IT Operations Objectives in 4 Practical Steps, (written by myself, Paul Love and George Spafford).

PS: Follow me on Twitter: I’m @RealGeneKim!