The Phoenix Project was an easy and enjoyable read about Bill Palmer, a manager in the IT department who unexpectedly gets promoted to VP of IT Operations. To succeed in this new role, Bill had to expand his view from just his group to the organization as a whole in order to master the “Three Ways” for how to evolve from a dysfunctional group of departments to an integrated DevOps team.
While Bill navigated around innumerable Severity 1 incidents, one involved a security-related change where the implementation was untested prior to deployment. The event ended up affecting a critical business system, causing a substantial amount of unplanned work. This change, among a few other reasons I won’t spoil, leads into a confrontation involving the security team.
This was a pretty mundane event in the book itself, but it touched on a very important concept around figuring out the best way to implementing security measures while minimizing risk to the business.
Security is becoming more important as every day passes, but security could also end up as a double-edge sword if not implemented right. It’s important to understand how each environment works, including but not limited to inter-asset communication, compliance needs, and/or any legacy/proprietary devices that have specific requirements. For example, when scoping a PCI environment, understanding what brings an asset into compliance is crucial.
If the PCI environment consists of only retail stores, but a security tool with a centralized console is implemented at the corporate office with communication to the retail stores, it’s possible the PCI scope was expanded to include corporate servers depending on how the console communicates to the PCI environments. We’ll call the potential “unplanned,” or unexpected work from implementing a security measure. That’s the risk of security.
To account for the risk of security, we’ll need to not only understand why a security measure is needed but also how it’s achieved. What is the worst case scenario if the security measure backfires? Will a failure cause a loss of visibility into the environment or something more severe like taking down a business critical resource?
I’ve spoken with someone that had an automated patching system in response to detected vulnerabilities that worked great for the majority of time, but it also caused a substantial amount of unplanned work when an automated patch started impacting legitimate traffic to one of their sites.
If failure can cause production issues that impact the business, it may be worth asking the following question: is there another way we can achieve the same result with less risk?
The risk also fluctuates depending the type of environment. The CIS top 20 security controls ranked the inventory of hardware/software assets as the most critical controls. To achieve this inventory in a corporate IT environment, an automated discovery tool is often used. Such a solution scans the network to find what’s out there.
However, if we did this same scan in an industrial manufacturing network, there could be some very real consequences of impacting the production line by scanning a fragile device that becomes out of sync with the rest of the line.
In conclusion, when considering putting a security measure in place, understanding how it accomplishes the need is just as important as why it’s needed. Ignoring the risk of implementing security could result in unintended consequences that can be minimized or avoided all together. I also recommend taking a little bit of time out of your day to enjoy the Phoenix Project and learn all about the Three Ways.
If you are interested in learning more about how the future of change management and how DevOps and security teams are now actively collaborating as peers, then please attend this webcast with the author of the Phoenix Project, Gene Kim and Tim Erlin of Tripwire.
In the webcast, speakers will discuss case studies that demonstrate how DevOps succeeds in large, complex organizations, such as General Electric, Raytheon, Capital One, Disney and Nordstrom. In almost every industry, organizations are replicating the same groundbreaking approach and are succeeding.