Earlier this week, I was working with my friend Mike Poor from Intelguardians on an upcoming webinar that we’re doing together on understanding virtualization security risks, and practical steps any organization can take to mitigate them. As we were capturing our screenshots on a semi-production ESX server, I had several a-ha (or maybe it’s holy crap) moments.
It started as we were talking about the VM admin privileges, and observed the sometimes hilarious nature of privileged access…
- VM Admins have unprecedented amounts of privileged access
- When I started to bug our ESX owner too much for stuff, he put me into a role called VM Power User
- Even VM Power Users have a scary amount of privilege. In fact, I was doing a VM move of an 80GB image, and I kept on getting “Data I/O error” from the VI Client. I logged onto the Service Console, and started to try using SCP or FTP, and kept on trying until the entire ESX server crashed (no pings, no VI client access, nothing). My first reaction was, “Oh crap. I crashed it. They should never have given me this much access!” In reality, however, it was the ESX owner who accidentally messed up a vmKernel port group, and it’s been down since. Which leads me to…
- VM servers can be fragile and prone to errors, which can cause the entire virtualized computing environment to go down, even when run by skilled engineers and administrators. This can be exacerbated when there are “many fingers in the pot,” all working concurrently.
But, as Mike and I shifted our focus from the operational risks to the information security risks, it really hit me that…
- VM Admins really do have a breathtaking amounts of privileged access: consider the number of critical IT services that contain confidential personally identifiable information (PII), and the risk of an unscrupulous admin copying the entire VM datastore. (Forget about backup tapes, which are difficult and cumbersome to carry and hide!)
- This risk also extends to the VM Power Users like me, who was granted this status because I was too much of a pain in the a**for the ESX owner
- From a security oversight perspective, this underscores the need for effective control of access: we must ensure that all privileged VM admins (and powerful accounts) are reconciled, mapped to an authorized staff member and an approval by the appropriate manager.
- And the above, of course, underscores the need to control all the configurations that control access. (See my previous post “If A Virtualization Misconfiguration Or Security Vulnerability Exists Within An “ESX Appliance,” Does It Really Exist?”)
- And of course, because trust is not a control, we must ensure that security is aware of all adds, removes and changes for these classes of accounts.
As Mike put it, virtualization does wonders to solve the IT asset management problem, but creates some huge nightmares for the data containment problem. We both agreed that when dealing with sensitive systems and data, partitioning data to separate ESX clusters with no shared access makes a lot of sense.
“Eh, what’s this? What could go wrong?”
And don’t forget to register the webinar Mike and me present on how to mitigate virtualization security risks here!