Earlier this week, I read a blog entry by the Lone Sysadmin about our recently released Tripwire ConfigCheck. He noted that it found some misconfigurations in his VMware ESX server, and then goes on to ask a very interesting question:
Now my question is: is ESX 3.5 an appliance or a host OS? Do I actually want to make the recommended changes? Will it mess up something in the future when a patch from VMware assumes something about my environment that isn’t true because I’ve changed it? Exactly how much do I want to go messing around with things like NTP settings when the recommended way to configure NTP is through VirtualCenter?
I look forward to a time when ESX 3i is on par with ESX 3.5, but in the interim do I change things to gain a little security and run the risk of problems later? Is ESX a Linux distribution or is it an appliance?
I am intrigued by this question, because while it may sound philosophical (e.g., “When a tree falls in a lonely forest, and no animal is near by to hear it, does it make a sound?” ), this is actually a fundamental question about IT controls scoping. You must answer this question first before asking who is responsible for them (the question that esteemed Chris Hoff asks on whether it’s VM admin, security, etc…). Luckily, this has a very precise and logically rigorous answer that everyone in IT management (IT operations and security) should be able to answer.
Q: Do any of the VMs running on the virtual infrastructure provide any critical IT functionality required for IT operations, security or compliance?
A: Most likely yes, if we have any production services running on the VMs, or if there are confidentiality or regulatory requirements for the systems or data.
Q: Does VMware ESX rely on critical functionality provided by underlying host OS?
A: Yes. The correct operation of the VMware ESX server (or virtually any virtualization manager) almost entirely depends on the correct functioning of the underlying OS, including the kernel functionality and the userland utilities. One of the most important functionality by the host OS is the privileged accounts (e.g., root) and file ownerships to restrict access to VM systems and data.
VMware published what these configuration settings are in the “VMware Security Hardening Best Practices” document.
Q: Can you bypass the application controls in the management applications (e.g., VirtualCenter, VI Client)?
A: Yes. Although VMware strongly recommends that you use their management tools for managing their virtual infrastructure, in ESX, many use the Service Console to do work that can’t be done in the management tools, such as automation or hairy break/fix work. But, consider the implications of hitting ‘Enter’ in the screen below…
This would make all the VMware ESX datastores readable, allowing anyone to access the ESX datastore to copy all the VM systems and data. (Why bother hacking the VMs when you can just steal the whole thing?)
Configuring this is so important that it is one of the automated test that ConfigCheck does.
If the answer to the above questions are all “yes,” then we can say conclusively that the services being run on the ESX Server rely on not only the VMware ESX functionality, but also the correctness of the configurations of the underlying host OS.
In other words, it is not enough to call the host OS an “appliance,” and remove them from IT management’s area of responsibility. Misconfigurations and security vulnerabilities create risk, regardless of whether you call the ESX Server a “server” or “appliance.”