Over the last few months I’ve been cataloging the complaints I’ve heard about the deprecation of VMware ESX, in favor of ESXi. I’ve been running 100% ESXi since shortly after the vSphere 4.1 release. In the words of Samuel L. Jackson as Jules in Pulp Fiction, “well, allow me to retort!”
“I have software installed on the Console OS, and I need to keep doing that.”
ESX wasn’t really a Linux box, it was an appliance. Sure, in a lot of ways it looked like a Linux box, but it was missing a lot of useful packages, with no maintainable way to add them. Yes, you could copy the RPMs from Red Hat Enterprise Linux, but then you’d have software installed on the machine you’d have to patch manually. You also run the risk of conflicts, support issues, and general chaos. You were better off from the start if you defined it as ESX, not Linux, and considered the console OS a hypervisor delivery method. Nothing more.
The appliance-like nature of ESXi is way more pronounced than ESX was. I applaud them for reducing the surface area of their products, and also reducing the amount of time and effort needed to maintain software packages that are irrelevant to the hypervisor. When it comes down to it, by embracing the appliance-like nature of the product you open yourself to all the cool new possibilities, like disklessness or replacing traditional drives with SD cards, autoconfigured PXE-booted stateless hosts, central configuration, patching by just rebooting, etc.
“I run third-party software which requires the Console OS.”
VMware has shipped APIs for interacting with ESX & ESXi for many years, and certainly at least since the first release of ESXi in late 2007. If you’ve kept up with your service & support on those products you are probably entitled to a version of whatever you’re running that supports the APIs. Of course, then you have to upgrade…
“Everybody talking about how great ESXi is really means that ESXi 4.1 is great. We’re still at ESX 3.x and ESXi 3.5 isn’t as full-featured.”
Correct. ESXi 4.1 is the first release where there’s feature parity with old-school ESX, and to be honest it was the first ESXi I really, truly, was willing to run.
Personally, if I was still on 3.x I’d be looking at getting to 4.1 pretty soon, because you’re going to start falling off vendor support matrices. When VMware does another major release ESX 3.0 will be four major versions behind, and your VMs will probably be multiple virtual hardware revisions behind, too. Given my personal experience, it’s usually a lot easier to get to the current release from the last release, if only because support staff are usually more familiar with it, and it’s been tested more by developers.
“ESXi is great, blah blah blah, but our environment is really stable, and if it ain’t broke don’t fix it.”
Hey, it’s up to you. My opinion is that all software is broken at all times, having bugs and security holes and missing features. It’s just a question of whether you notice or care. As one of my local VMware guys, Rob, put it the other day, “you don’t have to upgrade; ESX will be supported for many years.”
As far as stability goes, VMware is one of those companies whose software releases work well with the “wait for the first service pack” approach. By an “Update 1” release all the major issues are usually found and worked out.
“ESXi doesn’t support traditional logging, like ESX did.”
KB 1016621 outlines how to configure remote syslog logging for all versions of ESXi. You’ll need to set up a remote syslog host first, though. Besides, good security practice is to keep logs somewhere other than the host they were generated on, so if someone breaks in you still have the logs even if the local copies were tampered with.
“I need the esxcfg-* commands to configure hosts.”
“I need to configure jumbo frames and that can’t be done from the GUI.”
“I need to have console access to maintain SSL certificates, etc.”
Three words: vSphere Management Assistant (vMA for short).
It’s a CentOS-based virtual appliance that IS a Linux box, and can be treated like one. It’s got the vSphere CLI installed which includes all the commands you’d normally want to run from the console. You just have to specify a server, username, and password now, or configure vi-fastpass.
If you don’t like the idea of a virtual appliance just build your own and install the vSphere CLI yourself. This is also a good way to solve the remote syslog host issue. You can also turn on “Remote Tech Support Mode (SSH)” as well (see below).
“I want to be able to SSH to the hosts.”
“I have headless machines in remote locations and SSH is a must.”
You can SSH into ESXi, and if you’re using 4.1 or newer it isn’t a hack or workaround. You can enable “Remote Tech Support Mode (SSH)” from the Security Profile section of the configuration tab. It will put a warning message in the Summary tab for that host, though, so you may not want to leave it on all the time. Goes without saying that it also is another vector for attack.
Frankly, while I used to SSH to my ESX hosts all the time I don’t miss it at all now with ESXi. I do everything I need to do through the vMA or the CLI.
An upgrade like this is also a good time to start using any iLO/BMC/DRAC out-of-band management features of your servers, both local and remote. Not only is it great for remote maintenance, you can also enable Distributed Power Management and the Standby/Wake functionality.
“I do chargeback with the ‘du’ command on the datastores themselves.”
Use Remote Tech Support Mode on one host, then. Or script it in PowerCLI. Or use the API. Or dump that data from vCenter using the export functions. Lots of options. Switching to an API-based approach has the advantage of exposing the provisioned/not-shared/used storage numbers more easily, if that’s of interest.
“All this ‘do everything with the CLI’ talk just means that passwords are stored in scripts now. We won’t do that.”
Actually, VMware has been working to improve this. The vi-fastpass functionality on the vMA lets you authenticate to ESXi hosts with either a stored & obfuscated set of credentials or, with 4.1 and later, your Active Directory credentials. William Lam has a good write up on it over at virtuallyGhetto.
Keep in mind that vi-fastpass isn’t part of the CLI, it’s a vMA-specific feature.
“I don’t have time to learn all this new stuff.”
Nobody does. There are good business cases to be made for doing it, though, like saving more money, not getting hacked, better DR & HA options, staying supported, etc.
Make a case, build a plan, schedule some time, get it done.