Why I Like Virtualization (And Why Hardware Sucks)

I was asked why I like virtualization and why chroot jails aren’t a better way to do things, at least on UNIX-like OSes. To figure out why I like virtualization, let’s start with what I don’t like about hardware:

• Failures. Something is always going wrong, whether it’s a fan, disk, power supply, etc. More servers means more failures. We buy warranties to help with this, but warranties cost money. It takes time to deal with failed components, too.
• Firmware. It is hard to update firmware levels. Every device is different, and a bunch of update methods end up requiring you to go out to the box with a USB stick or a floppy disk. That takes a lot of time, usually at times of the day I’d rather be somewhere else (like sleeping).
• Cables. I hate cabling. It costs a lot of money, a foot at a time. It gets tangled. It gets unplugged. It gets mislabeled. It takes a lot of time and vigilance to do right.
• KVM. My KVM system is IP based. It uses Cat5 cable and has USB dongles that attach to the hosts. It costs a ton of money. The dongles have strange incompatibilities which make it an adventure to connect a server. It’s also another cable to manage, another system to maintain, another drain on your time.
• Racks. Racking a server means I have to go to my data center, which may not be in the same building or city as me. I have to worry about available rack space, power in the rack, and cooling. I have to worry about two post or four post racks, which type of holes in the rack, and whether I have the right screws. Racks cost a lot of money, too.
• Power. I hate power cords. They get tangled, messed up, unplugged. We order short power cords to help with that, but those cost money. To keep things running we have a UPS. UPSes are expensive and require maintenance. Speaking of money, I hate paying power bills, too.
• Cooling. Cooling requires equipment, which in turn requires maintenance. It also requires power. Maintenance and power require money. Did I mention that I don’t like giving other people money? I want to keep it and buy myself cool things. :-)

It basically comes down to money and time. Time is really money, though, because to get more time you have to hire someone to help.

So how do you reduce these costs? Simple: have less hardware. Why can’t we have less hardware?

• Applications aren’t good at sharing. They require specific versions of other software, different from what the other applications require. For example, an application written in Perl might require DBD::Oracle 1.14, another 1.19. Now I need two different copies of it and it isn’t simple anymore, especially if the applications assume that DBD::Oracle will be installed in /usr/lib/perl5.
• It can be hard to figure out performance problems on a machine that is doing a lot of different things.
• It is hard to tune a machine for performance with multiple different applications. Do I tune for Apache or MySQL?
• Customers have wildly different security requirements.
• Customers have wildly different maintenance window needs.
• Customers want to build clusters. How do you share a cluster with an application that isn’t clustered?
• Customers just don’t want to share. They don’t want anything to do with another project or customer. They want their own machines and want to have their way with them. Coordination between customers is sometimes impossible.
• Customers want separate development, test, QA, and production environments. They want to be able to do load testing and other crazy things without impacting or being impacted by someone else’s software. Unfortunately, development, test, and QA environments will sit mostly idle over their lifespans.

Approaches like chroot jails are great for single, well-known applications. The more applications you need to run in a chroot jail the harder it gets to maintain, though. You end up needing second, third, fourth copies of libraries, binaries, etc., especially if you’re trying to chroot interactive users. Automatic patching processes like yum, up2date, etc. don’t update your jails so you have to do that manually. The jail doesn’t address things like differing maintenance windows, or any of the non-technical problems of sharing a machine. This approach may have performance benefits, but to be honest performance is usually not a big factor.

Most of the problems in sharing a machine are actually problems with sharing the operating system. Virtualization decouples the hardware from the operating system, and because of that we can solve all the problems of sharing a machine by choosing not to share at all. No chroot jails to maintain, no worries about versions of software, no endless coordination meetings just to schedule a reboot of the server.

So why do I like virtualization? It lets me get rid of hardware but doesn’t force me to manage complex situations that arise from a shared OS. For the operating system it is business as usual, which means relative simplicity and well-understood processes. I like that.