As time goes to infinity the stability of a computer system goes to zero.
I don’t know if anybody else has said that, but it’s a law in my life as a system administrator. As you install and remove software you change systems in subtle ways. These changes aren’t reproducible, they’re undocumented, and they are unpredictable. At the very least you end up with extra files littering your systems. At the worst your system becomes less stable. Now imagine what happens when you install a new operating system, the biggest of applications, over the top of an old one?
This leads me to a corollary:
Do not upgrade systems in place. Always rebuild them.
Upgrading a system just lets the rot live longer. By rebuilding a system, rather than upgrading in place:
– you start at a known state, free of the errors and bad practices of your predecessors. You can implement new policies, procedures, and tools without having to worry about retrofitting the old system.
– you start at a known state, free of the errors and bad practices of your vendors. You can follow your vendor’s recent installation documentation, rather than inheriting their old, bad practices.
– you ensure that the documentation for a system is current, and ensure that those responsible for the system still understand it.
– you can test the reinstallation procedure easily, without having to take an image of the server and restore it somewhere else. Unless you have identical hardware to clone to, cloning a server to test an upgrade may introduce other little problems. It certainly isn’t the most clean of tests.
– you can do the rebuild in parallel with the working system if you have new hardware. Sure, you could clone the system and do the upgrade in parallel that way, but what if you want to get a new server? Will you be able to reliably move the old system image to a new machine? What sort of hardware-level problems and incompatibilities are you dragging forward with you?
Another thing to consider is that vendors usually don’t test compatibility using upgraded operating systems. It isn’t worth their time. They test with fresh installs. When they say “Certified under Windows Server 2003” they don’t mean “Certified under Windows Server 2003, upgraded from Windows 2000, upgraded from Windows NT 4.” They are always speaking of fresh installs.
I’ve never had a completely successful in-place operating system upgrade. I’ve always had odd problems afterwards, things that would never happen on a fresh install that take me hours to fix. On the whole, in-place upgrades seem to take 150% more time to complete than fresh installs, and that’s a number I quote to my customers when they complain that a rebuild will take too long. Once I explain all of the points above they usually see it my way, especially when I explain that a rebuild is a good long term investment, if only for the documentation.
Save yourself and your customers time, money, and downtime, both in the short term and in the long run, by not doing in-place upgrades.
Update: Okay, so I was thinking about this topic all last night as I was doing a clean install of Microsoft Vista on my home PC. While that was happening my blog software, Omea, was offline, stored nicely on an external drive. Had it not been offline I would have read the Lifehacker post on how to make the Vista upgrade go smoothly if you can’t do a fresh install. This is the second time I’ve run into Lifehacker like this, I swear I’m not copying you! :-)
Comments on this entry are closed.
I cannot count the number of times an in place upgrade has created more problems than just going through the complete rebuild of the server in the first place. Fortunately, I now carry the same mentality of only performing rebuilds of production equipment. Dependencies these days are almost to the point of exponential in comparison to the days of yore.
Amen, brother. It’s almost impossible to manage dependencies on systems, like Microsoft OSes, where there is no package management.
In fact, that’s a whole other rant, why MS doesn’t use package managers for their own stuff. It’d make updates so much easier to sort out. Plus when they say “you need the Visual Studio C++ Runtime Libraries” you could fire up a little client and just auto-download the latest package.
Agreed. I’d say that the .NET framework makes up for DLL hell somewhat, but it still doesn’t fully correct the issues of the above said.