If It Ain't Broke Then Today Is Not The Day To Fix It

“What are you up to today?” a friend asks. It’s the Friday of the week before Christmas, and everybody is taking most of the next week off.

“Not much, just doing some paperwork and writing stuff. I can’t really start anything — everything on my to-do list runs the risk of breaking something.”

“Everything? There’s got to be some small stuff you can do.”

“Yeah, but the small stuff is on large, complex systems. You ever have a fish tank?”

“Nope, just cats.”

“Fish tanks have this thing called the nitrogen cycle. You feed the fish, they crap, bacteria eats the crap, eventually turns it into nitrates, and plants in the tank fertilize themselves with it. Fish eat the plants, cycle starts over.”

“Sure,” he says, wondering where I’m going with this.

“A mature tank is balanced, and every part of the cycle is matched to the others. There’s enough bacteria to handle the waste, there are enough plants to even things out, etc. But then someone comes along and makes a small change, like cleaning the filter. Seems harmless, but that’s where a lot of the bacteria live, and things can get out of whack for a while.”

“Won’t it fix itself when the bacteria grows back?”

“Yeah, but that’s not my point. Computer systems are like fish tanks, and small changes to a stable system might cause big problems later. Like when nobody is around over the holidays to notice that all hell is breaking loose. You’re sitting in church late on Christmas Eve and you get a page because that little change you made botched the log rotation on your big web servers and now they have full filesystems and are all down. Nice shot.”

“So you don’t do any actual work all day?”

“I write a lot of documentation on days like today.”

9 thoughts on “If It Ain't Broke Then Today Is Not The Day To Fix It”

  1. Yup, good analogy. It’s always a good idea to implement a change freeze in IT before Christmas and usually not end it until the 1st week of January when people come back to work – nothing sucks more than working on Christmas Day or New Year’s Eve just because someone decided they couldn’t wait a couple weeks to do something… now is the time for documentation – you know, that stuff you’ve probably neglected thus far. 😉

  2. Bob, I absolutely agree with this! In fact, we make it a policy to never make changes on ANY Friday or day before a holiday for this very reason.

Comments are closed.