When can I get bailed out?  I’ve made several bad decisions in my life and I’d like to be compensated for them. And by “compensated” I mean “massively compensated, like dump trucks full of cash.” After all, why should I be responsible for my own actions and decisions?

If I were Honda or Toyota I’d be getting my lawyers ready to sue a lot of people.

Question/Issue:
Is ESX server VMotion supported with SAV and SEP?

Solution:
Symantec does not support ESX server VMotion at this time.

Vendors are shameless. They charge you a ton for support, then they’ll do whatever they can to point the finger at somebody else when you call.  It’s one thing to put a disclaimer in for performance issues. Virtualization sometimes exposes weird performance issues, and if it’s a performance issue you’re having you might need to do some work to troubleshoot it on your own if it isn’t a blatant, completely reproduceable problem with the vendor’s software (like CPU-sucking spin locks).

It’s another thing to say that their software isn’t supported at all, or to say that a problem must be reproduced on physical hardware. Most of the problems I’ve ever called a vendor about are explicit functionality problems (bugs).  Physical hardware, virtual hardware, it doesn’t matter: their software just doesn’t do what it’s supposed to.  A vendor’s support staff should be competent enough, and professional enough, to sort out a bug report from a performance problem and act accordingly.

Oracle doesn’t support anything in VMs (as per an Oracle employee last week to me). Lyris doesn’t support anything in VMs (last time I checked, a few months ago). Symantec doesn’t support anything in VMs (technically they said VMotion, but for an enterprise VMotion goes hand in hand with VMs). What other vendors are as clueless as these three? I think I’m going to start making a list.

I’ve been having the strangest problems with a virtual machine I’m trying to deploy. It boots but won’t come up properly on the network. Services will start but complain about the network, or just be unresponsive. I can’t ping it, either. I’ve deployed several other virtual machines today from this same image, so it isn’t the image. Regardless, I redeployed it. Still messed up. I double-checked the network settings, /etc/hosts, /etc/resolv.conf, gateway devices, netstat, route, everything. Nothing is wrong. I changed the IP address to something else, and it works great. I checked with my NOC to see if the IP I’d been using is firewalled, blackholed, or otherwise administratively unusable. Nope. I switch back, and it goes back to failing. OMFGWTFBBQIAMSOFRUSTRATEDWTF.

Turns out my hostmaster had set the A record to 192.168.77.74, rather than 192.168.74.74. Not surprisingly, a lot of stuff seems to care about that. The IP looked right, though, so I didn’t notice it until after a few hours. A few hours of my life I’ll never get back, that is.

“Excuse me?”

“What are you looking at in Wikipedia?”

“The article on X-Men.”

“Tough day at work, I suppose.”

“Um, I’m trying to figure out a naming scheme for the 10 new servers I’m bringing in. That okay with you?”

“Oh, sorry.”

Just because you think I’m not doing work doesn’t mean you’re right.

(also, great site for naming schemes: namingschemes.com)

To hell with cloud computing. Clouds are puffy crap that float lazily by. Is that what you want out of your service provider? Just floating by without a care in the world?

It is time for tornado computing. Or hurricane computing. Real wrath of God type stuff. I want an architecture that knocks me off my feet, whips my apps around and hurls them half way through a tree. I don’t want my data intact for some script kiddie to steal. I want it like a frog in a blender; unrecognizably processed with a taste only I care for.

So to that end I am setting half of my air handlers to “Freakin’ Steaming,” the other half to “Ice Storm,” and locking the doors until the screaming stops. By this time tomorrow you should have some form of cloud computing in the data center, maybe a squall somewhere over the mainframe if you’re lucky. Viva La Revolucion!

Interestingly enough, that pretty much sums up my feelings, too. Service providers don’t seem to address the DR, legal, privacy, and security concerns that corporations have, don’t seem to care, and even go so far as a Microsoft rep telling a coworker of mine that “it’s no big deal as every bit of information about you is practically out there already.” Given that sort of attitude how can I do anything but build my own cloud?

If you can’t figure out why perhaps you should find a different profession.

Seriously.

I’m fine if you don’t keep backups because you’ve thought about it and you are taking a calculated risk. However, having to explain why backups are valuable to someone who, until this moment, I considered a peer is ridiculous.

It’s like having to explain what DNS does to someone who calls themselves a network administrator. I’ve done that, too.

Because my officemate has started to refer to it as “the second coming of Jesus.”

I wrote the new phone a haiku:

Dear 3G iPhone
Wish you were here already
So people shut it.

Not that I’m tired of hearing about it. Nope.

I hear it will have infinite battery life. And it will read your mind. And it will have 1 TB of storage. And it will find your keys when you lose them. And it will feed your cats. I can has 3G iPhone.

Arggh.

I’ve always been a huge fan of tip #6 in “The Pragmatic Programmer:” Don’t Repeat Yourself. As soon as you repeat yourself you risk the different copies getting out of sync, which causes problems and confusion. Putting a fully-qualified domain name (FQDN) in /etc/hosts as well as in DNS means that at some point later in life the two will be out of sync.

“It’s only on a couple of hosts, for testing.”

First, if it’s on more than one host you are repeating yourself. Second, testing is fine but now it’s another thing to remember to fix when you roll into production. It also means that production will potentially be different than your test environment.

“We don’t need to put this in DNS.”

Why not? DNS is a database built to solve the host name to IP mapping problem and it’s good at it. Perhaps you have more of a political problem with whoever runs your DNS.

“You have automated tools that can maintain synchronized copies of /etc/hosts on multiple machines, so what’s the big deal?”

Just because we can doesn’t mean we should. Plus, you’re still repeating yourself. The machines can still get out of sync with each other and/or with DNS.

“If we don’t put this in DNS then it won’t get hacked.”

I’m pretty sure hackers know how to use IP addresses. This is security through obscurity, which doesn’t work.

“Well, they won’t know that the server is our database/app/web server if they can’t resolve the name.”

A quality port scanner can often tell what services are on what ports, even if you are running services on non-standard ports. In short, if you are relying on the lack of DNS to prevent hacking you’re in trouble.

If you really want you can use ACLs in BIND to restrict who can query certain DNS zones.

“I want entries in /etc/hosts for performance reasons.”

A caching name server on the host may also increase performance and still get its information from DNS, which does not violate the don’t-repeat-yourself clause.

“I want entries in /etc/hosts for reliability reasons.”

Again, perhaps a caching name server locally would fix the problem. And if you have unreliable DNS you are probably having other problems, too. Perhaps you should fix that.

“DNS is tricky to administer, and the files are simpler.”

Maybe, but if you have a service, application, or system that needs DNS entries you should probably figure it out. Eventually you’ll have to know something about DNS. Editing files is simple until you get more than one server, and then the effort to keep the hosts files synchronized is usually better spent keeping DNS up to date.

“We define host names to have different IPs using /etc/hosts, which is how we do load balancing.”

You can do the same thing with round-robin DNS entries.

“We define host names to have different IPs, based on the functionality the server needs. So ‘database’ is 192.168.10.20 if it only reads and 192.168.10.21 if it needs to write.”

That sounds very confusing. Perhaps you could just register ‘database-read’ and ‘database-write’ in DNS and teach your app which one to use?

“Hosts files are a proven, reliable technology.”

*sigh* So is DNS…

• First, I hate voice mail. Email me instead.
• You don’t have to tell me what time it is. Voice mail is time-stamped, and it usually doesn’t matter that much.
• You do need to say who you are, because voice mail doesn’t record that. Do this in your first sentence. If you are with a vendor you should say that, too, especially if I’m waiting for a call from you.
• Please use your full name. You might be one of my closest friends but sometimes phones make people sound weird, cell phones cut out, and background noise sometimes makes it hard to figure out which “Bob” you are.
• If you called to have a conversation with me just tell me to call you back. A conversation is where two people talk to each other. My voice mail is not me, so you’re just talking. Talking != a conversation.
• Tell me why you called, using one sentence or less. Extra points if the whole message is a sentence or less.
• Tell me where you want me to call you back. Don’t assume my phone or voicemail has a log of your missed call, though if you’re sure I have your number it’s fine to tell me to call your cell, etc.
• Don’t leave me a message saying only that you’re going to try calling me somewhere else. Not useful.
• If the message includes an address or a phone number say it twice, slowly, so I can write it down. The first time you say the phone number I’ll be scrambling for a notepad, and replaying the message just for a missed number sucks.

That’s about it. *Seems* simple. Thanks for listening. :-)

What did I find as I proceeded? It wanted to install the Google Toolbar. Did I have the Google Toolbar already installed? No. So why is the default action to install it, unless I opt out?

Apple’s taken some heat lately for their decision to push Safari to anybody who runs their Apple Software Update utility. I didn’t want Safari, but unless I opt out of it I’ll get it. Now Sun and Google are doing the same thing with the Google Toolbar. Users know that if they don’t update their software they’ll get hacked, and Sun, Google, and Apple abuse that by pushing unwanted applications (Safari) and spyware (Google Toolbar) to systems that didn’t already have it. It isn’t enough that they allow you to opt-out. They do this knowing that most users aren’t going to opt out, either because they don’t know any better or because they miss the option.

If there was ever a reason for a law to be passed, this is it.

———————-

[0] Ever-moronic because the updater seems to like starting two, three, or four copies of itself, all sitting in my system tray. Maybe they think that I’ll be more inclined to update if the updater takes over the entire bottom of my screen.

I was going to say more[1] but isn’t there some rule that if you can’t say something nice don’t say anything at all?

————

[0] I didn’t want to link to it, but I couldn’t see a way around it.

[1] Those that know me should feel free to speculate. :-)

Microsoft has added Silverlight to Windows Update, but in contrast, it’s an optional update. Microsoft 1, Apple 0.

Apple, it’s called “opt-in.” Check it out sometime.

1. Typos are errors. Errors are things you didn’t want to happen. I argue that if you intend to misspell something it cannot be a typo, for it is not an error.

2. Misspellings make you look stupid. Taking a few seconds to check your work with a spell checker helps you avoid looking stupid.

3. The attention you will get from misspellings and misquotes will be from people that want to point out the mistake. Don’t confuse comments pointing out your mistakes for actual conversation. It isn’t. Whatever point you were making is lost on those people, because they already judged you by your mistakes, not by your content.

4. Though the strategy works okay for Will Farrell, I contend that if the only way to get people to notice you is by looking stupid you are in trouble. Get better content.

Fabricated words can be fun, and misquoting a saying in a cheesy way might be okay. Most of Nick Cernis’ post seems designed to drive up comment counts without adding any value, though, and in the process it makes you, the author, look dumb. Where is the value in that? My advice is to write something worth reading, use proper grammar and spelling, have a real conversation with your readers, and seem like someone worth reading again.

What bothers me most about Virtual Infrastructure 3.5 is that overall it is a step backwards. Sure, there are new features, but each new feature has some Achilles heel that makes it hard to use. The RCLI is a major problem, and any feature that relies on it suffers. ESX 3i? Not with the RCLI, and not until it is feature-identical to normal ESX 3.5. Storage VMotion? Neat, but there are two people in my organization that can do a storage migration now, and that’s it. I can’t, I won’t, roll this out to my operations staff, mainly because it’ll be a disaster. The document alone would be the single most convoluted document I’ve written in years, again because of the RCLI. Why isn’t that part of VirtualCenter? Update Manager? The first time I ran Update Manager I lost the ability to VMotion anything. I called support to report the problem and ended up having to fix it on my own. VirtualCenter 2.5? 30% of what I do in VirtualCenter 2.5 gives me errors, and when I repeat the same action it succeeds. Out of the box VMotion causes CPU spikes, which can be resolved with a workaround, but that workaround doesn’t seem to work for me. I opened a support case and was told to wait several days for the update to take effect. Are you kidding me? Instead I figured out my own way of getting it to work. I’m not even going to open a support case about my administrative network interfaces dropping out, because what’s the point? VMware support is useless. They’ll search the KB, tell me to reboot the ESX Server with running VMs on it, I’ll object, they’ll ask for a support dump and tell me they’ll get back to me, and I’ll fix it on my own as I do with everything else. Distributed Power Management? Actual round-robin multipathing? All listed as experimental, which is basically using released software as a beta test. Will I use them? Absolutely not.

Now, a single developer created a GUI for Storage VMotion, and it just solidifies what I’ve been thinking all along: every new feature is about half done, shoved out the door as fast as possible so that they could say they have it, with little thought about how people use the products and even smaller amounts of testing. At best the features don’t work right. At worst they compromise the basic functionality of the product, making it far less stable than Virtual Infrastructure 3.0 ever was, running on the same hardware. It’s disappointing. I was a huge VMware fan prior to this, and could be again, but the honeymoon is over. My partner is off cavorting with sexier things while I’m at home paying the bills and taking care of the kids. This marriage needs counseling; I just hope we can meet half way.

You know why that sucks? It guarantees a rise in the amount of crap people carry on. Which in turn guarantees a rise in boarding times. Which also guarantees that I will continue to be shut out of my allotted foot of space overhead by jerkfaces with their three foot long wheeled monstrosities. Then I have to keep my backpack under my feet for the whole flight. I’m more than six feet tall and I don’t like that. Why do they get to use three feet of space and I can’t even get the 10 inches my backpack needs?

You know what I’d rather see them do? Figure out a better way to gate check all those damn wheeled bags. Or, improve the baggage process and make people check their luggage, period. That would also help security checkpoint delays, too.