Lessons of a VI3 Upgrade

I am just about finished with my VMware Virtual Infrastructure 3 upgrades, and I have made a few observations about the process based on what happened to our clusters. 1. Don’t do an in-place upgrade. Sure, it might work, but it seems unnecessarily complex. What I did was remove a server from a cluster and use it to start a whole new VI3 cluster, including its own VMFS 3 storage. Then I’d scp the disk images from the ESX 2.5.3 cluster to the ESX 3 machine. When I freed enough capacity in the old cluster I’d remove another machine, reinstall it, and tack it into the new VI3 cluster. This plan worked great. It had several benefits, including being able …

Read More

Wisconsin VMUG Still Going Strong

Sorry folks for being out of the game for a bit. Last week got busy with flash floods in our data centers, projects going sideways, and the Wisconsin VMware User’s Group meeting on Friday. Some of you have commented on some of my posts and I want to get back to them with a response. Like a flash flood, though, I find myself being swept away in time by dozens of small fires that need extinguishing. The VMUG was at The American Club in Kohler, WI. It’s the only five-star hotel in the Midwest, and it was really awesome that Fujitsu sponsored it. Even though it was 90° Fahrenheit outside the beer was cold. There was lots of news about …

Read More

Musings on What I Want From Storage Virtualization

Brian over at stereoroid.com commented on my last post about what I want from a storage virtualization engine. Brian, I hope you don’t mind but I’d like to answer outside of the comments section. Your comments were not counterpoint, they were more in the realm of adding clarity, something which my other post may have lacked. I hope this doesn’t scare people away from making comments. 🙂 I really appreciate them. I’m most familiar with EMC and IBM high-end and midrange offerings, and a smattering of whatever LSI Logic is called now (StorageTek/FastT/Engenio/etc.). I don’t know very much about HP EVAs. From what you’ve said it sounds like they are very much like EMC and IBM’s high-end offerings where there …

Read More

What I Want From A Storage Virtualization Engine

Storage virtualization engines suck. Sure, they make it easy to move your data around. They also add cache, which helps a bit. While they’re doing this they also add another point of failure on your SAN, another potential performance bottleneck, another system to learn how to use. They’re clumsy, they’re feature-poor, and I think they have a real long way to go. I’d think about them more positively if they did a few things for me: 1) Automate array-level failover. Take IBM’s Storage Virtualization Controller, for example. It can mirror your data to another array, but if the primary array dies your hosts will lose their storage. You then have to go into the SVC and promote the second copy …

Read More

VMware License Server

So why do I get this feeling, this instinctive repulsion, for VMware’s new license server setup? Is it because FlexLM sucks? Maybe it’s the FlexLM license server I run for a customer where the thing hoses the Linux box if anything is wrong with the network connection. Not that it happens often, but once is once too many. I’m just wondering when the VMware license server is going to take a dump all over my setup like that. It’ll probably be while I’m on vacation somewhere. Maybe it’s just the fact that VMware totally messed up my license key deployment for VI3. Instead of getting the 20 CPUs of standard VIN licenses I got 8, mailed to my purchasing dude. …

Read More

VMware Workload, Environment, and I/O

If there is one thing about the intarweb that really, really sets me off it’s when people reply privately to public inquiries. I usually see this in list archives where someone posted “Hey, I’m seeing a certain problem, has anyone else? Email me off the list.” Hell no. You posted your question publicly, let the response be public, too. Now I have the problem and I have no idea what the solution was, because you kept it private. Why the person replying to you replied privately is beyond me, too. The internet is a big conversation, you idiots, and you’re whispering. If you comment on a post here there are three things that might happen: 1) Nothing. I don’t have …

Read More

VMware I/O Problems

I had previously mentioned I/O problems I was having with my VMware ESX Servers. I didn’t really elaborate on the problems I was having, but having been asked, I will. This was happening to me using ESX Server 2.x against EMC CLARiiON CX700s. It also appears to be a problem under ESX Server 3. The problems I was having were caused by logical block addressing (LBA), a feature of the PC BIOS that reworks the disk geometry so that a disk always appears to have 1024 tracks/cylinders and 63 sectors per track/cylinder. It’s a hack around some old limits on PC hardware. On storage attached to PCs the first cylinder is the master boot record (MBR), partition tables, etc. Because …

Read More

VMware L2 cache

The other day I had a del.icio.us bookmark for a paper from Dell on VMware ESX Server performance on Dell 2850 and 6850 hardware. If you read it take note of the tests with LAMP stack servers. A 6850 with four 3.33 GHz CPUs with 8 MB of cache could only support 2 more VMs in their benchmark than a server with four 3.66 GHz CPUs and 1 MB of cache. That’s fairly interesting, because CPUs with a lot of cache on them are usually quite a bit more expensive. This seems to imply that L2 cache is not a big factor in VMware ESX Server VM performance, at least for that particular LAMP workload. In this case the difference …

Read More

VMware vs. Xen vs. Microsoft

A comment made: Yeah, sure, your costs might drop, but you really think their Linux support will be better than VMware’s support? VMWare’s support is horrible enough, and they don’t make a competing product. I’ve actually had really good experiences with VMware support. The only bad part about it has been that they are learning about their products just as we are. Take support for our EMC CLARiiON CX700s. We were having horrible I/O performance, and the solution for it, after lots of back & forth with VMware support, ended up being found by me in a pair of slides from VMworld. The compelling reason to continue using VMware VirtualCenter and ESX Server products is that in 3.0, which I …

Read More