ICT-Freak.nl has a nice post on hot-adding & hot-removing VMDKs from a Linux virtual machine. However, like most guides out there, it doesn’t mention the number one thing you need to do for optimal disk I/O: align your partitions.
From my Linux Virtual Machine Tuning Guide:
Logical Block Addressing is a common addressing scheme for disks on PCs. However, under this scheme the master boot record causes partitions to start at a block that isn’t a power of 2. This isn’t a huge deal for individual disks, but for shared storage where a LUN is actually striped across many different disks a single read or write by the guest OS causes twice as much I/O on the storage array. The misaligned partition has blocks that straddle the stripes on the array, and instead of reading a single stripe the array has to read from, or write to, two stripes. This isn’t a big problem on one or two VMs, but when hundreds of VMs have misaligned I/O the effect is crippling.
Source: Multiple, including EMC & NetApp product documentation, some blog posts. NetApp also supplies tools, mbrscan & mbralign, to help you identify and fix these situations.
The fix for the blog post in question is just another set of fdisk commands. After creating the partition, but before exiting fdisk:
Command (m for help): x Expert command (m for help): b Partition number (1-4): 1 New beginning of data (63-401624, default 63): 128 Expert command (m for help): w Expert command (m for help): q
Ta-dum! You get all your I/O performance. Giving Mr. Fokkema the benefit of the doubt it’s possible he doesn’t need to do this in his example setup. You might need to do it in your production setup, though, and you should follow the instructions of your disk array vendor. Chances are that if you’re using shared storage this is an important step for you. Don’t overlook it!
Thanks for the tip. Just added the fdisk commandos to the post.
I think we do need to keep in mind that if you can it is recommended. But after a P2V it is not always “needed”. Some arrays are less effected by misalignment than others.
Agreed, but, as an example, there are a lot of EMC & NetApp users I’ve seen that don’t practice this, and they end up not liking their I/O performance. I think it’s important to get this sort of optimization into an environment early, so that template VMs are set up correctly. Else you discover, 500 VMs later, that you’ve been doing it wrong and it’s a big problem now…
Thanks guys! Good post, Arne — I didn’t actually know the command to hot-remove a volume. Now I do!
Wouldn’t the alignment need to be done at the esx-host level, not inside the vm?
It needs to be done at both levels, but for many vendors vSphere will do the ESX-level alignment automatically. It does not do it for VMs automatically.