How To Grow Linux Virtual Disks in VMware

There is plenty of documentation for how to grow a VMware virtual disk file (VMDK), but there doesn’t seem to be a lot of documentation for Linux users who A) are using LVM and B) have aligned their partitions for performance reasons. I was asked how to do this and I thought I’d post it here for everybody. I basically have two suggestions. One is straightforward but you end up with two partitions in your volume group. The other is a little scarier but you’ll have only one partition in the end. This last method is what I was asked about, and what I’ll go into detail explaining.

Obligatory, seemingly obvious statement: have a backup of your system. This is also a great reason to have a test VM or two to practice on. Nothing destroys machines faster than doing bad things to a partition table, and while I intend this to be accurate you should live by the motto “trust, but verify” with stuff you find on the Internet. The nice thing is that you can take a snapshot of the VM and recover easily.

Option 1: Grow the VMDK file, then create a new partition with fdisk. Align its beginning to be on a 128 byte boundary (use fdisk’s expert mode to make the beginning sector of the new partition evenly divisible by 128). Reboot to pick up the partition table changes. Run “pvcreate” on the new partition. Use “vgextend” to grow your volume group. Easy.

Option 2: Delete and recreate your partition using fdisk. Let’s assume you’re running something like Red Hat Enterprise Linux 5, and you’ve installed it using LVM to one VMDK file, where you have two partitions: one for /boot, and one as the physical volume for the LVM. My default installations look something like this, from fdisk’s point of view:

Disk /dev/sda: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          25      200748+  83  Linux
/dev/sda2              26        3916    31254438   83  Linux

If your setup is more complex than this you’ll have to improvise.

1. Once you’ve resized the VMDK file I suggest taking a snapshot while everything is still working. That way if you completely hose your VM’s partition table you have a shot at trying again.

2. Run “/sbin/fdisk -u -l /dev/sda” to determine the size of the partitions in sectors. You need to do this if you’ve aligned your partitions. You should get something like:

11:37am phlox/plankers [~] 5$ sudo fdisk -u -l /dev/sda
Disk /dev/sda: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders, total 62914560 sectors
Units = sectors of 1 * 512 = 512 bytes
   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *         128      401624      200748+  83  Linux
/dev/sda2          401664    52420094    26009215+  8e  Linux LVM

3. Above, in bold, you’ll see the number you care about the most: the starting sector of /dev/sda2. In my case it’s 401664. It will be different for you. Copy all that output into a text file somewhere and save it.

4. Delete the partition and recreate it, with the new end point (which will be the default):

11:48am phlox/plankers [~] 8$ sudo /sbin/fdisk /dev/sda

The number of cylinders for this disk is set to 3916.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): p

Disk /dev/sda: 32.2 GB, 32212254720 bytes
255 heads, 63 sectors/track, 3916 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          25      200748+  83  Linux
/dev/sda2              26        3263    26009215+  8e  Linux LVM

Command (m for help): d
Partition number (1-4): 2

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 2
First cylinder (26-3916, default 26): (I pressed 'enter')
Using default value 26
Last cylinder or +size or +sizeM or +sizeK (26-3916, default 3916): (I pressed 'enter')
Using default value 3916

Command (m for help): t
Partition number (1-4): 2
Hex code (type L to list codes): 8e
Changed system type of partition 2 to 8e (Linux LVM)

5. Before you save the partition table, change the beginning sector of /dev/sda2 to match the old configuration exactly by going into expert mode. In my case it was 401664. It is different for you, use your original value:

Command (m for help): x

Expert command (m for help): b
Partition number (1-4): 2
New beginning of data (401625-62910539, default 401625): 401664

6. Write the new configuration:

Expert command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.

7. Heed the warning about the kernel using the old partition table and reboot. If the VM comes back up, cool. If not, um, revert to your snapshot and figure out what went wrong.

8. Once the machine is back run “vgdisplay” and note the free space (Free PE / Size).

9. Run “pvresize /dev/sda2″ to detect the new space.

10. Run “vgdisplay” again and note the free space. It should have grown, and you’re set.

As always, please leave a comment if something is wrong here or there’s a better way to do this (but please don’t suggest any utilities that can’t handle partition alignment).

Comments on this entry are closed.

  • I prefer to keep my system on one vmdk and having application data etc on another. It’s usually the data VG that needs to be extended, preferably online.

    When creating the data VG I use the disk device (/dev/sdb for example) as a PV instead of creating a partition on it. This way it’s very simple to do an online extension of the VG without any downtime.

    1. Extend VMDK file
    2. # echo 1 > /sys/block/sdb/device/rescan
    3. pvresize /dev/sdb
    4. lvextend xxxx
    5. resize2fs xxxx

    Note: There is currently a bug in RHEL 5.2 that requires the VG to go offline before the OS can see the new size of the disk but this is fixed in 5.3.

  • Christian,

    You’re forgetting about partition alignment, which is a best practice for decent performance on your enterprise disk arrays (at least EMC and NetApp). Without partitions you can’t align your I/O.

    Your method would work fine on arrays without an alignment requirement.

  • Is there a [easy]way to align a partition that already has OS installed? I have around 50 templates with OS and applications already installed on one disk. I don’t want to start all over again and recreate all templates from the scratch with correct partition alignment.

  • Regarding alignemnt,

    LVM adds a header before the first extent, this can be modified during PV creation by using the –metadatasize option. You can see where the first extent is located with pvs -o +pe_start

    So there is really no need for partitions to be able to align the I/O.

  • Great article. After following the above on CentOS 4 df still shows the old free space. The following fixed the problem: run “ext2online /dev/VolGroup00/LogVol00″ replacing VolGroup00/LogVol00 with the logical volume name that shows under df.

  • Hi,

    Thanks for a great article.
    I uses centos 5, and my df would still not report the new allocated size, so after some searching I found I had to add this:

    [root@ece-dev ~]# resize2fs /dev/VolGroup00/LogVol00
    resize2fs 1.39 (29-May-2006)
    Filesystem at /dev/VolGroup00/LogVol00 is mounted on /; on-line resizing required
    Performing an on-line resize of /dev/VolGroup00/LogVol00 to 1409024 (4k) blocks.
    The filesystem on /dev/VolGroup00/LogVol00 is now 1409024 blocks long.

    Cheers,

    Ole

  • Great instructions and very helpful.

    Just to complete the process it appears I needed two extra steps based on the comments above to make the new space fully available on my CentOS 5.3 VM. (continuing from 10 above..)

    11. Use lvextend to increase the space available for the logical volume by the amount of free space displayed with vgdiplay. The command arguments will be different for you:

    # lvextend -L+12G /dev/VolGroup00/LogVol00

    Extending logical volume LogVol00 to 18.88 GB
    Logical volume LogVol00 successfully resized

    12. Use resize2fs to increase the size of the filesystem. The command argument will be different on your system:

    # resize2fs /dev/mapper/VolGroup00-LogVol00

    Filesystem at /dev/mapper/VolGroup00-LogVol00 is mounted on /; on-line resizing required
    Performing an on-line resize of /dev/mapper/VolGroup00-LogVol00 to 4947968 (4k) blocks.
    The filesystem on /dev/mapper/VolGroup00-LogVol00 is now 4947968 blocks long.

    13. (Were you lucky?) Run “df” and check that the partition has increased as expected.

  • Thanks a lot for this instructive web page !
    It’s exactly what I was looking for : expanding a disk of a Linux virtual machine using LVM.

  • Great article, and great comments. Matt’s comment was a good followup on the article, and worked fine for me on my testsystem (CentOS 6.2 x64)

  • Great article! I’m running CentOS 6.3 (2.6.32-279.19.1.el6.x86_64) and the *one thing* I’ll add is that the volume name is /dev/mapper/vg_centosx060tl-lv_root.

    Just so lower-knowledge folks like me don’t get thrown by the “VolGroup” terminology. As with Chris Thorjussen my system is happily reporting an extra 6GB of space.

Previous Post:

Next Post: