How To Resize a Xen Virtual Disk

One of the challenges that comes with managing Xen domU instances is dealing with virtual machines that run out of disk space. If the initial setup was not done carefully, you may find that resizing the virtual machine is a bit more involved than it sounds, especially if the domU is backed by a disk image or single physical partition.

In this post, I’ll demonstrate two different methods for adding more disk space to a Xen virtual machine. Unless otherwise indicated, all systems described here are running fully up-to-date versions of CentOS 5.6 (Final), and all Xen domU instances are using file based disk images.

When a file-backed virtual machine runs out of space, one way to gain additional space is to add a second file-backed virtual disk. Note that you could also add a physical disk, and the steps would largely be the same, but that is left as an exercise for the reader.

Create a new virtual disk

First, create a sparse file to use as the new disk image. In this case, we will create a new 10G image file. Note that creating a sparse file in this way will be very fast, but it will only actually use 1M of space on disk (as reported by df), so keep in mind that the actual usage can increase up to the total size of the file you are creating (here, 10G).

# cd /var/lib/xen/images
# dd if=/dev/zero of=new_disk.img oflag=direct bs=1M seek=10239 count=1

Add the Disk to the Virtual Machine

Once you have the new disk image created, you will need to add it to the virtual machine. First, modify the disk entry of the config file so that the disk is recognized when the virtual machine is restarted. In this case, the second entry is the one being added:

disk = [
    "tap:aio:/var/lib/xen/images/original.img,xvda,w",
    "tap:aio:/var/lib/xen/images/new_disk.img,xvdb,w"
]

Note: Be sure that you use a device name that is not already being used by the domain. In this case, only xvda was in use so using xvdb for the new disk device name is fine.

At this point, you can restart the virtual machine to have the disk recognized, but you can also dynamically add the new storage to the domU without restarting by using the xm utility and the block-attach sub-command (substitute the name of your domain in the following command):

# xm block-attach MyDomain tap:aio:/var/lib/xen/images/new_disk.img xvdb w

You can use the block-list sub-command to confirm that the device was added to the running virtual machine; prior to running xm block-attach, only one device would have been listed here:

# xm block-list MyDomain
Vdev  BE handle state evt-ch ring-ref BE-path
51712    0    0     4      6      8     /local/domain/0/backend/tap/3/51712  
51728    0    0     4      8      12    /local/domain/0/backend/tap/3/51728  

Once the domU knows about the new disk, you can configure it for use. Log into the virtual machine and use fdisk to create a new partition table. Here we create a single partition using the default type of 83 (Linux):

# fdisk /dev/xvdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel. Changes will remain in memory only,
until you decide to write them. After that, of course, the previous
content won't be recoverable.


The number of cylinders for this disk is set to 1305.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
   (e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1): 
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-1305, default 1305): 
Using default value 1305

Command (m for help): p

Disk /dev/xvdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot      Start         End      Blocks   Id  System
/dev/xvdb1               1        1305    10482381   83  Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
# 

Once the partition table is initialized, there are two paths you can take to add the new space to the virtual machine:

In general, my preference is to add the space to an LVM group, but there are valid reasons for both methods so each is be presented here for completeness.

Mount the new partition traditionally

This is the best option if you need to create isolated space for a specific purpose, or if you do not have LVM available to you on the virtual machine. In either case, the first step is to create a filesystem on the new partition. Here, the standard ext3 format is used:

# mkfs.ext3 /dev/xvdb1

Next, a directory must be created to serve as the mount point for the new partition. I like to create mount points with permissions 000 to prevent data from accidentally being written to the directories when the partition is not mounted; any files living in the mount point will be hidden when the partition gets mounted, and making it non-writable, is a good way to help prevent that.

# mkdir -m 000 /extraspace

Create an entry for the new mount point in /etc/fstab (so that it is mounted on boot) and mount the partition. Make changes the the following mount options as needed for your system:

# echo "/dev/xvdb1 /extraspace ext3 defaults 1 3" >> /etc/fstab
# mount /extraspace

Finally, use df to confirm that the new partition is mounted and available for use:

# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                       45G   40G  2.4G  95% /
/dev/xvda1             99M   22M   73M  23% /boot
tmpfs                 1.0G     0  1.0G   0% /dev/shm
/dev/xvdb1            9.9G  151M  9.2G   2% /extraspace

Incorporate the new space into an LVM group

As an alternative to mounting the new disk in the filesystem, you can also add it as a physical volume to an existing LVM volume group. This is the best option if you need to increase the size of an existing disk transparently without disrupting existing operations.

If you choose to add the disk this way, start by following the procedure for creating the new partition table using fdisk as described previously, but be sure to use the t command to change the partition type to 8e (Linux LVM). The additional output of fdisk for this portion of the process will resemble the following:

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): 8e
Changed system type of partition 1 to 8e (Linux LVM)

Command (m for help): p

Disk /dev/xvdb: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

    Device Boot      Start         End      Blocks   Id  System
/dev/xvdb1               1        1305    10482381   8e  Linux LVM

Command (m for help):

Once the partition has been initialized, use the LVM tools to create a new physical volume with the xvdb1 partition that was just created. Then add the physical volume to the volumegroup of your choice, and extend your logical volume to use all available space.

In this example, I am using volume group VolGroup00 and logical volume LogVol00; these are the defaults for a standard CentOS LVM configuration.

# pvcreate /dev/xvdb1
  Physical volume "/dev/xvdb1" successfully created
# vgextend VolGroup00 /dev/xvdb1
  Volume group "VolGroup00" successfully extended
# lvextend -l+100%FREE -r VolGroup00/LogVol00
  Extending logical volume LogVol00 to 55.88 GB
  Logical volume LogVol00 successfully resized
resize2fs 1.39 (29-May-2006)
Filesystem at /dev/mapper/VolGroup00-LogVol00 is mounted on /; on-line resizing required
Performing an on-line resize of /dev/mapper/VolGroup00-LogVol00 to 14647296 (4k) blocks.
The filesystem on /dev/mapper/VolGroup00-LogVol00 is now 14647296 blocks long.

Note: LVM will automatically re-size the filesystem on the logical volume, which is nice.

Finally, compare the file system disk space usage with the output of the previous df command to confirm that the additional space is indeed available:

# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
                       55G   40G   12G  78% /
/dev/xvda1             99M   22M   73M  23% /boot
tmpfs                 1.0G     0  1.0G   0% /dev/shm

Conclusion

In the end, adding additional space to a Xen virtual machine is not that difficult. Please let me know if you have found this tutorial helpful, or if you have had similar experiences modifying disks on a Xen domU.

Stay tuned for my next article on migrating a Xen virtual machine from a file based container to an LVM-backed block device.

Chris Abernethy
PHP Wrangler, MySQL DBA, Linux SysAdmin and all around computer guy, developing LAMP applications since Slackware came on 10 floppy disks.

One Comment on "How To Resize a Xen Virtual Disk"

  1. Guillermo says:

    Great write-up, I’m regular visitor of one’s website, maintain up the
    excellent operate, and It is going to be a regular visitor for a long time.

Got something to say? Go for it!