How to get more I/O performance from Citrix XenServer

hard driveperformancexen

I have a Citrix XenServer 6.1 installed on a HP Server (1x Intel Xeon E5630 @ 2.53GHz (4 cores, 8 threads, RAM: 752MB for dom0 and 55286MB for the guests). The server has 2 scsi mirrored disks and is used for dev/testing.

The host runs 5 guests that I powered on or off during the tests. Most of the perf tests gave poor results. But I'd like to tune Xen: for instance, a "dd if=/dev/zero …" gives 130MB/s on the host, but only 75MB/s on a lone guest.

If I run a dd if=/dev/urandom on a guest, the CPU is saturated and I get 7MB/s.

Via tune-vcpus, I managed to give 8 vCPUs to dom0 and I got less perfs. The default is 4 vCPUs and I'd like to give dom0 only 1 vCPU but it doesn't work.

Here is what I tried:

Edit /etc/sysconfig/tunes-vcpus > NR_DOMAIN0_VCPUS=1 and MAX_NR_DOMAIN0_VCPUS=1
/etc/init.d/tune-vcpus start min
reboot the host

I also tried

/opt/xensource/libexec/xen-cmdline -set-dom0 blkbk.reqs=256

to get more perf from the storage it doesn't change anything.

And I enabled Qos on the VBDs and gave the highest priority to the one where I do my tests.

After all that, I don't get any gain in I/O performance. Is there anything else to do?

Best Answer

You can try to find a good value for max_sectors_kb. By default it set to 512 or 1024. For example you can set it to 128 and test again (dom0 as well as domU).

echo 128 > /sys/block/[your blockdevice]/queue/max_sectors_kb

This setting is not persistent. Put an entry into /etc/rc.local to set it on startup.

Please post your results.

Related Solutions

Linux – Xen machine can see others on subnet but not gateway

Show your xen bridge/route script (that actually provides network for domU).
What's the difference in tcpdump output on dom0 external interface when you ping host A and gateway from domU ?

DRBD as a block device for XEN VM (Centos 5.3)

Well, seeing as though I pulled my hair out solving this, let me answer my question and save someone else the trouble of hair pulling :)

Solution:

After playing around, plenty of Googling and repartitioning etc... I came to a setup that works like a charm.

There is probably a quicker way to do this but I am not going to over complicate this answer

I did a standard install with the partitions like this (I have a 500g hard drive) :

/boot 100mb 
/swap 4gb 
/     40gb

The balance of the disk space is to be left as unpartitioned space.

Then, I created a primary partition called /dev/sda4 by following these steps:

~: fdisk /dev/sda
~: (fdisk shell) p4   (for primary partition # 4)
~: (fdisk shell) t    (hit t and enter to edit the partition type)
~: (fdisk shell) 08e  (Linux LVM)

Reboot the server so that the new partitions can take effect.

Now create logical volumes by:

~: pvcreate /dev/sda4
~: vgcreate xenvg -s 4M /dev/sda4    # (xenvg is the name of my virtual group, you can rename it)
~: lvcreate -L400G -n xenroot xenvg  # (xenroot is going to be my drbd  resource and root partition for my DomU)
~: lvcreate -L4G  -n xenswap xenvg   # (xenswap is my swap file for my DomU)

Now that you have the correct partitioning you can go ahead and install DRBD with the following config file directives (drbd.conf)

Just displaying the 2 important directives here...

{
device  /dev/drbd0;
disk    /dev/xenvg/xenroot;
}

Your XEN VM config file needs to look like this (again, just the important one)

{
disk = [ "drbd:xenvm,xvda,w","phy:xenvg/xenswap,xvdb,w" ]
}

I hope this helps someone out there...

Related Topic