Linux – Windows 8 on iSCSI with LIO target: thin provisioning

This is possible and is actually far easier than I thought it was.

Get your NFS server up and running, use XenCenter (or the xe CLI) to add a new SR to the pool with type of "NSF VHD" and create all your VM's as normal. They will be stored in .vhd format on the NFS server. You can then use your first VM as the parent in a chain of VM's. There are some limitations to the length of the chain but you can read about those in the XenServer docs if you want.

One extremely good piece of advice I picked up during my research was, "DO NOT, EVER, expand your vdi if you have snapshots." If you do you'll have a great time trying to put things back together again.

If I understand thin provisioning correctly then it could really cause problems if you aren't monitoring your VMFS filesystems growth closely and allow your VMDKs to fill up your VMFS volumes. You've seen in your testing that thin provisioned disks tend to grow to fill their available space quickly and that they cannot reclaim space that may be free inside the OS.

The other option is creating sufficiently sized VMDK files to handle your current usage and expected spikes in growth and just add more VMDK files as your application data usage grows. New VMDK files can be added live to a VM, you just have to rescan (echo "- - -" > /sys/class/scsi_host/host?/scan). You can partition the new disk, add it to your LVM and extend the filesystem all live. This way you are always aware how much space is allocated to each of the VMs and you can't accidently run your VMFS out of space from inside a guest.

As far as whether to partition or not if the disk is only going to be used by LVM, I always partition. Partitioning the disk prevents any warnings about bogus partition tables from coming up when the machine boots and makes it clear that the disk is allocated. It's a bit of voodoo but I also make sure to start the partition at 64 to help make sure the partition and filesystem is block aligned with the underlying storage. It's hard to detect and categorize as you usually don't have something to easily compare against but if the OS filesystem isn't aligned properly with the underlying storage then you can end up with extra IOPS required to service requests which cross block boundaries on the underlying storage.