How to efficiently share host’s folder with KVM guests

9pkvm-virtualizationqemustorage-area-networkvirtio

I am configuring the service which stores plenty of files uploaded by nginx in /srv/storage dir on host system. These files are processed by worker KVM guests which may create new files or assign extended attributes to existing ones. Files are never overwritten but eventually deleted by one of worker.

So host server has file write speed about 177MB/s. KVM image is QCOW2 file stored on host filesystem and image achieves ~155MB/s inside KVM instance thanks to this virtio setting:

<driver name='qemu' type='raw' cache='none' io='native'/>

However I can't get such results for shared folder. I got max 40MB/s with virtfs aka virtio 9p. It seems there is no AIO equivalent for mount:

mount -t 9p -o trans=virtio,version=9p2000.L uploads /srv/storage

I was also thinking about:

  • no NFS – extended attributes are missing
  • no GlusterFS – works but performance worse than virtio because networked, kind of overkill on single hardware machine,
  • maybe sharing LVM volume for r/w? – actually the folder is stored in separate partition, however I read somewhere that LV can't be r/w shared because this may cause fs corruption.
  • keep uploaded files on QCOW2 and share it by all parties?
  • keep nginx and uploaded files in KVM instance on QCOW2, share the image with all guests somehow?
  • iSCSI – possible with single partition?

So how to efficiently share host's folder with KVM guests with extended attributes working).

Best Answer

CIFS can do extended attributes. You can set it up with Samba on Linux.