Use ext3 as a shared fs if I add a lock manager

I would guess that you can give N VMs and access to the same block device but only read only. You note that word guess.

Things will fall over quickly if any of the VMs start modifying things since any disk caching the other VMs will now be invalid.

The difficulty in having file locking information on disk is that no one has done that yet. Without deep thought, the idea of an atomic read/modify/write transaction on a disk is probably the base problem.

I guess that you could take a typical file system that is already multi-threaded, figure out where the lock(s) is/are used and change that to some sort of read/modify/write operation on the disk.

The other problem with read/modify/write on the disk is that if you do a lot of locking, your performance will be pretty bad since every change to anything on the disk requires:

• lock the disk (seek, read/modify/write)
• read the sector you want to change (seek/read)
• make the change and write (write)
• unlock the disk (seek, read/modify/write)

It probably doesn't help, but I would look at something like BSD Jails as a more lightweight VM system if you really don't want networking up.

The way GFS2 locking works, you may be seeing serious performance problems by pointing each node to a different NFS server:

If another node requests a glock which cannot be granted immediately, then the DLM sends a message to the node or nodes which currently hold the glocks blocking the new request to ask them to drop their locks. Dropping glocks can be (by the standards of most file system operations) a long process. Dropping a shared glock requires only that the cache be invalidated, which is relatively quick and proportional to the amount of cached data.

Dropping an exclusive glock requires a log flush, and writing back any changed data to disk, followed by the invalidation as per the shared glock.

[...]

Due to the way in which GFS2's caching is implemented the best performance is obtained when either of the following takes place:

• An inode is used in a read only fashion across all nodes.
• An inode is written or modified from a single node only.

Additionally, support documents as those from Red Hat indicate that POSIX locks on NFS shares would induce problems, so only Active/Passive clustering configurations where NFS is exported from a single active node at any given time and no file access to the GFS2 file system is performed except the one set up through the NFS service would be supported. Obviously, this should take care of any kind of unforeseen locking interactions between NFS and GFS2, but probably is not what you wanted to see.