Does anyone know how shared access (multiple Initiators) to a single iSCSI target is supposed to work.
I have got it semi-working with Linux (CentOS-7) target and Win 2012 server / Win Hyper-V 2012 R2 Initiators. Problem is each instance seems to get it's own version of the NTFS partition.
The reason for doing this is so that I can have my virtual machine files hosted on the remote iSCSI target and migrate the actual running virtual machines between servers. Without sharing the iSCSI target I cannot see any way of doing this.
Best Answer
To have multiple initiators share a single target, whether over ISCSI, Fibre Channel or other SAN solution, you need a cluster-aware filesystem. VMWare ESXi does this with VMFS. Veritas offers one in Veritas Cluster Suite. Sun offered one back in the day that would cause no end of troubles. Oracle did this with RAC, until they got wise and changed to transaction replication instead of shared block-storage. Microsoft offers NTFS Cluster Shared Volumes, which is a Windows clustered filesystem. I believe Red Hat has one, though I have never played with it.
Clustered filesystems have a lot more nuanced work that stand-alone filesystems, since they have to understand quorum-counting, fencing, dead-peer detection and propagation. If they cannot do these things, a split-brain scenario (each node thinks he has the sole access to the shared resource, and cannot check with the other nodes) would enable you to corrupt all your data, or at least get into an inconsistent (and hard to recover from) state.
Another thing to be aware of: most of these are OS-specific. Obviously VMFS is only ESXi (and I think only if it is connected to VCenter). NTFS CSV is only Windows 2012. Oracle RAC is for database clusters running the Oracle database. I think Veritas is the only application-independant/cross-operating-system solution in the mix.
Veritas Cluster Suite is complex and not cheap. Also, you need to understand service groups, order of bring-up/tear-down and clustering/fencing for it to work well. If you miss the section on distributed locking, contention and forcible lock-breaking (I got pulled into a meeting with other architects just as we were beginning this module), you will regret.
Veritas Cluster Suite creates very tightly-coupled clusters, in which one node can induce symathy-sickness into another node. Symantec (who bought out Veritas recently) has a pair of classes that you should probably attend before taking a cluster based on Vx Cluster Suite into production, since it is both powerful and dangerous (like a car or a chainsaw or a side-arm).