I have a setup in which one SAN storage connected by FibreChannel is accessible by two hosts (at block level). And I want to create two lvm2 physical volumes on the storage: pv1 and pv2, for now being the only members of the two volume groups: vg1 and vg2 respectively. My hosts should use them exclusively and "don't touch the other".
Since SAN is visible by both hosts, the lvm structure will be visible by both. (I tested it: creating logical volumes on one host results in them being visible to the other under /dev/mapper). I want to make sure that host1 will only modify vg1 and host2 will only modify vg2 to prevent data corruption.
Is it guaranteed that when I don't explicitly execute any lvm commands, lvm daemons and kernel won't reshuffle extents/optimise the volume group belonging to the other host, thereby corrupting it?
Best Answer
No, it won't corrupt anything just by having the VGs visible on both systems, but having the VGs visible on both hosts is dangerous (due to the likelihood of some process, script, or human messing up and picking the wrong disk to operate on). You can filter them out, so that when performing PV scans, each host ignores the LUN that it shouldn't be touching. Thusly, each will only detect one VG at a time. This is useful in your scenario, and many others.
As an aside (and before I delve into this), you might consider handling this at the SAN level and separating these LUNs into separate targets. That way the base block devices never even get picked up by the "wrong" initiators in the first place. I would recommend this option over others unless there is a really good reason you want the LUNs visible on both hosts simultaneously.
Onward, we're looking at creating filters within the LVM configuration. This is controlled via a series of regular expressions to restrict "pvscan" from detecting certain block devices (or classes of block devices). The file that we make this change in is /etc/lvm/lvm.conf
Within that config, you will find (about half way down) a line that states "global_filter=". The file should give some hints on how to use this, but let's go over some concepts.
First, you should be allowing and restricting block devices based on paths that are not subject to change. DO NOT use things like /dev/sda to select a disk. That can change, and your filters wouldn't be reliable. Instead, use something like /dev/disk/by-id/ or /dev/disk/by-path/. Both of these directories contain symbolic links that use actual information about your LUNs to identify them. by-id uses the SCSI ID, and by-path uses the physical host path and bus type. Use by-id when possible, as it's the most reliable. Because these are symbolic links in the aforementioned directories, you can use `ls -l' to match them up with the /dev/sd* paths they correspond to.
Using that by-id path, you can make a global LVM filter to make scans reject that undesirable block device in future scans. When constructing this line, put allow entries first (denoted by "a"), and reject entries afterwards (denoted by "r"). An example of what that filter might look like this:
global_filter=[ "a|.*|","r|/dev/disk/by-id/id-of-unwanted-disk.*|" ]The statements are encapsulated by square brackets. Statements are encapsulated by quotes, and are separated by a comma. Bars are used to open and close statements that contain slashes (like these do). A dot is put before an asterisk in that reject statement per regex standard to define a catchall wildcard. This line says, "Allow everything but this one drive, and all partitions that may come from this drive." This reject drive in question would be the LUN you don't want LVM to look at.
When you've saved this edit, test it out with "pvscan". If it went that way you wanted, your undesireable VG will no longer show. Do the same thing on the other host, and you have yourself a safe filter.
But seriously, isolate these LUNS using iSCSI targets if you can. It's much easier, and doesn't require configurations on both hosts to succeed.