Linux – Exporting Volume Group From One Server To Another Difficulties

So I managed to solve this eventually. There is a problem (bug) with detecting logical volumes, which is some sort of race condition (maybe in my case regarding the fact that this happens inside KVM). This is covered in the following discussion. In my particular case (Debian Squeeze ) the solution is as follows:

• backup the script /usr/share/initramfs-tools/scripts/local-top/lvm2
• apply the patch from mentioned bug report
• run update-initramfs -u

This helped me, hope it'll help others (strangely, this is not part of mainstream yet).

Link to patch: _http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=10;filename=lvm2_wait-lvm.patch;att=1;bug=568838

Below is a copy for posterity.

--- /usr/share/initramfs-tools/scripts/local-top/lvm2 2009-08-17 19:28:09.000000000 +0200
+++ /usr/share/initramfs-tools/scripts/local-top/lvm2 2010-02-19 23:22:14.000000000 +0100
@@ -45,12 +45,30 @@

  eval $(dmsetup splitname --nameprefixes --noheadings --rows "$dev")

- if [ "$DM_VG_NAME" ] && [ "$DM_LV_NAME" ]; then
-   lvm lvchange -aly --ignorelockingfailure "$DM_VG_NAME/$DM_LV_NAME"
-   rc=$?
-   if [ $rc = 5 ]; then
-     echo "Unable to find LVM volume $DM_VG_NAME/$DM_LV_NAME"
-   fi
+ # Make sure that we have non-empty volume group and logical volume
+ if [ -z "$DM_VG_NAME" ] || [ -z "$DM_LV_NAME" ]; then
+   return 1
+ fi
+
+ # If the logical volume hasn't shown up yet, give it a little while
+ # to deal with LVM on removable devices (inspired from scripts/local)
+ fulldev="/dev/$DM_VG_NAME/$DM_LV_NAME"
+ if [ -z "`lvm lvscan -a --ignorelockingfailure |grep $fulldev`" ]; then
+   # Use default root delay
+   slumber=$(( ${ROOTDELAY:-180} * 10 ))
+
+   while [ -z "`lvm lvscan -a --ignorelockingfailure |grep $fulldev`" ]; do
+     /bin/sleep 0.1
+     slumber=$(( ${slumber} - 1 ))
+     [ ${slumber} -gt 0 ] || break
+   done
+ fi
+
+ # Activate logical volume
+ lvm lvchange -aly --ignorelockingfailure "$DM_VG_NAME/$DM_LV_NAME"
+ rc=$?
+ if [ $rc = 5 ]; then
+   echo "Unable to find LVM volume $DM_VG_NAME/$DM_LV_NAME"
  fi
 }

It seems this is not possible from the commandline. striping is quite different from the default linear allocation scheme. So while you can grow a file system across multiple physical volumes (PV), you do not get the "load balancing" advantages of striping.

To grow the filesystem, you would do something like this:

pvcreate /dev/mapper/md2_crypt
vgextend crypto /dev/mapper/md2_crypt
umount /dev/mapper/crypto-srv
fsck.ext4 -C0 -f /dev/mapper/crypto-srv
lvresize -l +100%FREE /dev/mapper/crypto-srv
resize2fs /dev/mapper/crypto-srv
mount /srv

In the above example, md2_crypt is a crypto volume on top of a RAID-1 array, but it could be any device. crypto is the volume group (VG) and crypto-srv is the logical volume that holds the filesystem mounted on /srv.

Reminder: you should really use at least a RAID-1 array because if a PV fails in a linear logical volume (LV), you can loose all the data.

This will not make the /srv partition striped across the PVs. This is because there are no userland tools to support linear to striped conversions. Logically, this would be in the lvconvert command, but that only supports moving from linear to mirror mode and back, for example. And with all the new development going into BTRFS, I would be surprised if this ever documented.

One thing you can do however, is have the new data on the drive be striped. That is rather tricky, but it can be done. Let's assume we want to stripe crypto-tmp, the LV for /tmp. We first need to free up space for the stripes in the first PV:

pvmove -n crypto/tmp /dev/mapper/md1_crypt

This will reallocate the LV across the PVs, in my case it moved it all to the second PV:

root@foo:~# pvs --segments -o+lv_name,seg_start_pe,segtype
  PV                    VG     Fmt  Attr PSize PFree  Start SSize  LV   Start Type
  /dev/mapper/md1_crypt crypto lvm2 a--  2.73t 10.00g     0   2384 root     0 linear
  /dev/mapper/md1_crypt crypto lvm2 a--  2.73t 10.00g  2384    256 swap     0 linear
  /dev/mapper/md1_crypt crypto lvm2 a--  2.73t 10.00g  2640   2560          0 free
  /dev/mapper/md1_crypt crypto lvm2 a--  2.73t 10.00g  5200 709901 srv      0 linear
  /dev/mapper/md2_crypt crypto lvm2 a--  1.82t  1.81t     0   2560 tmp      0 linear
  /dev/mapper/md2_crypt crypto lvm2 a--  1.82t  1.81t  2560 474077          0 free

Now you can grow the filesystem (say we double its size) and tell LVM to stripe the new data:

root@foo:~# lvresize -i 2 -l 5120 /dev/crypto/tmp
  Using default stripesize 64.00 KiB
  Extending logical volume tmp to 20.00 GiB
  Logical volume tmp successfully resized

Now the new data for /tmp is striped across the two PVs:

root@foo:~# pvs --segments -o+lv_name,seg_start_pe,segtype
  PV                    VG     Fmt  Attr PSize PFree Start SSize  LV   Start Type
  /dev/mapper/md1_crypt crypto lvm2 a--  2.73t 5.00g     0   2384 root     0 linear
  /dev/mapper/md1_crypt crypto lvm2 a--  2.73t 5.00g  2384    256 swap     0 linear
  /dev/mapper/md1_crypt crypto lvm2 a--  2.73t 5.00g  2640   1280 tmp   2560 striped
  /dev/mapper/md1_crypt crypto lvm2 a--  2.73t 5.00g  3920   1280          0 free
  /dev/mapper/md1_crypt crypto lvm2 a--  2.73t 5.00g  5200 709901 srv      0 linear
  /dev/mapper/md2_crypt crypto lvm2 a--  1.82t 1.80t     0   2560 tmp      0 linear
  /dev/mapper/md2_crypt crypto lvm2 a--  1.82t 1.80t  2560   1280 tmp   2560 striped
  /dev/mapper/md2_crypt crypto lvm2 a--  1.82t 1.80t  3840 472797          0 free

But the original data is still in linear mode. It may be possible to move the filesystem to the striped extents, but consider that an exercise to the reader at this point. :)

See also Growing a LVM volume with striping and Redhat LVM changing stripes?.

Also note that the reverse (converting striped to linear) is possible, however: Is it possible to convert striped logical volume to linear logical volume?.