Debian – LVM meta data missing from a device

debiandebian-lennylvmmdadm

Here is the situation. I have a MD device, /dev/md0, which is fully functional, but seems to have no LVM meta data. As far as everything I know how to do is concerned, it isn't part of a volume group and it doesn't show up as a physical volume. On the other hand, it is a boot partition and it fills this function with no problem. I can also fail one of the devices in it and re-add them and it rebuilds the array without problems. Why then do I care? Well, every time I update grub I get a bevy of "error: unknown LVM metadata header" messages. This isn't a problem, but it is bugging the heck out of me.

Here are some details:

# uname -a
Linux redacted 3.9-1-amd64 #1 SMP Debian 3.9.6-1 x86_64 GNU/Linux

# pvck /dev/md0
Could not find LVM label on /dev/md0

# mdadm --detail --scan
ARRAY /dev/md/0 metadata=1.2 name=redacted:0 UUID=Stuff
ARRAY /dev/md/1 metadata=1.2 name=redacted:1 UUID=Stuff
ARRAY /dev/md/2 metadata=1.2 name=redacted:2 UUID=Stuff
# lvscan -v
Finding all logical volumes
ACTIVE            '/dev/vg01/home' [232.83 GiB] inherit
ACTIVE            '/dev/vg01/storage' [3.87 TiB] inherit
ACTIVE            '/dev/vg00/root' [59.53 GiB] inherit

# pvdisplay -v
Scanning for physical volume names
--- Physical volume ---
PV Name               /dev/md2
VG Name               vg01
PV Size               4.09 TiB / not usable 512.00 KiB
Allocatable           yes (but full)
PE Size               4.00 MiB
Total PE              1073097
Free PE               0
Allocated PE          1073097
PV UUID               Stuff
--- Physical volume ---
PV Name               /dev/md1
VG Name               vg00
PV Size               59.53 GiB / not usable 4.93 MiB
Allocatable           yes (but full)
PE Size               4.00 MiB
Total PE              15239
Free PE               0
Allocated PE          15239
PV UUID               Stuff

# cat /proc/mdstat 
Personalities : [raid1] [raid6] [raid5] [raid4] 
md2 : active raid5 sdc1[0] sdf1[3] sde1[2] sdd1[1]
      4395406848 blocks super 1.2 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]
md1 : active raid1 sda2[0] sdb2[1]
      62423992 blocks super 1.2 [2/2] [UU]
md0 : active raid1 sda1[2] sdb1[1]
      96244 blocks super 1.2 [2/2] [UU]
unused devices: <none>  

# mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,relatime,size=10240k,nr_inodes=1011774,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=810888k,mode=755)
/dev/mapper/vg00-root on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=1621760k)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
/dev/md0 on /boot type ext4 (rw,relatime,data=ordered)
/dev/mapper/vg01-home on /home type ext4 (rw,relatime,stripe=384,data=ordered)
/dev/mapper/vg01-storage on /storage type ext4 (rw,relatime,stripe=384,data=ordered)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)

# file -k -s /dev/md0 /dev/sda1 /dev/sdb1
/dev/md0:  sticky Linux rev 1.0 ext4 filesystem data, UUID=redacted (needs journal recovery) (extents) (huge files)
/dev/sda1: sticky Linux Software RAID version 1.2 (1) UUID=redacted name=redacted:0 level=1 disks=2
/dev/sdb1: sticky Linux Software RAID version 1.2 (1) UUID=redacted name=redacted:0 level=1 disks=2

Any ways to fix this or should I just live with it?

Best Answer

When creating a PV on a device, the LVM label sits (by default) in the second 512B sector. If the device is a whole disk on which grub is to be installed, then this is exactly where grub's core.img sits. The LVM metadata, for which there is by default one copy per disk, sits after the label, starting in the fifth sector. (An optional second copy of the metadata can be located at the end of the disk.) That metadata size defaults to 255 sectors. You can change the default location of the label to anywhere in the first 4 sectors, using the --labelsector option to pvcreate, but that doesn't look sufficient as grub's core.img appears to need the 62 sectors immediately after the MBR. (The MBR takes up the first sector and contains grub's boot.img.)

It is possible to chainload grub, using one bootloader in the MBR to load boot.img/core.img located somewhere else on disk. Usually "somewhere else" would mean in the volume boot record, the space before the first partition and after the MBR + other interfering data. The problem is that typically the data section of the PV begins immediately after the metadata section. There is a --dataalignmentoffset option to pvcreate that could open up some room here, but that approach might be an exercise for the reader.

See (grub2): http://en.m.wikipedia.org/wiki/GNU_GRUB

See (lvm2): http://www.centos.org/docs/5/html/Cluster_Logical_Volume_Manager/physical_volumes.html

I would suggest that the way around this, rather than using the whole device as a PV, is to create a single partition on your disk. Use that as the PV for LVM. Then the PV/LVM label & metadata will be at the beginning of the partition, rather than at the beginning of the disk. That will leave the 62 sectors after the MBR (and before that first partition) free for core.img.

I'll add that GPT throws a spanner into these works, which I don't begin to understand. [Ok, the comment makes it sound easy enough.]

Related Topic