Lvm – Remove a duplicate lvm physical volume uuid

lvmmdpvraid

After upgrading a rhel 5 server today I rebooted to the new kernel: curr=2.6.18-371.el5PAE prev=2.6.18-348.18.1.el5PAE.

In the boot sequence I saw a message indicating Logical Volume Management was starting and then almost immediately I saw this and was offered a rescue shell:

Found duplicate PV BPF…ayV: using /dev/sdc1 not /dev/md3.

Note: /dev/sdc1 and /dev/sdb1 are members of the raid1 array /dev/md3.

From this, I assumed that the lvm2 software thinks that /dev/sdc1 and /dev/md3 are pv's with the same UUID and that the lvm2 software was choosing to ignore /dev/md3 and using /dev/sdc1.

I powered down and unplugged the drive for sdc and restarted. Unexpectedly, the system booted without me noticing any problem. Of course, md3 was degraded.

I powered down, plugged in the drive that I unplugged, rebooted, and the system started again without me noticing any problem. Of course, md3 was still degraded but something unexpected happened.

The filesystem within the troubled logical volume was mounted.

I executed pvdisplay and saw the same above error. Of course, when I tried to add sdc1 back into md3 it wouldn't let me because it was in use by the lvm2 software.

I unmounted the filesystem and ran e2fsck on the lv device path. No problems there (but there should have been problems).

There are really four related questions (sorry). Assuming 3's answer is 'yes or sorta', then the answer to 4 is what I need. I asked the first two because I'm assuming that I need to understand their answers to make sense of any answer to the last two.

  1. Why is the filesystem ok if the logical volume was originally made up of a pv of /dev/md3 instead of /dev/sdc1?

  2. Shouldn't the /dev/sdc1 be different from /dev/md3 so as to prevent the logical volume from being consistent with respect to the physical volumes within in? This might be answered by question 1.

  3. Can I fix my problem by removing the pv information from /dev/sdc1 and adding /dev/sdc1 back into /dev/md3?

  4. If the answer to #3 is yes, then how do I go about it without trashing the logical volume and its filesystem?

Some history:

I've never executed "pvcreate /dev/sdc1" so I have no idea why this should be happening. It is true, however, that /dev/sdc has been troubling me lately in that smartmon (sp?) will tell me it can't read the smart data or it can't even see the device. I'll fix the problem either by (a) rebooting, (b) reboot+bios hang+power down+reset sata cable+power on, or sequence b but replace the sata cable instead of just reseating it.

Best Answer

  1. I'm not sure you asked the question you think you asked, but /dev/md3 is the same as /dev/sdb1 and /dev/sdc1 since it's a mirror set.

  2. No, it shouldn't.

  3. No, that will create data loss for you.

  4. N/A

You can probably get rid of this error message by modifying your /etc/lvm.conf file to change the filter to reject sdb* and scd* devices, regenrate your initrd, then reboot.

Related Topic