I have a logical volume called storage
. It contains two ~1TB physical volumes: /dev/sdc
and /dev/sdd
. smartctl -a /dev/sdd
informs me that /dev/sdd
is failing. I can still read data off it and I have just backed up all contents off the LV.
I have about 200GB of actual data — little enough that one of the PVs can hold all of it. I want to remove /dev/sdd
from the logical volume and transfer all its data to other PVs (/dev/sdc
) on this LV.
pvmove
fails because all extents are allocated.
# pvmove /dev/sdd /dev/sdc
WARNING: No free extents on physical volume "/dev/sdc".
No specified PVs have space available.
Problem: How do I ensure there is no "real" data on /dev/sdd
and safely remove it?
I've elided the other LVM organization not relevant to this question from the output below.
# lvdisplay --maps
--- Logical volume ---
LV Path /dev/spinny/storage
LV Name storage
VG Name spinny
LV UUID LeJEIR-0Zvf-YBLD-GrCT-Kvv0-mcvz-4a66Ib
LV Write Access read/write
LV Creation host, time archiso, 2020-01-01 12:32:05 -0500
LV Status available
# open 0
LV Size <1.82 TiB
Current LE 476934
Segments 2
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 254:1
--- Segments ---
Logical extents 0 to 238466:
Type linear
Physical volume /dev/sdc
Physical extents 0 to 238466
Logical extents 238467 to 476933:
Type linear
Physical volume /dev/sdd
Physical extents 0 to 238466
# pvdisplay --maps
--- Physical volume ---
PV Name /dev/sdc
VG Name spinny
PV Size 931.51 GiB / not usable 1.71 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 238467
Free PE 0
Allocated PE 238467
PV UUID fMOJ38-TiZZ-DspR-hOy3-4aGn-lKXo-4mYSrU
--- Physical Segments ---
Physical extent 0 to 238466:
Logical volume /dev/spinny/storage
Logical extents 0 to 238466
--- Physical volume ---
PV Name /dev/sdd
VG Name spinny
PV Size 931.51 GiB / not usable 1.71 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 238467
Free PE 0
Allocated PE 238467
PV UUID XJ2D5p-l3iP-VBlR-peXB-up1y-ZpNh-UDkcfB
--- Physical Segments ---
Physical extent 0 to 238466:
Logical volume /dev/spinny/storage
Logical extents 238467 to 476933
# blkid
/dev/sdc: UUID="fMOJ38-TiZZ-DspR-hOy3-4aGn-lKXo-4mYSrU" TYPE="LVM2_member"
/dev/sdd: UUID="XJ2D5p-l3iP-VBlR-peXB-up1y-ZpNh-UDkcfB" TYPE="LVM2_member"
/dev/mapper/spinny-storage: UUID="108a53d4-dc5a-48b7-971f-b62c617fdb15" BLOCK_SIZE="4096" TYPE="ext4"
# smartctl -a /dev/sdd
smartctl 7.1 2019-12-30 r5022 [x86_64-linux-5.6.11-arch1-1] (local build)
# ... clipped ...
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 001 001 051 Pre-fail Always FAILING_NOW 14946
3 Spin_Up_Time 0x0027 170 169 021 Pre-fail Always - 2466
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 80
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
9 Power_On_Hours 0x0032 095 095 000 Old_age Always - 3728
10 Spin_Retry_Count 0x0032 100 253 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 253 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 80
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 19
193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 66
194 Temperature_Celsius 0x0022 103 096 000 Old_age Always - 40
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 183 183 000 Old_age Always - 2919
198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
# 1 Extended offline Completed: unknown failure 90% 3680 -
# 2 Extended offline Completed: read failure 90% 3663 2064
Best Answer
Your "storage" LV is 2TB in size. You cannot remove half of the physical space without damaging it.
What you can do, is shrink the size of the filesystem, then resize the LV, and then you can safely remove the PV from the volume group.
umount /dev/spinny/storage
fsck -C0 -f /dev/spinny/storage && resize2fs /dev/spinny/storage 800G
.lvresize -L900G spinny/storage
pvs
at this point should show that there is no data on sdd and you can safely take it out withvgreduce spinny /dev/sdd
.You may now grow the LV and filesystem to fill sdc if you prefer:
lvresize -l 100%PV spinny/storage
resize2fs /dev/spinny/storage