this is on nexenta open storage. I have a zpool that is in UNAVAIL state because of some disks.
# zpool import -F
pool: delta
id: XYAZA
state: UNAVAIL
status: One or more devices are missing from the system.
action: The pool cannot be imported. Attach the missing
devices and try again.
config:
delta UNAVAIL insufficient replicas
raidz2-0 UNAVAIL insufficient replicas
c0t5000C500573978EBd0 ONLINE
c0t5000C50057397933d0 ONLINE
c0t5000C50057397C47d0 UNAVAIL cannot open
c0t5000C5005739997Fd0 UNAVAIL cannot open
c0t5000C50057399F87d0 ONLINE
c0t5000C5005739A90Bd0 ONLINE
c0t5000C5005739ABC7d0 UNAVAIL cannot open
c0t5000C5005739AC0Fd0 ONLINE
raidz2-1 DEGRADED
c0t5000C50057398657d0 ONLINE
c0t5000C50057398E03d0 ONLINE
c0t5000C50057398EF7d0 ONLINE
c0t5000C50057398FAFd0 ONLINE
c0t5000C5005739A39Fd0 UNAVAIL cannot open
c0t5000C5005739AAFFd0 ONLINE
spare-6 DEGRADED
c0t5000C5005739AC3Fd0 UNAVAIL cannot open
c0t5000C50057398EDBd0 ONLINE
c0t5000C5005739AE9Bd0 ONLINE
~# zpool import -FXf delta
cannot import 'delta': I/O error
Destroy and re-create the pool from
a backup source.
- How can I get this zpool import work fine?
- Are the disks that are pointed out 'UNAVAIL', how can I get them repair/replaced?
Update: 7 Sep, 2018
After finding the disks to slot mapping on Nexenta, reseating the disks in delta volume, I could get one disk ONLINE in raidz2-1 group but the 3 disks in raidz2-0 group still continue to show as UNAVAIL.
# zpool import -F
pool: delta
id: XYAZA
state: UNAVAIL
status: One or more devices are missing from the system.
action: The pool cannot be imported. Attach the missing
devices and try again.
config:
delta UNAVAIL insufficient replicas
raidz2-0 UNAVAIL insufficient replicas
c0t5000C500573978EBd0 ONLINE
c0t5000C50057397933d0 ONLINE
c0t5000C50057397C47d0 UNAVAIL cannot open
c0t5000C5005739997Fd0 UNAVAIL cannot open
c0t5000C50057399F87d0 ONLINE
c0t5000C5005739A90Bd0 ONLINE
c0t5000C5005739ABC7d0 UNAVAIL cannot open
c0t5000C5005739AC0Fd0 ONLINE
raidz2-1 DEGRADED
c0t5000C50057398657d0 ONLINE
c0t5000C50057398E03d0 ONLINE
c0t5000C50057398EF7d0 ONLINE
c0t5000C50057398FAFd0 ONLINE
c0t5000C5005739A39Fd0 ONLINE
c0t5000C5005739AAFFd0 ONLINE
spare-6 DEGRADED
c0t5000C5005739AC3Fd0 UNAVAIL cannot open
c0t5000C50057398EDBd0 ONLINE
c0t5000C5005739AE9Bd0 ONLINE
I found a single spare disk for another online
pool on the appliance.
Now the question is
1. Should I detach/attach the spare to raidz2-1 group which has only one disk UNAVAIL under sub category of spare-6 ? or
2. Should I detach/attach the spare to raidz2-0 group which still has 3 disks UNAVAIL ?
Given that this pool is offline (after export) and is not importable now.
I looked at some history on how this pool was created, it was created with the below command in the past
zpool create -m /volumes/delta -o autoreplace=on delta raidz2 c0t5000C500573978EBd0 c0t5000C50057397933d0 c0t5000C50057397C47d0 c0t5000C5005739997Fd0 c0t5000C50057399F87d0 c0t5000C5005739A90Bd0 c0t5000C5005739ABC7d0 c0t5000C5005739AC0Fd0 raidz2 c0t5000C50057398657d0 c0t5000C50057398E03d0 c0t5000C50057398EF7d0 c0t5000C50057398FAFd0 c0t5000C5005739A39Fd0 c0t5000C5005739AAFFd0 c0t5000C5005739AC3Fd0 c0t5000C5005739AE9Bd0
Does that indicate raidz2-0 and raidz2-1 are two RAIDZ mirror pools?
If yes, should I prioritize fixing raidz2-1 group of disks to become online over raidz2-0 (which has 3 disk failures)?
Best Answer
Your spare disk has already been automatically attached to
raidz2-1
and has been resilvered. Normally, you would now detachc0t5000C5005739AC3Fd0
and remove it form the system because it has failed once (or repurpose it on another system and hope for the best, if you don't have any resources to spare).But that would not help you, as
raidz2-0
is still down. If any vdev in a zpool fails, your whole pool and all data on it is lost. Each vdev will fail if the configured parity cannot be upheld, because enough disks have failed inside it. In your case, each Z2 vdev can sustain the loss of two disks, but one of them has lost three.First, you have no real spare now. And even if you removed a working disk from
raidz2-1
(not recommended!), you would still gain nothing until you fix your failed disks inraidz2-0
. The parity data has to come from somewhere, and in this case it has to come from your failed disks. If you don't get a t least one of them to work, your best bet would be to restore from backup.No, that means your pool consists of two striped vdevs, each configured with raidz2 redundancy and 8 disks. You can lose 2 disks in each vdev at most. If you lose 3 in any of them, it's game over.
In ZFS, all redundancy is handled at the vdev level. The pool is always striped over all existing vdevs. If any vdev fails, the pool fails. Mirroring is a type of redundancy for a vdev and is incompatible with raidz1/z2/z3 (you can mix and match vdevs with different redundancy types, but the weakest one will define the strength of your pool and the performance will suffer, so this is not recommended).
No, you must first fix
raidz2-0
.This doesn't help you now, but maybe in the future: It is advisable to mix and match manufacturers and models and even manufacture dates, to even out cases of "a bad batch of drives". For example, if you bought 4 disks of each of the 4 major manufacturers (Seagate, Western Digital, Hitachi, Toshiba) and split them 2-2 on your vdevs, even if all disks of one manufacturer failed at the exact same time, you would still have a working pool.