I just added a spare disk to my array:
Unit UnitType Status %RCmpl %V/I/M Stripe Size(GB) Cache AVrfy
------------------------------------------------------------------------------
u0 RAID-6 OK - - 256K 5587.9 RiW ON
u1 SPARE OK - - - 1863.01 - OFF
VPort Status Unit Size Type Phy Encl-Slot Model
------------------------------------------------------------------------------
p0 OK u0 1.82 TB SATA 0 - ST32000542AS
p1 OK u0 1.82 TB SATA 1 - ST32000542AS
p2 OK u0 1.82 TB SATA 2 - ST32000542AS
p3 OK u0 1.82 TB SATA 3 - ST32000542AS
p4 OK u0 1.82 TB SATA 4 - ST32000542AS
p5 OK u1 1.82 TB SATA 5 - WDC WD2002FYPS-02W3
When I did, tw_cli said this:
Creating new unit on controller /c0 ... Done. The new unit is /c0/u1.
WARNING: This Spare unit may replace failed drive of same interface type only.
What does that warning mean?
Recently, I added a spare to another almost identical machine (with 9650SE as well) with no problems. The only difference was that in the other machine, the array was made up of WDC WD2002FYPS-02W3 disks, to which one was added. In this array, the array was made up of ST32000542AS disks to which a WDC WD2002FYPS-02W3 was added.
It's warning about the interface type, but all are SATA, so I don't get it.
Could this have something to do with the fact that the WDC WD2002FYPS-02W3 is a 4k sector drive and the ST32000542AS is not?
Which brings up a side-question: how does the 9650 SE align it's data? Is it compatible with 4k? Or maybe it's only 4k-aligned when it detects 4k sector disks (in which case, adding a 4k drive as spare will present a problem).
Best Answer
The warning about interface types can be ignored for your hardware, because 9650SE is a SATA-only controller. However, the same tw_cli utility can be used to manage newer cards like 9690SA which support both SATA and SAS drives, but mixing SATA and SAS in the same array is not allowed.
WDC WD2002FYPS does not seem to use 4k sectors — at least the specification sheet does not mention ”Advanced Format” anywhere, so there should not be any alignment issues with it. This drive is even included in the official compatibility list for 9650SE (but even though KB 15573 says that limiting speed to 1.5 Gbps is needed only for 9690SA, the actual compatibility list for 9650SE says “Only compatible if link speed set to 1.5 Gb/sec; see KB 15573”).
3ware 9xxx series controllers place DCB (array configuration data) at the end of each drive, and the user data is written starting from LBA 0 (e.g., see this and this posts), so there should not be any alignment issues from the controller itself.
But there is an alignment issue which is present even with 512-byte-sector disks (tested on 9690SA, not sure if 9650SE has the same problem). When you create multiple volumes from a single array either with autocarving (
/cx set autocarve=on, or the equivalent option in BIOS) or with the “Boot Volume” option in BIOS, the specified volume size will be decreased by 1 (512-byte) sector, and all volumes except the first will be unaligned with respect to underlying RAID stripes. This can be observed in thetw_cli /cx show diagoutput — the controller log includes starting sector numbers for volumes in hex form, which should normally end with 00000 (1 GiB is 0x200000 sectors), but in the bad case they end with FFFFF, FFFFE, …, so you will see something like:or (with a 40 GiB boot volume and 2048 GiB autocarving — note the alignment shift accumulation):
(Well, in this case the 0xffffffff autocarve size limitation is reasonable, but cannot be an excuse for breaking the alignment of the following volume — either the excess sector should be skipped, or the maximum allowed autocarve size should be set to 2047 GiB.)
The only option which works properly is
tw_cli /cx add ... vol=a:b:c:d(with explicit specification of sizes for all required volumes except the last one). It gives properly aligned start sectors — e.g., forvol=2000: