HP SmartArray P400i marks a good drive as a failed, what can I do about it

hard drivehardware-raidhp-prolianthp-smart-arrayraid

I have the HP ProLiant DL360 G5 server with the SmartArray P400i RAID controller. The server itself is pretty old, but it still works normally. The only issue is the RAID controller, which marks good drives as failed. It happens quite often, almost every day. Here is the typical output of ssacli utility:

# ssacli ctrl all show config
...
   Array A (SATA, Unused Space: 0  MB)

      logicaldrive 1 (931.5 GB, RAID 1, Interim Recovery Mode)

      physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SATA HDD, 1 TB, OK)
      physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SATA HDD, 1 TB, Failed)

Detailed information:

# ssacli ctrl slot=0 pd 1I:1:2 show detail

Smart Array P400i in Slot 0 (Embedded)

   Array A

      physicaldrive 1I:1:2
         Port: 1I
         Box: 1
         Bay: 2
         Status: Failed
         Last Failure Reason: Not ready bad sense
         Drive Type: Data Drive
         Interface Type: SATA
         Size: 1 TB
         Drive exposed to OS: False
         Logical/Physical Block Size: 512/512
         Firmware Revision: SN03
         Serial Number: ...
         WWID: ...
         Model: ATA     ST91000640NS
         SATA NCQ Capable: True
         SATA NCQ Enabled: True
         PHY Count: 1
         PHY Transfer Rate: 1.5Gbps
         Sanitize Erase Supported: False
         Shingled Magnetic Recording Support: None

After the server is rebooted, the RAID controller detects the drive again, marks it as an OK, and rebuilds the array. The array works well until the next failure. I have no idea about why this is happening. Are there any ways to solve this problem without buying the new RAID or HBA controller? SoftRAID is acceptable. Currently I see these options:

Make a JBOD-like setup with two RAID 0 logical drives, each containing a single physical drive, but I don't know if it will help.
Tune the RAID controller so I won't exclude failed drives from the array, but I don't know how to do this.

Best Answer

I think the drive is bad. You can check this by SMART attributes of this drive.

When RAID controller finds a read/write/verification error on a drive, it marks this drive as FAILED. At this time the drive detects this error and starts the replacing sector procedure. It increases the current pending sectors counter and tries to read bad sector successfully. After successful read data from bad sector the disk writes one to a preallocated sector, decreases the current pending sectors counter and increases the reallocated sectors counter. Nonzero values of these SMART counters show that you have disk problems.

After successful sector reallocation procedure the RAID controller can successfully rebuild the disk array.

Also disk has SMART attributes which show errors in data transfer via the interface cable. A bad cable can give same symptoms of RAID controller actions. But disk problems occur more often than cable problems.

Please read SMART on Wikipedia

Related Solutions

HP SmartArray P400 – How to Repair Failed Logical Drive

The answer is not pleasant. There's a high probability that your array is in a "waiting for rebuild" state, where there's another failing disk in the RAID5 array set that's preventing the recovery from completing. This is why you should avoid RAID5 these days. It doesn't help that these are SATA drives... The likelihood of problems is even higher. Try powering the system off (letting the drives spin down) and powering back on. Follow the prompts at the BIOS array screen and choose the F2 option to "reenable all logical drives". This may kickstart the rebuild process.

Otherwise, it's a rebuild/recovery with new disks.

How to initialize two logical drives on a HP P400i controller without reboot

You are missing a step here. When you remove the old logical drives and the physical disk in slot 4, you have to do the following:

possibly remove Array B
insert a new disk
create a new Array B on the unassigned disk
create logical drives of the requisite size on the new Array B

This can all be done hot in Windows (and usually Linux). The new logical drives will appear upon refresh (F5) of the Disk Manager utility.

Now, you can do this via the hpacucli utility, but it makes much more sense to use the web-based GUI tool, cpqacuxe. You'll understand the steps a bit better there.

Also, I think the continued use of a single-drive RAID 0 array is a bad idea. You do have some interesting options like adding the 72GB disk to Array A and carving out more logical drives. That would help performance and resilience...

Edit: What you're looking to do is not possible without a reboot.

You essentially want to move an array from one server to another. This is possible, but there are rules governing the process. From the HP Smart Array Controllers User Guide:

- *The server is powered down*.
- The array does not have failed or missing drives.
- No spare drive in the array is acting as a replacement for a failed drive.
- The controller is not performing capacity expansion, capacity extension, or RAID or stripe size 
migration.
- The controller is using the latest firmware version.

And the steps to actually perform a move are:

Back up all data before removing any drives or changing configuration. This step is required if you are moving data-containing drives from a controller that does not have a cache module.
Power down the system.
Move the drives.
Power up the system.
Observe the POST messages:
- If a 1785 POST message appears, the drive array did not configure properly. Continue with step 6.
- If a 1724 or 1727 POST message appears, drive positions were changed successfully and the configuration was updated. Continue with step 7.
If the array did not configure properly, power down the system immediately to prevent data loss. Return the drives to their original locations. Restore the data from backup, if necessary.
Verify the new drive configuration by running ORCA or ACU.

So a poweroff/reboot is an essential step in this process. You can't avoid it for what you're trying to do.

Best Answer

Related Solutions

HP SmartArray P400 – How to Repair Failed Logical Drive

How to initialize two logical drives on a HP P400i controller without reboot

Related Topic