HP Smart Array P410 Stuck in Ready for Recovery 00.0%

hphp-smart-arrayraidsupermicro

We are using smart array disk controller P410 by HP on our supermicro server.

Sadly of the HDDs in a RAID10 Array was damaged and we were forced to change that specific hard disk.
After 3 days and rebooting the server 2 times we are still seeing the very first warning message after changing the HDD which is saying:

Warning Status Messages ((Ready for Recovery) Logical Drive 1 (931.5 GB, RAID 1+0))
776 (Ready for Recovery) Logical Drive 1 (931.5 GB, RAID 1+0) is queued for rebuilding.

We are worried about the issue and we decided to check the firmware update and hopefully that is up to dated and there is no update available for that.

It is noticeable that we have changed the RAID CARD with a new one with the same model as well.
our raid device information:

Firmware Version 6.40
Number of Ports 2 (Internal only)
Number of Arrays 3
    Smart Array P410 in Slot 1
Bus Interface: PCI
Slot: 1
Serial Number: PACCR9SXRCQH
Cache Serial Number: PAAVPID12031NLH
RAID 6 (ADG) Status: Disabled
Controller Status: OK
Hardware Revision: C
Firmware Version: 6.40
Rebuild Priority: Medium
Expand Priority: Medium
Surface Scan Delay: Not Available
Surface Scan Mode: High
Queue Depth: Automatic
Monitor and Performance Delay: 60 min
Elevator Sort: Enabled
Degraded Performance Optimization: Disabled
Inconsistency Repair Policy: Disabled
Wait for Cache Room: Disabled
Surface Analysis Inconsistency Notification: Disabled
Post Prompt Timeout: 15 secs
Cache Board Present: True
Cache Status: OK
Cache Ratio: 25% Read / 75% Write
Drive Write Cache: Enabled
Total Cache Size: 512 MB
Total Cache Memory Available: 400 MB
No-Battery Write Cache: Disabled
Cache Backup Power Source: Batteries
Battery/Capacitor Count: 1
Battery/Capacitor Status: OK
SATA NCQ Supported: True

We also runned DIAGNOSTIC REPORT Wizard and this the report of our device:

https://www.dropbox.com/s/vy6bo07xaraea1a/report-7c62988a-00000874-00000000.zip

This is a very frustrating situation, Server is working but one of the HDDs of a RAID10 ARRAY is not recovered and joined to the RAID 10 Array.

What should we do and how to resolve the issue?

This is also the output of this command in HP commandline: ctrl all show config detail

https://www.dropbox.com/s/zpadsxcx1emqlvi/ConfigurationsRAID.txt

Best Regards


I resolved the issue by changing those 3 HDDs if i faced with the issue i will follow your recent advice.

After changing the HDDs i booted the server with BIOSUPDATE cd of RAID CONTROLLER.
I deleted that logical drive and re-created that and recovered the server using BARE METAL BACKUP

Everything seems fine and i don't see any error and warning in the ARRAY CONFIGURATION UTILITY.

But i see something not normal.
In ACU when i click on more information for that newly created logical drive there is a section on which partitions of this drive are described and i see this suspicious line:
Partition Number: 1, Size: 100 MB, Mount Point: Unknown

Mount point is drive C but why it is unknown for the RAID?
Server boot up normally.

enter image description here

enter image description here

I think this should be fixed. Do you have any idea about this?

Best Answer

Reading your configuration, I see:

8 disks total...

  • Disks 1,2,3,4 are in a RAID 1+0 array.
  • Disks 5,6 are in a RAID 0 stripe.
  • Disks 7,8 are in a RAID 0 stripe.

I'm not going to ask why you have two RAID 0 arrays. Surprisingly, they're healthy!

It looks like disk 2 was replaced. It's paired with disk 4. You could likely have READ errors on disk 4 that are preventing disk 2 from rebuilding. These are basic 500GB SATA disks, and all of the disks have a number of BUS errors. I actually don't see explicit read/write error counts on the individual drives...

You actually may just have a Supermicro drive backplane issue.

Disks 1,2,3 have Write Retries Failed (0x2b) as the "Last Failure Reason"

If you want to get into the detail of the Array Diagnostic Report, please see this guide.