Adding additional disks to HP P2000 SAS Storage

I've wrestled with this question for a while. There are a number of factors determining how many disks should go into a RAID5 array. I don't know the HP 2012i, so here is my generic advice for RAID5:

• Non-recoverable read error rate: When a non-recoverable read error occurs, that read fails. For a healthy RAID5 array this is no problem since the missed read can be found in the parity information. If one happens during a rebuild, when the entire RAID5 set is read in order to regenerate the parity info, it can cause the entire RAID5 array to be lost. This rate is measured like this: "1 per 10¹⁴ bits" and is found on the detail tech-specs for drives. You do not want your RAID5 array to be any more than half that size. Enterprise drives (10K RPM SAS qualifies) can go longer than Desktop drives (SATA).
  • For an example of this spec, Seagate Barracuda ES.2 data-sheet.
• Performance degradation during rebuilds: If performance noticeably sucks during rebuilds, you want to make sure your array can rebuild quickly. In my experience write performance tends to suck a lot worse during rebuilds than reads. Know your I/O. Your tolerance for bad I/O performance will put an upper limit on how large your RAID5 array can get.
• Performance degradation during other array actions: Adding disks, creating LUNs, changing stripe widths, changing RAID levels. All of these can impact performance. Some controllers are very good about isolating the performance hit. Others aren't so good. Do some testing to see how bad it gets during these operations. Find out if restriping a 2nd RAID5 array impacts performance to the first RAID5 array.
• Frequency of expand/restripe operations: Adding disks, or on some controllers creating new LUNs, can also cause the entire array to redo parity. If you plan on active expansion, then you'll be running into this kind of performance degradation much more often than simple disk failure-rate would suggest.

RAID6 (double parity) is a way to get around the non-recoverable read error rate problem. It does increase controller overhead though, so be aware of the CPU limits on your controllers if you go there. You'll hit I/O bottlenecks faster using RAID6. If you do want to try RAID6, do some testing to see if it'll behave the way you need it to. It's a parity RAID, so it has the same performance penalties for rebuilds, expansions, and restripes as RAID5, it just lets you grow larger in a safer way.

Which is better and whether it matters depends on what you want to do with the disk capacity.

If you need to totally isolate the IO between the three virtual disks then having three RAID groups makes sense. If you have IO requirements for individual volumes that will benefit from being able to make use of more peak IO\Bandwidth than can be delivered by 8 disks then you may be better off making one larger RAID group (to get all the spindles into one pack) and splitting that up into the multiple virtual disks.