Convert a Smart Array RAID5 array to a RAID6 (ADG) variant

The terminology you are looking for is a "RAID level migration".

According to this, it's possible. I haven't done it, but the procedure looks like you should add the new drive as a hotspare to the existing array, then use mdadm to update the raid level and the number of raid devices..

You'll need a recent mdadm to do this: mdadm-2.6.9 (eg, centos 5.x) doesn't seem to support it, but mdadm-3.1.4 (eg ubuntu 11.10) does:

   Grow   Grow (or shrink) an array, or otherwise reshape it in some way.  Currently supported growth options including changing the active size of component devices and
          changing the number of active devices in RAID levels 1/4/5/6, changing the RAID level between 1, 5, and 6, changing the chunk size and  layout  for  RAID5  and
          RAID5, as well as adding or removing a write-intent bitmap.

EG, add a new hotspare device, /dev/sdg, to the RAID5 array first:

$ sudo mdadm --manage /dev/md/md0 --add /dev/sdg

Then convert into a RAID6 array and make it rebuild to a clean state. The --raid-devices 4 tells you how many drives you have in total in the new array.

$ sudo mdadm --grow /dev/md/md0 --raid-devices 4 --level 6

I have no idea how quick this will be though. In my experience with doing raid level migrations on hardware RAID controllers, it's been quicker to create the new array from scratch and recover your backup to it.

Totally love this question - thanks for asking it!

Now I have a cunning plan - take this time to virtualise and make all future work super-easy.

The plan would be to get a temporary/rented server, install basic (free) ESXi onto it, take your server down for a few hours and convert the current server to a VM on the ESXi box, install the vmtools and test it while isolated from your real users.

If this testing goes well you can schedule to do the same thing again, but for real this time!

What you'd do is shutdown your server again, convert it to a VM again but then remove all of your disks and set them aside as a fall-back option. Then add the extra CPU/memory/cache and create your new array on your new disks. Install ESXi on this new array and simply copy the VM from the temp ESXi box to the newly upgraded one, start it and install the vmtools.

If all goes well then you should see a huge performance increase from the extra CPU, memory and disks - yes you'll lose a few percent of that performance to the virtualisation layer but now you'll have an easily transported server that can be moved, upgraded etc. as it'll be encapsulated inside a VM. If it goes badly simply re-insert the old disks and you're back to where you were.

Does this sound feasible? I know you don't have the budget for paid support from VMWare but your setup would be super-easy and we could help with the config of this if you had specific questions, plus you get to 'play' with this temp server anyway - don't forget that temp machine doesn't have to have oceans of CPU, memory or disk performance, just enough to hold the new VM.

Hope this helps.