Need to recover RAID 6 array

1. For such issues you're best-off calling Dell support, even if the server is very old and definitely out of warranty and EoL, they might still help you through a RAID rebuild - I know I have.
2. Generally speaking, losing a single drive in a raid5 shouldn't bring your system down, if it has - you're in trouble with more than a single drive loss
3. What you have done might have ruined your raid array completely, but there might still be hope, the procedure you need to do is called re-tagging.

Here are the steps to re-tag a failed raid5:

1. Determine which drive failed first. This can be found by reading the controller logs
2. Remove all drives from the server, make sure you remember which drive was in which slot, it is important
3. Start the server up, enter the controller bios
4. write down all the configuration parameters of the failed raid5 - stripe size, caching setup etc
5. Clear all logical disks, so that the controller has no configuration at all
6. Power server down, insert all disks as they were
7. Power server up, enter controller bios again
8. Create a new raid array, exactly the same as the old one was, all parameters and disk addition order
9. The disk that was the first to fail should be marked as offline
10. This should bring you to a state where you have a usable raid5 in a degraded state.

If re-tagging fails, you're better off taking the disk set to a lab.

If it succeeds, you should boot the system, back it up, and then update the controller and hdd firmware. it is possible that the faulty drive is OK, and it went offline because of a firmware bug, and an update might fix it.

After the update, insert the faulty drive and try to rebuild, if rebuild fails - replace the drive.

You can also run a DST test on the drive to make sure it is usable

This is several questions in one, so I'll try to address a few of them.

I typically set Smart Array controllers to leverage a higher write cache ratio. I prefer to have 75% write cache because the OS (using the XFS filesystem) caches aggressively. XFS will make a difference, but what are you tuning for? Are you tuning to simply achieve specific numbers, or is there an application requirement driving this?

ext3 isn't the fastest filesystem out there. But you have some mount options (e.g. noatime) and journal settings you could tweak.
I don't use LVM, especially with HP controllers that can provide many of the same benefits.
You have I/O scheduler and elevator settings (e.g. noop or deadline, in this case) that can be tuned, but that's a function of your application's actual needs.

If you do go with XFS, try a basic config then try some advanced configuration settings. Over time, I've ended up with mount parameters very similar to the one in the original link.

I just ran the following iozone command line on an XFS partition contained within a DL380 G5 with P400i, 12GB RAM and 8 x 146GB 10k drives. The elevator is set to deadline:

Command line used: iozone -t1 -i0 -i1 -i2 -r1m -s24g

initial writers  =  348957.75 KB/sec
rewriters        =  335130.03 KB/sec
readers          =  132851.70 KB/sec
re-readers       =  137116.27 KB/sec
random readers   =   35774.41 KB/sec
random writers   =  250618.38 KB/sec