I want to build let's name it a lowcost Ra*san which would host for our social site the images (many millions) we have 5 sizes of every photo with 3 KB, 7 KB, 15 KB, 25 KB and 80 KB per Image.
My idea is to build a Server with 24x consumer 240 GB SSD's in Raid 6 which will give me some 5 TB Disk space for the photo storage. To have HA I can add a 2nd one and use drdb.
I'm looking to get above 150'000 IOPS (4K Random reads).
As we mostly have read access only and rarely delete photos i think to go with consumer MLC SSD. I read many endurance reviews and don't see there a problem as long we don't rewrite the cells.
What you think about my idea?
– I'm not sure between Raid 6 or Raid 10 (more IOPS, cost SSD).
– Is ext4 OK for the filesystem
– Would you use 1 or 2 Raid controller, with Extender Backplane
If anyone has realized something similar i would be happy to get Real World numbers.
UPDATE
I have buy 12 (plus some spare) OCZ Talos 480GB SAS SSD Drive's they will be placed in a 12-bay DAS and attached to a PERC H800 (1GB NV Cache, manufactured by LSI with fastpath) Controller, I plan to setup Raid 50 with ext4. If someone is wondering about some benchmarks let me know what you would like to see.
Best Answer
I would consider a hybrid solution which could be achieved with OpenSolaris, SolarisExp 11, OpenIndiana, or Nexenta. Hybrid pool would be a lot less costly, and with a few thousand bucks worth of RAM, you will have your 150k+ IOPS with mostly normal spinning disks. At Nexenta we have many, many customers who do just exactly this. ZFS is a robust filesystem, and with enough RAM and/or SSDs for additional Read/Write caching you can have a very robust solution at a relatively low cost. With Nexenta Core, which is community, you get an 18TB at no cost at all. Of course, a new release of OpenIndiana would allow a lot of the same functionality. Add to this snapshots, cloning, replication usinf ZFS send/recv and you can build a SAN that will give any EMC a run for its money at a far lower cost. Lots of SSDs are nice, but there are other options, some not half-bad.