DL380p G8 SSD cache

We've been using varnish on ssd drives for the last 9 months, it has worked extremely well for us. We previously used a squid memory only cache with a carp layer. It worked, but memory fragmentation was a real problem requiring frequent restarts. Squid 2.x also will only use one core which makes it rather inefficient on current hardware.

For our site, which is very cache friendly, we see about about 10% cpu usage on an 8 core machine serving 100Mbit/s of traffic. In our tests we run out of bandwidth before we hit cpu limits with 2 1Gb ports.

I do have some advice for running varnish with an ssd cache.

• Random write performance really matters. We tried several vendor's for ssd drives before settling on the intel x-25m. We've seen some post as little as .1MB/s for 4k random writes, we get 24MB/s 4k random writes with the x-25m.

• Raid0. The cache in 2.0 is not persistent, so no need to worry about redundancy. This does make restarts hurt, but those are rare. You can do things like load a new config and purge objects without restart.

• mmap mode. The varnish cache can be mmap'd to a file or use swap space. Using swap has not worked well for us, it tends to use more i/o bandwidth to serve the same amount of traffic. There is a 4 sector readahead in the linux swapin code, we wrote a patch to remove this but have not tried it in production.

• Deadline scheduler. With 2.6.28+ this is ssd aware and performs well. We tried noop but found that deadline was fairer as i/o bandwidth becomes limited.

• Disable read ahead. Since there is no rotational delay, no point in reading extra data just because you might need it. i/o bandwidth is precious on these things.

• Run 2.6.28+. mmap of a lot of space on linux gives the memory manager a good workout, but the split lru patches help a lot. kswapd cpu usage dropped a lot when we updated.

We've posted our vcl file as well as several tools we use with varnish at link text. The vcl also includes a neat hack implementing a very fast geoiplookup server based on the maxmind database.

If you're using a server equipped with a Smart Array P400 controller, you're dealing with a G5-era 300-series ProLiant (DL360 G5, DL380 G5, etc.) or a G4/G5-era 500-series ProLiant (DL580, ML570). All of those systems were eclipsed in 2009 or before, so your system is several revisions behind and you're leaving performance on the table...

That said, you can use the Intel SSD's on your P400 controller. Understand that the P400 is only a 3Gb/s SAS controller.

Upgrade the controller firmware first.

They will provide some advantage on writes. Adjust your RAID controller's cache ratio to 75:25 write:read. By having a battery-backed unit (BBWC), the lack of a supercapacitor on your Intel 520 SSD is not an issue. (you'd have the same concern with spinning disks and no BBWC, right?)

Since storage space is your goal, you could also add four more 146GB disks and expand your existing array. There's a benefit there if you have the drive bays to support.

Either way, you have several options.

Edit:

Notes on SSD's and P410 controllers:

What RAID controller comes with DL360 G7?

HP P410 RAID + Samsung 830 SSDs + Debian 6.0 - What performance to expect?