Given the performance levels you are trying to hit then you should bear a couple of things in mind.
The earlier suggestion to format the drive holding your data set with an NTFS cluster size > 25k is a good one, 32k seems like a good choice for your use case. The purpose of doing this is to ensure you're avoiding have to deal with fragmentation if at all possible, and lowering the file system overhead associated with reading in a single file.
Also I'd suggest taking a look at your RAID stripe size. If the nature of your data set results in (mostly) sequential IO then a larger stripe size is more beneficial, if it's random then a smaller stripe size will be smarter, just don't make it any smaller than your files system cluster size. Given the description of what you're trying to do I'd say your IO profile is mostly random so a stripe size of 64k would be fine but it might be worth experimenting with.
What's very important is to make sure the partition is aligned - you will want to use Diskpart.exe on your systems, generally setting an alignment offset of 64k does the trick with standard cluster and RAID stripe sizes but a 1024k offset is used in Vista\Windows7\W2K8 as it will ensure alignment even with larger stripe & cluster sizes. There is a very good article on SQL server performance from Microsoft here that explains the reasons why this is important to do for high performance drives\arrays. The short version of this is that a poorly aligned partition can degrade IO performance by 15-30%.
For SSD's the same general rules apply but the underlying behavior of reads\writes is vastly different. Rather than dealing with 512byte disk sectors at the most fundamental level IO on SSD's uses much larger blocks. Reads tend to be in fixed page sizes of between 4k and 128k, writes involve buffering and large erase block sizes (in the megabyte range). The key thing for you (where read IO is important) is that you want to set your RAID stripe size to be a multiple of the read page size for the disk type you select (I think the Intel X-25's all use a 128k read block) and you want to set your alignment offset to some number that ensures it is greater than that. The standard recommendation of a 64k partition offset would be a poor choice if the SSD has a 128k read page size for example. Because of the asymmetric nature of SSD writes it isn't easy to optimize SSD RAID arrays for write performance but that's a whole 'nother story and involves lots of cache.
There's a nice article on optimizing SSD RAID here on the OCZForum, it's aimed at enthusiast setups but they get the general message right for anyone trying to roll their own SSD RAID from off the shelf kit as far as I can tell.
Finally if your read IO pattern is predominantly sequential the 8 15k drives can (in theory) hit somewhere in the ballpark of 800Meg/sec if not more. Two SSD's, even Intel X-25E's, will only hit about half that. I'd guess that your read IO pattern is biased enough towards random IO to negate this but under the right circumstances your 8 15k SCSI drives can be a lot faster than the two SSD's. This is borne out by your testing. Looking at those numbers though I'd say that some work with partition alignment and stripe sizes will help quite a bit.
Best Answer
We've been using varnish on ssd drives for the last 9 months, it has worked extremely well for us. We previously used a squid memory only cache with a carp layer. It worked, but memory fragmentation was a real problem requiring frequent restarts. Squid 2.x also will only use one core which makes it rather inefficient on current hardware.
For our site, which is very cache friendly, we see about about 10% cpu usage on an 8 core machine serving 100Mbit/s of traffic. In our tests we run out of bandwidth before we hit cpu limits with 2 1Gb ports.
I do have some advice for running varnish with an ssd cache.
Random write performance really matters. We tried several vendor's for ssd drives before settling on the intel x-25m. We've seen some post as little as .1MB/s for 4k random writes, we get 24MB/s 4k random writes with the x-25m.
Raid0. The cache in 2.0 is not persistent, so no need to worry about redundancy. This does make restarts hurt, but those are rare. You can do things like load a new config and purge objects without restart.
mmap mode. The varnish cache can be mmap'd to a file or use swap space. Using swap has not worked well for us, it tends to use more i/o bandwidth to serve the same amount of traffic. There is a 4 sector readahead in the linux swapin code, we wrote a patch to remove this but have not tried it in production.
Deadline scheduler. With 2.6.28+ this is ssd aware and performs well. We tried noop but found that deadline was fairer as i/o bandwidth becomes limited.
Disable read ahead. Since there is no rotational delay, no point in reading extra data just because you might need it. i/o bandwidth is precious on these things.
Run 2.6.28+. mmap of a lot of space on linux gives the memory manager a good workout, but the split lru patches help a lot. kswapd cpu usage dropped a lot when we updated.
We've posted our vcl file as well as several tools we use with varnish at link text. The vcl also includes a neat hack implementing a very fast geoiplookup server based on the maxmind database.