Nginx with Memcache Vs Varnish

This bug report seems to suggest a workaround to your problem; you should give it a try.

https://www.varnish-cache.org/trac/ticket/849

A few additional links and comments:

• There was a recent benchmark comparing Nginx, Apache, Varnish, and GWan for serving 100 byte static files.
• The Cherokee site has a few benchmarks comparing Nginx and Lighttpd.
• More links: One, Two, Three, Four, Five
• Be aware that benchmarks typical have a narrow band of validity, especially at this large a request rate. For example, the first benchmark is fine if all you're serving is 100 byte files but results may be significantly different if using 1kb, 10kb or a range of static file sizes. A lot of published benchmarks also have the issue that the author has good experience with only one of the products, meaning it has a very good configuration, and not of the other products, meaning they have a suboptimal or even bad configuration, which will skew the results.
• At this large of a request rate there are going be to many things that can effect the benchmark results. Not only the web server configuration but the OS configuration and hardware itself. For example, using a SSD drive may increase request rates but may work with web server X better than Y.
• Don't just look at the big maximum request rates from benchmarks but consider the average and minimums as well. For example, look at the Apache results from the first benchmark link.
• The best benchmark is doing to be one you do yourself using the files your serving on the hardware you're doing to use. Even quick benchmarks may reveal issues and effects not apparent from other published results.
• At this scale will it really matter if you can serve 15k or 20k requests/second? For example, at 20kr/s a 1.7kb file will require around 100TB of bandwidth a month. For the amount of money that the bandwidth will cost to buy another server (or 10) will be cheap by comparison. I would not choose solely by which server gives the biggest "number" but also consider how easy it is to setup/use, does it match the required feature set, is it well supported, etc....

Personally, I have used Lighttpd for years and couldn't be happier with it. I'm actually surprised at it performed compared to Nginx in the Cherokee benchmark results.