Nginx – Can Varnish handle hundreds of thousands of requests per second? If not, can I cluster it

nginxreverse-proxyvarnish

I'm planning to have an architecture similar to:

  nginx - nginx - nginx - nginx
    \       \       /       /
             varnish 
    /           |         \
app server - app server - app server

(The app servers are all "identical" in the sense that a request can be routed to any of them by Varnish.) The varnish in that diagram would be processing (potentially) hundreds of thousands of requests per second. Can it do that?

I'd prefer to run multiple Varnish servers, for failover and performance reasons, but the immediate problem I see is that caching wouldn't have much use, because each request would hit a different Varnish server, until each of the Varnish servers had a copy of the cached object. What's the right way to do this? (Again, the app servers are identical to Varnish, it doesn't matter to which the request is routed. I'd like to have multiple Varnish servers (behind nginx's load balancing) processing the requests.)

Best Answer

I recently dealt with the same question. Varnish can handle quite a lot of requests per second, but you should test it with your setup (hardware, network, size of responses, hit ratio) to get an idea about performance numbers. Besides performance, there's the matter of failover to start balancing.

  • if urls are your cache key, you can setup a mechanism in nginx that chooses a specific varnish instance based on the url (varnish_instance = hash(url) modulo nr_of_varnishes). If varnish rewrites the url before it forwards it to a back-end or does a cache lookup, and different urls get rewritten to the same new url, then this trick isn't effective.

  • in my case I can't route based on the url at the loadbalancer. We use lvs-dr and simply don't know about the url at the balancer.

  • I've played with the idea to setup such a distribution mechanism in varnish. One could configure the other varnishes as 'back-ends', calculate a hash and route the request to the right varnish. The 'right' varnish does the back-end call and stores it in cache. Other varnishes could store the results as well but don't have to. This setup makes your varnish configuration more complicated, so think carefully before choosing such a path. Direct Routing (part of lvs-dr) makes it even more complicated.

In the end I chose a simple solution: distribute requests over 2 big varnish instances without any smart stuff. Results are calculated and cached twice, but varnish configurations were kept as simple as possible.

Related Topic