Nginx – Scaling a node.js application, nginx as a base server, but varnish or redis for caching

cachenginxnode.jsredisvarnish

I'm not close to being well versed in using nginx or varnish but this is my setup at the moment.

I have a node.js server running which is serving either json, html templates, or socket.io events. Then I have nginx running in front of node which is serving all static content (css, js, etc.).

At this point I would like to cache both static content and dynamic content to memory.

It's to my understanding that varnish can cache static content quite well and it wouldn't require touching my application code. I also think it's capable of caching dynamic content too but there cannot be any cookie headers?

I do use redis at the moment for holding session data and planned to use it for other things in the future like keeping track of non-crucial but fun stats.

I just have no idea how I should handle caching everything on the site. I think it comes down to these options but there might be more:

Throw varnish in front of nginx and let varnish cache static pages, no app code changes. Redis would cache dynamic db calls which would require modifying my app code.
Ignore using varnish completely and let redis handle caching everything, then use one of the nginx-redis modules. I'm not sure if this would require a lot of app code changes (for the static files).

I'm not having any luck finding benchmarks that compare nginx+varnish vs nginx+redis and I'm too inexperienced to bench it myself (high chances of my configs being awful).

I'm basically looking for the solution that would be the most efficient in terms of req/sec and scalable in the future (throw new hardware at the problem + maybe adjust some values in a config = new servers up and running semi-painlessly).

Best Answer

For the purely static content, you normally gain very little advantage by running Varnish in front of any mature web server such as nginx or Apache. (I can't speak for node.js because I have not yet used it.) The reason for this is that the kernel maintains a filesystem cache which stores recently accessed files in RAM. It's possible that Varnish does a slightly better job than the kernel at choosing exactly which files to keep in the cache and which to eject but it's also possible that it does a worse job. This will depend on what else your system does as well as the differences in the cache retention policies. The extra latency caused by having a proxy in front of your web server will far exceed any differences between Varnish and kernel filesystem caching.

You are correct about Varnish not caching responses sent with a Set-Cookie: header. It also skips looking at the cache if the request contains a Cookie: header. The reason for this is that there will be one such unique response for each and every user who visits any given page and each user is unlikely to re-visit the same page meaning that your cache would be full of entities that will never be used.

Varnish can cache dynamic content and this is where it shines. Let's say the front page of your website requires a PHP code compile and 20 database queries on cold caches. On warm caches (APC, memcached, Redis, MySQL query cache, etc.) it still requires lookups to memcached/Redis and doing a stat() on every PHP file included in the request. Assuming you have a reasonably well optimised site, this is still likely to be at least 1/10th of a second (and that's pretty well optimised; 1 - 3 whole seconds is far more common in my experience). Varnish will happily serve the same page in under 1/1000th of a second. But it means that your users can't be logged in on the front page of your website because they can't get their Cookie: headers but if this is acceptable to you, Varnish is an easy win.

Varnish also requires correct caching headers. If it isn't sure that it's safe to cache an object, it won't. If you meet all these requirements, Varnish might be the tool for you. That said, just putting the correct headers in place will cause the clients to cache the content themselves which can make a much larger difference than Varnish can.

If you lack the experience to benchmark your setup yourself, now is the time to get that experience. You are benchmarking to gauge the appropriateness of your configs. Try something and run ab over it, then change one thing and run ab in exactly the same way against that configuration. Now you know which one is "better". Rinse, repeat. (Also, always run ab twice and ignore the first result. This will make sure you only test warm caches and don't end up erroneously thinking one configuration is worse because it was tested against cold caches.)

Architecting a scalable solution for your website is a complex enough topic that it won't fit in a ServerFault answer. We can certainly help with individual parts of that (such as "How do I scale out my memcached server to a memcached cluster?") when you run into those problems.

Further reading:

I wrote an answer a while ago that had a section on finding the bottleneck when benchmarking.
Womble recently pointed me at an excellent post on finding bottlenecks.

Related Solutions

Nginx – How to Set Up as a Caching Reverse Proxy

I don't think that there is a way to explicitly invalidate cached items, but here is an example of how to do the rest. Update: As mentioned by Piotr in another answer, there is a cache purge module that you can use. You can also force a refresh of a cached item using nginx's proxy_cache_bypass - see Cherian's answer for more information.

In this configuration, items that aren't cached will be retrieved from example.net and stored. The cached versions will be served up to future clients until they are no longer valid (60 minutes).

Your Cache-Control and Expires HTTP headers will be honored, so if you want to explicitly set an expiration date, you can do that by setting the correct headers in whatever you are proxying to.

There are lots of parameters that you can tune - see the nginx Proxy module documentation for more information about all of this including details on the meaning of the different settings/parameters: http://nginx.org/r/proxy_cache_path

http {
  proxy_cache_path  /var/www/cache levels=1:2 keys_zone=my-cache:8m max_size=1000m inactive=600m;
  proxy_temp_path /var/www/cache/tmp; 


  server {
    location / {
      proxy_pass http://example.net;
      proxy_cache my-cache;
      proxy_cache_valid  200 302  60m;
      proxy_cache_valid  404      1m;
    }
  }
}

Nginx – how to use nginx as a caching reverse proxy to replace squid/varnish

You can use nginx as a caching reverse proxy. The following links will be helpful.

http://staff.adams.edu/~cdmiller/posts/nginx-reverse-proxy-cache/

http://forum.nginx.org/read.php?2,4979,46962

Best Answer

Related Solutions

Nginx – How to Set Up as a Caching Reverse Proxy

Nginx – how to use nginx as a caching reverse proxy to replace squid/varnish

Related Topic