Nginx – Use nginx as caching proxy with no conditional get from client

nginxPROXYreverse-proxy

I would like Nginx to be set up as a reverse proxy with caching in front of another web server (origin server). I understand how to do this much.

But what if I don't want to depend on the web browser to send an HTTP conditional get to take advantage of the caching? Instead, I want Nginx to make an HTTP conditional get to the origin server. If Nginx has stale content, it will update its cache with a new copy from the origin server.

The flow would be like this:

Browser (or other non-browser client) sends HTTP GET request to Nginx. (No conditional get headers).
Nginx sends HTTP GET for requested content to origin server INCLUDING a conditional get header (e.g. "If-Modified-Since").
If content has been modified, origin server responds with HTTP 200 and content that Nginx uses to update its cache. Otherwise, it responds with HTTP 304 "not modified", and Nginx does not update its cache.
Nginx responds with HTTP 200 and the requested content to the browser.

The rationale is to reduce the load on the origin server for HTTP requests from the browser that don't use a conditional get.

Best Answer

nginx added this as a new feature in development version 1.5.7.

Add proxy_cache_revalidate on; to your proxy cache configuration, and nginx will revalidate stale content with If-Modified-Since.

Related Solutions

Nginx – Disable caching when serving static files with Nginx (for development)

Since the answer is somehow hidden in the question - here is the solution for nginx in a VirtualBox environment as standalone answer.

In your nginx config (usally /etc/nginx/nginx.conf) or vhost config file change the sendfile parameter to off:

sendfile  off;

While sendfile is at the heart of Nginx's fame (blazing-fast low-level static file serving efficiency) it might be a bane for local development, e.g. Javascripts that change often and need to be reloaded. Nonetheless Nginx sendfile is smart and probably isn't most people's issue; check your browser's "disable cache" options as well!

Nginx – how to get nginx with proxy_pass and if_modified_since to return a 304-Not Modified

The problem is that nginx won't pass if-modified-since from the client through to the upstream server when caching is in effect (I believe it will depend on the presence of a proxy_cache setting). Nginx strips the if-modified-since header based on my investigation [1].

The solution is add proxy_set_header If-Modified-Since $http_if_modified_since to the front-end [2]. Fortunately, nginx doesn't try to be too smart here ... it faithfully replicates the header in the request to upstream, which is what we want.

The default Nginx behaviour makes sense if we're just caching static files, but if it's a more complex decision on the application server deciding whether to re-generate content, we need to pass it through!

In case anyone wants to look into this, I discovered this is happening by adding $http_if_modified_since to the nginx access log format, on the front-end server as well as the upsteam server. For a given request, the front-end server logged the if-modified-since header that had been sent by the client, but the same request was logged with an empty if-modified-since header until I applied the proxy_set_header fix.
As mentioned in Andrew Ty's comment on the question here.

Related Topic