Nginx – How to update Nginx reverse proxy cache by backend’s ETag/Last-Modified field

nginxreverse-proxy

I'm trying to get my Nginx reverse proxy work properly. However, I found that once a file is cached, it will never update/revalidate even the backend file is changed, which can be identified by ETag or LAST-Modified field. Can someone please kindly help me regarding to this issue?

Here is my setting:

proxy: proxy.test.com
backend: back.example.com

nginx settings:

proxy_cache_path /home/cache levels=1:2 keys_zone=cache_rev:16m inactive=14d max_size=12g;
proxy_cache_key "$scheme://$host$request_uri";
server {
    listen 80 default_server;

    resolver 74.82.42.42;
    server_name proxy.test.com;

    location ~ ^/ {
        proxy_pass http://back.example.com$request_uri;
        proxy_redirect off;
        proxy_buffering on;
        proxy_cache_revalidate on;
        proxy_pass_header Set-Cookie;

        proxy_cache cache_rev;
        proxy_cache_use_stale  error timeout invalid_header updating http_500 http_502 http_503 http_504;
        proxy_cache_valid 200  1d;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        add_header X-Cache-Status $upstream_cache_status;
    }
}

Here is the curl informationbefore and after backend file changed.

Before the change of index.html :

$ curl -I http://proxy.test.com/index.html
HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
Date: Sat, 12 Dec 2015 14:30:18 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 18283
Connection: keep-alive
Last-Modified: Sat, 12 Dec 2015 14:09:14 GMT
ETag: "261e044-476b-526b3fc1b6983"
Content-Language: zh-TW
X-Cache-Status: HIT
Accept-Ranges: bytes

$ curl -I http://back.example.com/index.html
HTTP/1.1 200 OK
Date: Sat, 12 Dec 2015 14:30:32 GMT
Server: Apache
Last-Modified: Sat, 12 Dec 2015 14:30:32 GMT
ETag: W/"261e044-476d-526b44eb57c31"
Accept-Ranges: bytes
Content-Length: 18285
Content-Type: text/html; charset=utf-8

After the change of index.html:

$ curl -I ttp://proxy.test.com/index.html
HTTP/1.1 200 OK
Server: nginx/1.4.6 (Ubuntu)
Date: Sat, 12 Dec 2015 14:30:54 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 18283
Connection: keep-alive
Last-Modified: Sat, 12 Dec 2015 14:09:14 GMT
ETag: "261e044-476b-526b3fc1b6983"
Content-Language: zh-TW
X-Cache-Status: HIT
Accept-Ranges: bytes

$ curl -I http://back.example.com/index.html
HTTP/1.1 200 OK
Date: Sat, 12 Dec 2015 14:30:53 GMT
Server: Apache
Last-Modified: Sat, 12 Dec 2015 14:30:53 GMT
ETag: W/"261e044-476b-526b450fe6a03"
Accept-Ranges: bytes
Content-Length: 18283
Content-Type: text/html; charset=utf-8

Best Answer

Etags alone will not help you. Etag is used for conditional requests by the client(browser), to control its own cache.

Nginx will not hit the upstream if it finds the resource in its cache. The point of a reverse proxy is to speed things up by not connecting to the upstreadm every time. You should either set the Expires: DATE/TIME or Cache-control: max-age=XXX headers (or better both to be sure) on the upstream server to let the cache know for how long it can keep the files cached. For dynamic content its best to use cache-control: no-cache.

Check these links for more detailed explanation on how HTTP/1.1 caching works:

If you want to explicitly remove something from the cache you'll have to implement a purge mechanism to delete the files you want to be updated.

Related Topic