Nginx – No Last-Modified or ETag Response Header on HTML

cachehttp-headersnginxWordpress

I am facing a rather odd issue with my web server. The server is configured with Ubuntu 20.04, Nginx v1.22.1, PHP 8.0, and MariaDB to host my WordPress website.

The issue is, the server is not sending any Etags or last-modified response headers for HTML pages, leading the browser to always get a 200 OK response, and never 304 Not Modified. I have configured browser caching with the ngx_http_headers_module module.

What's odd here is that both the Etags and last-modified headers, as well as the cache control headers are working flawlessly on all static assets, including JS, CSS, and images. But nothing for the HTML.

Below is my main config file for reference:

user jay;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 1024;
    multi_accept on;
}

http {

    ##
    # Basic Settings
    ##

    sendfile on;
    keepalive_timeout 60;
    tcp_nopush on;
    types_hash_max_size 2048;
    server_tokens off;
    client_max_body_size 64M;

    # server_names_hash_bucket_size 64;
    # server_name_in_redirect off;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ##
    # SSL Settings
    ##

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;

    ##
    # Logging Settings
    ##

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    ##
    # Gzip Settings
    ##

    gzip on;

    # gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 5;
    # gzip_buffers 16 8k;
    # gzip_http_version 1.1;
    gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    ##
    # Security
    ##
    add_header Strict-Transport-Security "max-age=31536000; includeSubdomains" always;
    add_header X-Xss-Protection "1; mode=block" always;
    add_header X-Frame-Options "SAMEORIGIN" always;
    add_header X-Content-Type-Options "nosniff" always;
    add_header Referrer-Policy "strict-origin-when-cross-origin" always;

    ##
    # Virtual Host Configs
    ##

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;

    server {
        listen 80 default_server;
        listen [::]:80 default_server;
        server_name _;
        return 444;
    }
}

And below is my site-level config:

# Expires map
map $sent_http_content_type $expires {
    default                    off;
    text/html                  epoch;
    text/css                   max;
    application/javascript     max;
    ~image/                    max;
    ~font/                     max;
}

server {
    listen 443 ssl http2;

    server_name www.example.com;

    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

    access_log /home/jay/example.com/logs/access.log;
    error_log /home/jay/example.com/logs/error.log;

    expires $expires;

    root /home/jay/example.com/public/;
    index index.php;

    location / {
        try_files $uri $uri/ /index.php?$args;
    }

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass unix:/run/php/php8.0-fpm.sock;
        fastcgi_index index.php;
        include fastcgi_params;
    }
}

server {
    listen 443 ssl http2;

    server_name example.com;

    ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

    return 301 https://www.example.com$request_uri;
}

server {
    listen 80;

    server_name example.com www.example.com;

    return 301 https://www.example.com$request_uri;
}

What I have done so far:

  • Eliminated the possibility of CloudFlare interfering with the response by completely removing it.
  • Removed all WordPress plugins as well as themes. Shifted to the default one.
  • Re-setup a new VPS with the same configuration.

But all in vain.

Can somebody please help? What could be causing this?


EDIT:

Here's the output for curl on the HTML:

HTTP/2 200 
server: nginx
date: Thu, 05 Jan 2023 14:59:14 GMT
content-type: text/html; charset=UTF-8
link: <https://www.example.com/wp-json/>; rel="https://api.w.org/"
x-fastcgi-cache: HIT
strict-transport-security: max-age=31536000; includeSubdomains
x-xss-protection: 1; mode=block
x-frame-options: SAMEORIGIN
x-content-type-options: nosniff
referrer-policy: strict-origin-when-cross-origin

The output stays the same even after repeated curl.

Below is the output for a static file, such the site's CSS:

HTTP/2 200 
server: nginx
date: Thu, 05 Jan 2023 15:01:50 GMT
content-type: text/css
content-length: 22256
last-modified: Wed, 04 Jan 2023 16:37:20 GMT
etag: "63b5ab40-56f0"
expires: Thu, 31 Dec 2037 23:55:55 GMT
cache-control: max-age=315360000
strict-transport-security: max-age=31536000; includeSubdomains
x-xss-protection: 1; mode=block
x-frame-options: SAMEORIGIN
x-content-type-options: nosniff
referrer-policy: strict-origin-when-cross-origin
accept-ranges: bytes

Best Answer

Welcome to ServerFault.

As per https://nginx.org/r/etag , etag can be enabled only for static content. Not for dynamic HTML content. Something similar could be applicable for "last-modified" headers, I believe. Anyway, if your aim is to add expires headers, then there are other ways to achieve this, such as using a full page caching plugin where the static HTML is generated for each post / page. Nginx goes through the config in a phased manner, expires may be applied only on static resources. I may be wrong, though. Might need to use debug to understand when expires is applied to a request.

Do you have any idea if it would be possible to add these same response headers on HTML documents cached by FastCGI cache?

No. There is no way to configure "etag" or "last-modified" headers for content cached by FastCGI cache.

If your aim is only to configure "expires", then you may use expires in the php location block like the following...

location ~* \.php$ {
    fastcgi_split_path_info ^(.+\.php)(/.*)$;
    if (!-f $document_root$fastcgi_script_name) { return 404; }

    # Mitigate https://httpoxy.org/ vulnerabilities
    fastcgi_param HTTP_PROXY "";

    include "fastcgi_params";
    fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
    fastcgi_index               index.php;
    fastcgi_pass                fpm;

    fastcgi_cache PHP_CACHE;
    fastcgi_cache_valid 200 10m;
    expires 10m;
    add_header "Cache-Control" "must-revalidate";
    add_header X-Cache $upstream_cache_status;

}

Here, "expires" works irrespective of FastCGI Cache. So, the above code will send cache-control: max-age=600 header to all PHP requests including to example.com/wp-login.php (even though it wouldn't be cached by FastCGI Cache). So, you may configure such URLs (/wp-admin/, /wp-comment-post.php, etc) in a different location block/s.

Related Topic