Nginx – Why NGINX Returns 301 for Nonexistent Resources

nginx

Sometimes this kind of lines pop up in my access.log : 

149.28.156.181 - - [13/Sep/2018:20:35:09 +0100] "GET /js/czjl.js HTTP/1.1" 301 184 "-" "Mozilla/5.0 ..."

The resource /js/czjl.js does not exist on my server. However, NGINX answered with 301 instead of 404. If I try the same request with Postman I get 

192.168.1.67 - - [13/Sep/2018:21:09:34 +0100] "GET /js/czjl.js HTTP/1.1" 404 1140 "-" "PostmanRuntime/7.3.0"

which is the expected behavior.

Why is NGINX returning 301 instead of 404 if those resource aren't on my server ?

Here is my config 

server {
    listen 80;

    server_name example.com www.example.com;

    location ~ /.well-known {
        allow all;
    }

    location / {
        limit_except GET {
            deny all;     
        }
        return 301 https://www.example.com$request_uri;
    }

    location ~ /\. { deny all;}

}

server{
    #FOR INTERNAL REQUESTS

    listen 8080;

    server_name 192.168.1.75;
    error_page 401 403 404 /404.html;
    location = /404.html {
            root /var/www/html/error/;
            internal;
    }

    location / {
        limit_except GET {
            deny all;     
        }
    }

    root /var/www/html;
    index index.html;
}

server {
    listen 443 ssl;

    server_name example.com www.example.com;

    location / {
        limit_except GET {
            deny all;     
        }
    }

    root /var/www/html;
    index index.html;
    error_page 401 403 404 /404.html;
    location = /404.html {
            root /var/www/html/error/;
            internal;
    }

    #SSL Config
    ...
}

Best Answer

I had the same log message on my Apache web server. There's two sides to this: why is the HTTP 301 redirect occurring and why isn't there a subsequent HTTP request logged after the HTTP 301 redirect.

On my web server, I have forwarding enabled from http to https, along with subdomain forwarding from any subdomain to the correct subdomain. This means that any http request or any request on the wrong subdomain, whether to a valid resource or an invalid resource, would first go through an HTTP 301 redirect. For example, when I access http://<mydomain>/js/czjl.js, Apache logs an HTTP 301 due to forwarding the http request to https first, but accessing https://<mydomain>/js/czjl.js will log an HTTP 404.

The way the web call to http://<mydomain>/js/czjl.js is created will also determine whether solely an HTTP 301 is logged, or if an HTTP 301 is logged immediately followed by an HTTP 404. For example, using this wget command triggers an HTTP 301 in my web server logs without a subsequent HTTP 404:

wget --max-redirect=0 http://<mydomain>/js/czjl.js

If I leave out the --max-redirect=0 option, the logs display the HTTP 301 followed by an HTTP 404. However, by ignoring redirects, the wget request was fulfilled after getting the HTTP 301 response from the web server. I suspect these end-users are using a similar "no redirects" option in their web calls, which results in the occasional HTTP 301 to a non-existent resource. Otherwise, our web servers would have logged an HTTP 404 after the HTTP 301.

Related Topic