How to Block Nginx Access Log Statements from Specific User Agents

amazon-elbconfigurationloggingmonitoringnginx

I am looking to turn off logging in the Nginx access log file from specific requests from http user agents.

Basically from the Amazon ELB Health Check and our external (Pingdom) monitoring. Since these come every few seconds, makes testing hard to sort through the logs.

"GET / HTTP/1.1" 200 727 "-" "ELB-HealthChecker/1.0"
"GET /login HTTP/1.1" 200 7492 "-" "Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)"

I was able to block logging for image files, but have not seen anything for incoming requests:

location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico|xml|svg)$ {
        access_log        off;
        expires           30d;
}

Thanks in advance!

So I tried the recommendation from @Gnarfoz, but had some interesting side effects. While those two "Health Checks" were not logged, Pingdom started to recognize the server as DOWN even while it was up and running. That is interesting as the Load Balancer did not, which it would have dropped the node we were testing if it had.

I put the MAP section in the HTML block below my logs:

access_log /www/access.log;
error_log /www/error.log;

map $http_user_agent $ignore_ua {
            default                 0;
            "~Pingdom.*"            1;
            "ELB-HealthChecker/1.0" 1;
    }

And I put the IF statement in my server block, with the default location:

location / {
                try_files $uri $uri/ /index.php?$args;

                if ($ignore_ua) {
                       access_log off;
                }
        }

When I did this, Pingdom started to generate 404 errors in the Error Log file:

2012/08/03 17:10:33 [error] 6250#0: *164 open() "/www/public_html/login" failed (2: No such file or directory), client: 10.xx.xx.xx, server: xxx.com, request: "GET /login HTTP/1.1", host: "xxx.com"
2012/08/03 17:11:32 [error] 6250#0: *240 open() "/www/public_html/login" failed (2: No such file or directory), client: 10.xx.xx.xx, server: xxx.com, request: "GET /login HTTP/1.1", host: "xxx.com"

Best Answer

Try this:

# map goes *outside* of the "server" block
map $http_user_agent $ignore_ua {
    default                 0;
    "~Pingdom.*"            1;
    "ELB-HealthChecker/1.0" 1;
}

server {
    # Things omitted for brevity

    location / {
        if ($ignore_ua) {
            access_log off;
            return 200;
        }
    }
}

The if part would probably need to be integrated into your appropriate location block.

Relevant nginx documentation: map, if, access_log

Related Topic