Rate Limit Misconfigured Bots in Nginx

It is really better to avoid using the "if" directive. When the key in limit_req_zone (and limit_conn_zone) is empty the limits are not applied. You can use this in conjunction with the map and geo modules to create a whitelist of IPs where the throttle limits are not applied.

This example shows how to configure a limit for both concurrent requests and request rate from a single IP.

http {
    geo $whitelist {
       default 0;
       # CIDR in the list below are not limited
       1.2.3.0/24 1;
       9.10.11.12/32 1;
       127.0.0.1/32 1;
    }

    map $whitelist $limit {
        0     $binary_remote_addr;
        1     "";
    }

    # The directives below limit concurrent connections from a 
    # non-whitelisted IP address to five

    limit_conn_zone      $limit    zone=connlimit:10m;

    limit_conn           connlimit 5;
    limit_conn_log_level warn;   # logging level when threshold exceeded
    limit_conn_status    503;    # the error code to return

    # The code below limits the number requests from a non-whitelisted IP
    # to one every two seconds with up to 3 requests per IP delayed 
    # until the average time between responses reaches the threshold. 
    # Further requests over and above this limit will result 
    # in an immediate 503 error.

    limit_req_zone       $limit   zone=one:10m  rate=30r/m;

    limit_req            zone=one burst=3;
    limit_req_log_level  warn;
    limit_req_status     503;

The zone directives must be placed at the http level, however the other directives can be placed further down, e.g. at the server or the location level to limit their scope or further tailor the limits.

For futher information refer to the Nginx documentation ngx_http_limit_req_module and ngx_http_limit_conn_module

The issue was me assuming that the linode.com graph peaks were true peaks. It turns out the graph uses 5 minute average data points, thus my peak appeared to be 24mbits when in fact I was hitting the 50mbit cap.

Now that they have raised it to 100mbits, my benchmarks immediately went up to the new outbound traffic limit.

If only I had noticed that earlier! A lot of my reasoning hinged on the idea that I was not hitting an outbound traffic limit due to that graph.

Now, I peak at 370 requests per second, which is right under 100mbps at which point I start getting a "backlog" of requests, and response times start going up.

I can now increase maximum concurrency by shrinking the page; with gzip enabled I get 600RPS.

I do still run into problems when I suddenly peak and the backlog of pending requests (limited by bandwidth) start to accumulate, but that sounds like a different question.

It's been a great lesson in optimization / reading this data / narrowing down the possible problems. Thank you so much for your input!