Nginx – Enforcing No Gzip on Nginx

gzipnginxreverse-proxy

I need to disable gzip in some cases, regardless of client's Accept-* request headers.

My Nginx's conf entry has the following directives:

gzip off;

more_clear_input_headers -t 'text/html' Accept Accept-Encoding;

proxy_set_header Accept "";
proxy_set_header Accept-Encoding "";

However, the following curl:

curl '$URL' 
-H 'Pragma: no-cache' 
-H 'Accept-Encoding: gzip, deflate, sdch' 
-H 'Accept-Language: en-US,en;q=0.8,he;q=0.6' 
-H 'Upgrade-Insecure-Requests: 1' 
-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36' 
-H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' 
-H 'Cache-Control: no-cache' 
-H 'Cookie:  "$COOKIE"'
-H 'Connection: keep-alive'  -v > /dev/null

Results with:

< HTTP/1.1 200 OK
< Cache-Control: no-cache, no-store, private
< Content-Encoding: gzip
< Content-Type: text/html;charset=utf-8
< Date: Thu, 01 Sep 2016 07:13:30 GMT
< Expires: -1
< Server: Reblaze Secure Web Gateway
< Set-Cookie: ...
< Set-Cookie: ...
< Set-Cookie: ...
< Set-Cookie: ...
< Set-Cookie: ...
< Set-Cookie: ...
< Set-Cookie: ...
< Vary: User-Agent
< Content-Length: 55455
< Connection: keep-alive

Note, the responding server is proxied by this Nginx.

Best Answer

Apparently, this is not an Nginx "misbehaviour" rather, a proxied device, F5 BigIP in this case, which is configured to compress responses regardless of request headers, as long as the User-Agent header matches certain patterns.

Related Solutions

Nginx – Configure php5-fpm for many concurrent users

I suspect your varnish cache is not caching anywhere near enough of the hits

here's what I would do in your situation:

Lower php max children to 100 or even 50 (if varnish does its job properly you don't need them) also remove the max requests line to allow the php processes not to respawn too quickly and thus prevent APC from being cleared too quickly which is also bad

also IF is not good according to nginx - http://wiki.nginx.org/IfIsEvil

I would change this line:

            if (!-e $request_filename) {
                    rewrite ^(.+)$ /index.php?q=$1 last;
            }

to:

try_files $uri $uri/ /index.php?$args;

If your version of nginx supports it (pretty certain if your nginx version is > 0.7.51 then it supports it)

you should also look at inserting the w3tc nginx rules direct into your vhost file to enabled proper disk enhanced caching of pages (which is faster than APC caching with nginx)

Take a look at the following varnish vcl which I use for sites - you will need to read through and edit a few things for your website - it also assumes that its only WP sites on the server and only 1 site on the server, it can easily be modified for more sites (take a look at the cookie section)

generic vcl: https://gist.github.com/b7332971a848bcb7ecef

With this config I would argue to remove fastcgi_cache to prevent any possible issues with a cache-chain occurring whereby trying locate any stray stale cache entries is more difficult

also tell w3tc that varnish is at 127.0.0.1 and it will purge it for you ;)

I deployed this to a server on Wednesday evening (with a few domain specific modifications) that was handling 2500 active site visitors it reduced load to less than 1 and the approx number of running php children was around 10-20 (this number does depend on number of logged in users and other factors like cookies) this server did have much more ram but the principle is the same, you should be able to easily handle the number of visitors you get at peaks

NGINX Reverse Proxy – URL Rewrite

Any redirect to localhost doesn't make sense from a remote system (e.g. client's Web browser). So the rewrite flags permanent (301) or redirect (302) are not usable in your case.

Please try following setup using a transparent rewrite rule:

location  /foo {
  rewrite /foo/(.*) /$1  break;
  proxy_pass         http://localhost:3200;
  proxy_redirect     off;
  proxy_set_header   Host $host;
}

Use curl -i to test your rewrites. A very subtle change to the rule can cause nginx to perform a redirect.

Related Topic