Linux – Error too many redirects in nginx + varnish+ SSL

httpslinuxnginxsslvarnish

I have a clustered infrastructure. I use nginx for SSL termination in front of varnish. The backends of varnish are apache web servers.and I also have a haproxy as a load balancer which sends HTTPS requests directly to nginx and sends HTTP requests directly to varnish servers. The problem is that when I start nginx everything is OK for some while, but after that I get the too_many_error_redirects in browsers when browsing ssl websites!! I think there is something wrong with my configurations, but I don't know which configurations (nginx or varnish) are the cause of this error. When I forward requests directly to the webservers everything is OK, so may be the varnish config has problem. here are my configurations:
Nginx config: domain_name.conf

server {
        listen 443;

        server_name mydomain.com;
        ssl on;

        ssl_certificate /etc/nginx/ssl/domain_name_bundle.pem;
        ssl_certificate_key /etc/nginx/ssl/my_key.key;

        ssl_session_cache shared:SSL:20m;
        ssl_session_timeout 10m;

        ssl_prefer_server_ciphers       on;
        ssl_protocols                   TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS +RC4 RC4";

        add_header Strict-Transport-Security "max-age=31536000";
        server_tokens off;
        proxy_pass_header Server;
location / {
            proxy_pass http://cache-servers;
            proxy_set_header X-Real-IP  $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto https;
            proxy_set_header X-Forwarded-Port 443;
            proxy_redirect    off;
            proxy_set_header Host $host;
        }
}
upstream cache-servers
{
        ip_hash;
        #cache servers
        server 192.168.1.11:8080;
        server 192.168.1.12:8080;
}

Varnish config:

vcl 4.0;

import directors;

# Check backend health
probe backend_healthcheck {
   .url = "/";
   .timeout = 10s;
   .window = 5;
   .threshold = 3;
   .interval = 5s;
   .expected_response = 200;

}

backend web1 {
    .host = "192.168.1.105";
    .port = "8080";
    .probe = backend_healthcheck;
}
backend web2 {
    .host = "192.168.1.106";
    .port = "8080";
    .probe = backend_healthcheck;
}
sub vcl_init {
    new apache = directors.round_robin();
    apache.add_backend(web1);
    apache.add_backend(web2);
}
sub vcl_recv {

    set req.backend_hint = apache.backend();

set req.http.X-Forwarded-For = client.ip;

if (req.method == "GET" && (req.url ~ "^/?mylogout=")) {
     unset req.http.Cookie;
      return (pass);
  }
  #we should not cache any page for Prestashop backend
  if (req.method == "GET" && (req.url ~ "^/admin70")) {
      return (pass);
  }
  #we should not cache any page for customers
  if (req.method == "GET" && (req.url ~ "^/authentification" || req.url ~ "^/my-account")) {
      return (pass);
  }
  #we should not cache any page for customers
  if (req.method == "GET" && (req.url ~ "^/identity" || req.url ~ "^/my-account.php")) {
      return (pass);
  }
  #we should not cache any page for sales
  if (req.method == "GET" && (req.url ~ "^/cart.php" || req.url ~ "^/order.php")) {
      return (pass);
  }

#we should not cache any page for sales
  if (req.method == "GET" && (req.url ~ "^/addresses.php" || req.url ~ "^/order-detail.php")) {
      return (pass);
  }
  #we should not cache any page for sales
  if (req.method == "GET" && (req.url ~ "^/order-confirmation.php" || req.url ~ "^/order-return.php")) {
      return (pass);
  }
if (req.method != "GET" && req.method != "HEAD") {
      return (pass);
  }

#pass feeds
  if (req.url ~ "/feed")
  {
        return (pass);
  }

if (req.url ~ "/wp-(login|admin)" || (req.method == "GET" && req.url ~ "^/admin") || (req.method == "GET" && req.url ~ "^/user"))
  {
        #unset req.http.cookie;
        return (pass);
  }

  #cache everything left behind
  return(hash);
}

sub vcl_backend_response {

    if  (!(bereq.url ~ "(wp-(login|admin)|admin)"))  {
      unset beresp.http.set-cookie;
      set beresp.ttl = 10m;
     }
    set beresp.grace = 2h;

}

sub vcl_deliver {

    if (obj.hits > 0) {
           set resp.http.X-Cache = "HIT";
    } else {
           set resp.http.X-Cache = "MISS";
    }
 if (obj.hits > 0) {
           set resp.http.X-Cache-Lookup = "HIT";
    } else {
           set resp.http.X-Cache-Lookup = "MISS";
    }
 unset resp.http.X-Varnish;
 unset resp.http.Via;
 unset resp.http.Server;
 unset resp.http.X-Powered-By;
#return (deliver);

}

Best Answer

The problem here is very common: you're doing redirect http -> https at application level.

Apache/PHP don't know they are already running in SSL, as apache is running in http (then passing to varnish, then passing to nginx).

The solution is simple: your PHP Application needs to have the PHP ENV variable $_SERVER['HTTPS'] = "on".

You can do in different ways:

in httpd.conf using apache SetEnv,
in .htaccess using again SetEnv
in your php scripts.

Plus: I would also do a redirect from http to https at varnish level: add a custom header like X-Nginx = on when request come from your nginx. In varnish read that header, if it doesn't exist, then redirect the user to https URL.

NOTE: If you're using wordpress (as it seems in your vcl file) don't forget to add this:

define('FORCE_SSL_ADMIN', true);

Related Solutions

Nginx – Configure php5-fpm for many concurrent users

I suspect your varnish cache is not caching anywhere near enough of the hits

here's what I would do in your situation:

Lower php max children to 100 or even 50 (if varnish does its job properly you don't need them) also remove the max requests line to allow the php processes not to respawn too quickly and thus prevent APC from being cleared too quickly which is also bad

also IF is not good according to nginx - http://wiki.nginx.org/IfIsEvil

I would change this line:

            if (!-e $request_filename) {
                    rewrite ^(.+)$ /index.php?q=$1 last;
            }

to:

try_files $uri $uri/ /index.php?$args;

If your version of nginx supports it (pretty certain if your nginx version is > 0.7.51 then it supports it)

you should also look at inserting the w3tc nginx rules direct into your vhost file to enabled proper disk enhanced caching of pages (which is faster than APC caching with nginx)

Take a look at the following varnish vcl which I use for sites - you will need to read through and edit a few things for your website - it also assumes that its only WP sites on the server and only 1 site on the server, it can easily be modified for more sites (take a look at the cookie section)

generic vcl: https://gist.github.com/b7332971a848bcb7ecef

With this config I would argue to remove fastcgi_cache to prevent any possible issues with a cache-chain occurring whereby trying locate any stray stale cache entries is more difficult

also tell w3tc that varnish is at 127.0.0.1 and it will purge it for you ;)

I deployed this to a server on Wednesday evening (with a few domain specific modifications) that was handling 2500 active site visitors it reduced load to less than 1 and the approx number of running php children was around 10-20 (this number does depend on number of logged in users and other factors like cookies) this server did have much more ram but the principle is the same, you should be able to easily handle the number of visitors you get at peaks

Nginx – how to disable varnish X Forwarded For header

The default vcl_recv function (which is appended to yours) contains this:

 if (req.restarts == 0) {
   if (req.http.x-forwarded-for) {
       set req.http.X-Forwarded-For =
           req.http.X-Forwarded-For + ", " + client.ip;
   } else {
       set req.http.X-Forwarded-For = client.ip;
   }
 }

..which is modifying the header. To prevent this from happening, you should have your vcl_recv implemented as a full function that always returns, instead of depending on the appending of the default behavior, which contains config that you don't want. Something like this:

sub vcl_recv {
    if (req.http.Range) {
      return(pipe);
    }
    if (req.request != "GET" &&
      req.request != "HEAD" &&
      req.request != "PUT" &&
      req.request != "POST" &&
      req.request != "TRACE" &&
      req.request != "OPTIONS" &&
      req.request != "DELETE") {
        /* Non-RFC2616 or CONNECT which is weird. */
        return (pipe);
    }
    if (req.request != "GET" && req.request != "HEAD") {
        /* We only deal with GET and HEAD by default */
        return (pass);
    }
    if (req.http.Authorization || req.http.Cookie) {
        /* Not cacheable by default */
        return (pass);
    }
    return (lookup);
}

Best Answer

Related Solutions

Nginx – Configure php5-fpm for many concurrent users

Nginx – how to disable varnish X Forwarded For header

Related Topic