How to deploy Node.js in cloud for high availability using multi-core, reverse-proxy, and SSL

Having just setup a project that is essentially identical to what you describe, I'll share my approach - no guarantees that it is 'the best', but it does work.

My server stack is

• Varnish (v3.0.2) - all interfaces, port 80
• Nginx (v1.0.14) - local interface, port 81
• Node.js (v0.6.13) - local interface, port 1337
• Operating system is CentOS 6.2 (or similar)

My Node.js app uses Websockets (sockets.io - v0.9.0) and Express (v2.5.8) - and is launched using forever. (The same server also has other sites on it - primarily PHP which use the same instances of Nginx and Varnish).

The basic intention of my approach is as follows:

• Single public port/address for both websocket and 'regular' data
• Cache some assets using Varnish
• Serve (uncached) static assets directly from nginx
• Pass requests for 'web pages' to nginx, and from their proxy to Node.js
• Pass web socket requests directly (from Varnish) to Node.js (bypass nginx).

Varnish config - /etc/varnish/default.vcl:

#Nginx - on port 81
backend default {
  .host = "127.0.0.1";
  .port = "81";
  .connect_timeout = 5s;
  .first_byte_timeout = 30s;
  .between_bytes_timeout = 60s;
  .max_connections = 800;
}
#Node.js - on port 1337
backend nodejs{
  .host = "127.0.0.1";
  .port = "1337";
  .connect_timeout = 1s;
  .first_byte_timeout = 2s;
  .between_bytes_timeout = 60s;
  .max_connections = 800;
}

sub vcl_recv {
    set req.backend = default;

    #Keeping the IP addresses correct for my logs
    if (req.restarts == 0) {
        if (req.http.x-forwarded-for) {
            set req.http.X-Forwarded-For =
            req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }

    #remove port, if included, to normalize host
    set req.http.Host = regsub(req.http.Host, ":[0-9]+", "");

    #Part of the standard Varnish config
    if (req.request != "GET" &&
      req.request != "HEAD" &&
      req.request != "PUT" &&
      req.request != "POST" &&
      req.request != "TRACE" &&
      req.request != "OPTIONS" &&
      req.request != "DELETE") {
        /* Non-RFC2616 or CONNECT which is weird. */
        return (pipe);
    }
    if (req.request != "GET" && req.request != "HEAD") {
        /* We only deal with GET and HEAD by default */
        return (pass);
    }

    #Taken from the Varnish help on dealing with Websockets - pipe directly to Node.js
    if (req.http.Upgrade ~ "(?i)websocket") {
        set req.backend = nodejs;
        return (pipe);
    }

    ###Removed some cookie manipulation and compression settings##


    if(req.http.Host ~"^(www\.)?example.com"){
            #Removed some redirects and host normalization
            #Requests made to this path, even if XHR polling still benefit from piping - pass does not seem to work
        if (req.url ~ "^/socket.io/") {
            set req.backend = nodejs;
            return (pipe);
        }

    #I have a bunch of other sites which get included here, each in its own block
    }elseif (req.http.Host ~ "^(www\.)?othersite.tld"){
        #...
    }

 #Part of the standard Varnish config
 if (req.http.Authorization || req.http.Cookie) {
        /* Not cacheable by default */
        return (pass);
    }

    #Everything else, lookup
    return (lookup);
}


sub vcl_pipe {
    #Need to copy the upgrade for websockets to work
    if (req.http.upgrade) {
        set bereq.http.upgrade = req.http.upgrade;
    }
    set bereq.http.Connection = "close";
    return (pipe);
 }
 #All other functions should be fine unmodified (for basic functionality - most of mine are altered to my purposes; I find that adding a grace period, in particular, helps.

Nginx config - /etc/nginx/*/example.com.conf:

server {
    listen *:81;
    server_name example.com www.example.com static.example.com;
    root /var/www/example.com/web;
    error_log /var/log/nginx/example.com/error.log info;
    access_log /var/log/nginx/example.com/access.log timed;

    #removed error page setup

    #home page
    location = / {
        proxy_pass http://node_js;
    }

    #everything else
    location / {
        try_files $uri $uri/ @proxy;
    }
    location @proxy{
        proxy_pass http://node_js;
    }

    #removed some standard settings I use
}

upstream node_js {
    server 127.0.0.1:1337;
    server 127.0.0.1:1337;
}

I am not particularly crazy about the repetition of the proxy_pass statement, but haven't gotten around to finding a cleaner alternative yet, unfortunately. One approach may be to have a location block specifying the static file extensions explicitly and leave the proxy_pass statement outside of any location block.

A few settings from /etc/nginx/nginx.conf:

set_real_ip_from 127.0.0.1;
real_ip_header X-Forwarded-For;

log_format  timed  '$remote_addr - $remote_user [$time_local] "$request" '
                   '$status $body_bytes_sent "$http_referer" '
                   '"$http_user_agent" "$http_x_forwarded_for" '
                   '$request_time $upstream_response_time $pipe';

port_in_redirect off;

Among my other server blocks and settings, I also have gzip and keepalive enabled in my nginx config. (As an aside, I believe there is a TCP module for Nginx which would enable the use of websockets - however, I like using 'vanilla' versions of software (and their associated repositories), so that wasn't really an option for me).

A previous version of this setup resulted in an unusual 'blocking' behaviour with the piping in Varnish. Essentially, once a piped socket connection was established, the next request would be delayed until the pipe timed out (up to 60s). I haven't yet seen the same recur with this setup - but would be interested to know if you see a similar behaviour.

You have two server blocks on port 81 with manager.domain.com configured as a host header - nginx has no way to know whether a request went through stunnel or not, so the first one wins and the redirect always occurs.

I'd recommend either having Varnish do the redirecting based on whether the request came from stunnel or not (check the request's client.ip - 127.0.0.1 means it's from stunnel), or have Varnish mark the requests from stunnel with a header so that nginx can decide how to handle them.