Nginx – *2 connect() failed (111: Connection refused) while connecting to upstream

It sounds like you haven't started and configured the backend for Nginx. Start php-fpm and add the following to nginx.conf, in the http context:

server {
    listen 127.0.0.1;
    server_name localhost;

    error_log /var/log/nginx/localhost.error_log info;

    root /var/www/localhost/htdocs;

    location ~ \.php$ {
        fastcgi_pass 127.0.0.1:9000;
        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;

        fastcgi_intercept_errors        on;
        error_page 404 /error/404.php;
    }
}

Use unix sockets, instead of TCP/IP. So, you'd do something like:

listen = /tmp/php.sock

If you don't want to do something like that, check your firewall rules. If you're using any sort of conntrack rules, you could easily be filling up the table for that. See if there's any log messages in dmesg about it. You shouldn't really be using conntrack on a high-load server, so disabling that is probably a good idea.

Also, check your ulimit settings. If the number of open files is set too low, you could run into issues.