Nginx – DNS failure crashing web site – CURL requests

Create a separate location for /status, something like this:

    location /status {
        fastcgi_pass unix:/tmp/php5-fpm.sock;
        include fastcgi.conf;

        fastcgi_intercept_errors        on;
        error_page 404 /error/404.php;
    }

I think you are probably running far too many concurrent php processes, but it's hard to know without more info on where your resource bottlenecks are. I imagine that you are probably constrained by Disk IO and/or CPU, and that all your parallel PHP processes are competing for those and slowing each other down. At some point the overhead of process switching becomes a significant factor, and you get less throughput rather than more by having lots of processes running. You may also be getting into, or risking, situations where you run out of RAM and start swapping, which is very bad. Trust in nginx being able to queue requests up and keep a higher throughput of quicker requests going while doing less of them simultaneously.

I'd generally go for anything from 5 to 50 PHP processes, with both ends of that range being a little exceptional. More usually 10-15. With very high performance disk systems, and more than the usual 16 or so cores it might make sense to have more processes, but that's usually a false economy compared to having a larger number of cheaper servers. In my experience, unless you have a lot of really badly written code, there's usually little benefit to having more than about 15 php processes in parallel on a single server, and if there's a benefit it's likely to be stability rather than throughput, in the face of pathologically long-running requests piling up and leaving no spare processes available.

If you have multiple code bases with separate process pools, you might want a large number of processes, but you probably don't want more than 3 to 5 processes per pool.

You do want a lot of nginx worker connections handling static files. There's unlikely to be any improvement beyond 4096, and only in unusual circumstances would you see a difference between 1000 and 4000. (Unless you are primarily serving static files - that's quite a different scenario, but since you're talking about php processes on this box I don't imagine that's the case here).

I suspect your timeouts are too long. If there's nothing going on, drop the connection and get on to the next one.