Nginx performance when running multiple instances on the same machine

nginxoptimizationperformanceperformance-tuning

I'm trying to understand the performance impact of having multiple nginx instances (masters) running on the same machine, rather than having them all load into a single instance using different server blocks. How does the use of multiple instances of nginx impact things like worker_process and worker_connections optimization?

I see tons of advice indicating that worker_process should mirror the number of cores, and at most should be double the number of cores. I'm also to understand that the worker_connections should match the ulimit, or be a bit under the ulimit. Making too many connections available, or having too many workers per core is supposed to hurt performance.

I have two cores and a ulimit of 1024, but I have 4 instances of nginx each of which has the following settings:

worker_processes 4;
worker_connections: 1024;

Doesn't this have the same effect as if I had worker_processes 16; and worker_connections 4069;?

Note: Let me make clear when I say nginx instances, I mean that there are 4 independent master nginx process each fed a different config file which has similar settings, and each with their own workers.

Note 2: This scenario is something I've inherited and is already in place. I am trying to figure out if I should change the way nginx is configured and have an informed reason for it.

Best Answer

From a system point of view, there is no inherent differences in running 4 masters with 4 server sections, or a single master with 16 server sections. It implements the same architecture : parallelized event-based processes.

The worker/core ratio must be accounted wrt the total of workers across all of your masters if you have several ones. This comes from several constraints :

  • make sure CPUs are not overloaded, thus number of workers should be <= number of cores
  • make sure parallelization and OS scheduling is used to its best, thus number of workers should be as high as possible
  • HTTP server workers are low on CPU usage and mostly wait on I/Os, thus it'as actually safe to allocate something between 2 or 4x the number of cores

It should be a little more efficient with a single master, since a few ressources like MIME maps and so on will only be loaded once. But that's a minor point.

It should be more efficient with a single master, because there is a single large pool of workers shared by all servers. If a single server momentarily requires most workers (say 16), it might get them. On multi-master configuration (say 4 masters with 4 workers), they only get to use at most what they have : 4 workers. On the other hand it might be the desired effect : strictly split in 4 instances to make sure each one always get at least the quarter of your host attention. But never more.

It should be easier to configure and maintain with 1 master (think: security updates).

It should be more resilient with 4 masters : you're allowed to crash or totally mess one master configuration without touching the 3 others.

Unless your 4 masters use different Nginx versions, you won't benefit from uber-optimisations like having the exact set of modules compiled in for each master.

Related Topic