Nginx – ELB, nginx, and multiple HTTPS applications

amazon ec2amazon-elbnginxphp-fpm

Let's say I have 3 different applications that all do HTTPS. Not all of them are high traffic so I run all 3 on the same "pool" of 3 EC2 instances.

domain1.com/domain2.com/domain3.com all have their A records pointing to Amazon ELB, which forwards :443 and :80 to nginx:80 on all three servers (so yes, ELB does the SSL termination). Because ELB is doing the SSL termination, I have three different ELBs instead of just one. So basically:

              domain1.com  domain2.com  domain3.com
                  |             |            |
                  |             |            |
                  V             V            V
                 ELB1          ELB2        ELB3
                  \             |             /
                   \            |            /
                    \           |           /
                     \          |          /
                      \         |         /
                       \        |        /
                        \_______|_______/
                                |
                                |
                                V
                   ---------------------------
                   nginx      nginx      nginx
                     |          |          |
                     V          V          V
                  php-fpm    php-fpm    php-fpm

Each of the three instances runs both nginx and php-fpm. That way when the load is too high, we just boot up another instance from the same machine image and it just joins the pool. The instances themselves are configured so nginx only connects to php-fpm running on localhost.

I don't know if this is the correct way to be solving this scenario.

  • Should I move SSL termination down to nginx so that I don't need n number of ELBs as HTTPS certificates? How would that affect the cpu utilization?
  • Should I be using a single nginx machine that connects to multiple php-fpm backends? Then I need to worry about scaling two different roles instead of just one.
  • How does this change if I want to start up instances in other regions, for example?

I hope this question is not inappropriate for SF.

Best Answer

You could run something like this:

          domain1.com  domain2.com  domain3.com
              \             |             /
               \            |            /
                \           |           /
                 \          |          /
                  \         |         /
                   \        |        /
                    \_______|_______/
                            |
ngix (static pages from local storage, 3 IP addresses, SSL termination)
                            |
                            |
                       ELB (optional)
                            |
                            |
                 -----------------------
                 |          |          |
                 V          V          V
                php        php        php

  • It would be a little bit cheaper, as 1 EC2 reserved small instance + 1 ELB is cheaper than 3 ELB.

  • You could be even cheaper if you could ditch ELB and just use normal DNS round robin (but you'll loose some ELB features like automatic failover).

  • Latency on serving static pages would not be affected by load on php-serving instances.

  • PHP serving would not compete for RAM with static pages OS cache.

  • You could even use apache mod_php instead of php-fpm is you don't need user separation between processes - it should be somewhat faster.

  • It would be cheaper to add more pages to serve (add one more IP instead of one more ELB).

But it also has some drawbacks:

  • You'd have have one more single point of failure (ngix server).

  • More complicated setup.

Related Topic