Nginx – Reverse Proxy Server Cloud Architecture (AWS + nginx)

amazon-elbamazon-web-servicesload balancingnginxreverse-proxy

Currently I cache responses using nginx + fastcgi. I'm in the process of moving my app to AWS. I'm architecting a horizontally scaled approach, but trying to determine where best to put nginx + fastcgi cache.

I found a few solutions from my own research, both with significant drawbacks.

  • Use AWS Elastic Load Balancer to balance a cluster of web apps. Each web app runs it's own nginx and web server stack. Drawback: the cache on each server needs to be warmed. Worst case 1/n the hit rate.
  • Put nginx + fastcgi in front of the AWS ELB so the cache is centralized for maximum hits. Drawback: single point of failure – if nginx dies traffic doesn't pass thru to the ELB. Added nginx instance.

I'd appreciate any suggestions of common architectures that overcome the drawbacks outlined above.

Best Answer

I've set up this stack many times on AWS. Option #1 has always worked well for me and is the one I normally choose because it's the simplest. I'm curious to know how much traffic you're dealing with where the less-than-ideal initial cache hits are an issue? I'm serving a few million pageviews a month on a pair of m1.small instances and they're barely scratched.

Other thoughts:

  • Nginx can use memcached as a caching store. Plug it into an ElasticCache cluster. This will give you a very fast, centralized datastore for multiple instances to connect to for cache contents. (I've never done this, but it looks doable. Please post results if you do attempt it.)

  • The SPoF problem mentioned in Option #2 can be mitigated by setting up an autoscaling group that contains a single instance. If it dies, it should come back online in a matter of a couple of minutes. You will need to fiddle with a boot script that automatically grabs the Elastic IP though.

  • Keep in mind that putting anything in front of a ELB will require that you use VPC. This is generally good practice anyhow, but it can have a steep learning curve.

  • CloudWatch Alarm Actions were just announced today. You could do some cool stuff with this to help minimize the SPoF issue.

Related Topic