Amazon EC2 504 Gateway Time-out

504amazon ec2amazon-web-servicestimeout

I have only a single amazon EC2 free-tier instance. It's hosting two Django sites that currently get almost no traffic, barely a handful of requests per day. The server is apache with mod_wsgi, and apache is configured with WSGIDaemonProcess, like

WSGIDaemonProcess mysite.com processes=4 threads=4 display-name=%{GROUP} user=djangoUser group=djangoUser python-path=/srv/mysite:/srv/mysite/venv/lib/python2.7/site-packages
WSGIProcessGroup mysite.com

for each of the two sites. The sites were previously hosted on linode with the same configs and setup without issue, so I'm kind of surprised to be having issues.

Very frequently (more than 50% of the time) when accessing the sites, I get a 504 Gateway Time-out and the access attempt does not even register in the apache error or access logs at all, so it's hard to debug.

I've seen similar threads here discussing elastic load balancing, but that is not my case. I'm not sure how to proceed to route out and resolve the issue.

Here is an arbritrary screenshot of top at a given time when a request is made:

top shot

EDIT

I eventually figured out this was a misconfigured fail2ban script that added my ip to an iptables blacklist for a finite time. My first request would work, but subsequent requests would be blocked by iptables until the time limit expired, leading the 504s.

Best Answer

EC2 t2.micro instances are horrible. There, I said it. Horrible. If you're running a non-interactive application, and you don't mind if a certain job takes 10 times longer to run than it should, then t2.micros can work OK. For any type of interactive web application, though, they're worthless.

My guess is that if you'd watch top during the periods where this is happening, you will see a high percentage of CPU steal and/or iowait. Sadly, the only thing you can do to remedy this is to upgrade to a larger instance.

If you do not need the more advanced features available to you in AWS, it's not a great solution as far as performance-per-dollar goes. Linode, DO, and other VPS providers handily out-perform EC2 instances of similar size.

Related Topic