Linux – TTFB on AWS EC2 Instance is Very Slow

amazon ec2amazon-efsamazon-elblinuxWordpress

I'm very new to hosting/server administration and my employer ask me to use this chance to learn and find a way to update our current hosting setup.

Our current setup was one EC2 instance that hosted multisite, it was being manage through Cpanel.

I did some research on AWS and this was new setup I'm trying to implement. On a high level this is the setup: We have a Load Balancer to round robin between two EC2 instances, the files for the website(s) is on a Network File System. The instances are also connected to a Relational Database. I might be over my head on this, but if there's a better solution please let me know.

Mini question on Load Balancer:

My Load balancer is listening on port 80 and 443. I added multiple SSL Certification into the https listener and it should be able to determine which certificate through SNI. Both of them are forwarding it to the same target group which the health check protocol is http. Something doesn't seem to sit well with me for this. If traffic come from port 443 connection wouldn't I just forward it to one of instance through port 80 instead of 443? It seem to be working for a very simple website (it just redirect to another website) though, so I didn't put too much thought on it.

This is the test result from webpagetest.com. I'm not sure on how to fix the TTFB given the ideal time is < 100ms where mine is 3036ms.

I think it might be because the EC2 instance I'm using now is the micro (free tier), so it might just be hardware issue. But given I only have one website running right now, I don't think this is an issue?

Or it might be because I've set the virtual host wrong?
I've setup a virtual host configuration file like so:

<VirtualHost *:80>
ServerName purpose-built.com
ServerAlias www.purpose-built.com
DocumentRoot /var/www/html/efs-mount-point/purpose-built/public_html
ServerAdmin admin@purpose-built.com
<Directory /var/www/html/efs-mount-point/purpose-built/public_html>
AllowOverride All
</Directory>
</VirtualHost>

Any direction on how to tackle this issue would be appreciated.

Best Answer

So it turned out that my instance was at 90% steal time from the top command. Stopping the instance and rebooting it fixed the issue.

It's still slow, but is workable (down to 2.4 seconds)

Related Solutions

SSL redirect on EC2 instance behind SSL terminating load balancer

untested.. but I try this

RewriteCond %{HTTP:X-Forwarded-Proto} !https
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [L,R]

http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/TerminologyandKeyConcepts.html#x-forwarded-proto

AWS EC2 Instance Auto Scaling Confusion

Let's go over your questions.

So basically I want my original instance to be running at all times. Then when it starts going over capacity I want the Auto Scaling Group to start launching instances and the Load Balancer to distribute the load across them. Is my thinking here sound?

I'd say yes, but I do have a couple reservations. If I understand correctly, you've placed your "main" instance outside of the auto scaling group. Theoretically, that would ensure that auto scaling doesn't kill off your original instance. There are a couple of things I'd like to mention:

You're not making full use of the possibilities of Auto Scaling. Auto Scaling not only enables your setup to scale, but it can also ensure limits. If, for whatever reason, your "main" instance dies, your auto scaling policy won't come into action. If you keep your instance in an auto scaling group with a min-size of 1, Auto Scaling automatically replaces the failed instance.
When auto scaling, it's often best practise to treat your instances as being "disposable", because that's how you build resilient systems. Don't depend on one instance to always be available.
You can set the termination policy for your auto scaling group so that it always terminates the newest instances first. That would ensure your "main" instance will be kept (as long as it's healthy). My previous comment still applies though.

When I make code and data changes to my original instance, do I have to remake the image my Launch Configuration uses?

I'd say no, but that's more of a design issue. Your image should describe the state of your server, but it shouldn't be responsible for code distribution. Consider a situation where you'd have to update your application because of an urgent bug, but your servers are under high load. Does updating your main server, creating an AMI, updating your launch config and killing off your auto scaled servers so they can be respawned with the latest AMI sound like an attractive solution? My answer to that would be no (again). Look into source code version control and deployment strategies (I'm a Rails developer 60% of the time and use git and capistrano, for instance).

There are situations where your approach would work as well and there is a lot of middle ground here (I would recommend also looking into Chef and userdata scripts). I myself actually rarely use custom AMIs, thanks to Chef.

What needs to be down with DNS names and IPs? I'm currently using Route 53, do I make that point to my Load Balancer and that's it?

Basically, yes. You can select the loadbalancer(s) that should be attached to new instances when creating your auto scaling group. I haven't used the GUI for Auto Scaling yet, but I'm quite sure it's in there somewhere. If not, the CLI still supports it. Point your Route53 record to your ELB alias and that's basically it.

Response to additional questions (2014/02/23):

If you're developing using Rails, I can highly recommend Capistrano for deployments, which can take a specific version of your app in your preferred version control system (like git) and deploy it to a number of servers in a specific environment. There are a bunch of tutorials out there, but Ryan Bates' revised (and pro) Railscasts on the subject are very concise, although you need a subscription to his website to watch both of them. Most of the other tutorials will get you going as well though. Fire up a new server with your AMI and a launch script that pulls a temporary clone of your git repo and runs a local Capistrano command to get your app going. This ensures that, later on, you can also deploy new versions of your application using Capistrano with just one command to all running servers.

Capistrano also provides a couple of other benefits, including enabling you to execute specific tasks on all or just one of your servers and roll back a deployment, which is very hard to accomplish using just git.

Best Answer

Related Solutions

SSL redirect on EC2 instance behind SSL terminating load balancer

AWS EC2 Instance Auto Scaling Confusion

Related Topic