amazon-web-services
Hosting Magento, as everyone knows, isn't like hosting other PHP applications. How feasible is it to run Magento in an Amazon Web Services environment in 2013?
What magic combination of AWS services does it make sense to use with Magento? What levels are smart defaults for a "run of the mill" store? (yes, I know, there are no run of the mill stores)
Which ones (EBS?) should be avoided?
Any tips, tricks, deployment strategies to avoid weeks of pain getting this setup?
We run on multiple c1.medium instances that are auto-scaled with a m1.db.small RDS instance, cache.t1.micro ElastiCache server for full page caching and CloudFront CDN served from two S3 sources (one for media, one for skin/js for more concurrent downloads per session). We use S3FS for EC2 instances to connect to the S3s. In addition, we use Lightspeed (now called Warp) full page caching, some custom block caching and Fooman Speedster Advanced.
We recently made some changes and found with the S3 system you need to set some symbolic links to expedite the process of PHP's file_exists function when a file is not present in a directory for the skin folder. With that set up, we are getting page load times under 1 second (cached) and 2-3 seconds not cached.
I like the c1.medium because the cost (we usually run spot instances at <$0.07/hr...unless it's Christmas time). Also, the compute optimized helps with the speed of processing all of the requests. With memory levels set correctly in php.ini, we haven't run into memory problems on that instance.
You do more traffic than we do, but caching is still the key. If you want to spend a little more, the current gen c3.large instances look enticing. They offer good computing power, decent memory, and SSD storage!
Be sure to set up any file system caching (like the cache for S3FS) on a local ephemeral drive to speed disk reads (and save money). With the setup I described, the disk read shouldn't be the bottleneck because most data is coming from ElastiCache for the full page and CloudFront/S3 for media.
I also highly recommend using a profiler to find and reduce bottlenecks. That is how we lowered non-cached page speeds from 6 to 2 seconds because of the file_exists function and symlinks.
To answer the other part of your question...I'm not sure the VPC would help much, except to block bad bots/DNS attempts before they hit your servers. Unless you have a ton of bad traffic, that shouldn't make a huge difference.
Update 3/4/2017:
This reply gets a decent amount of traffic and I've been getting some messages asking for help. Since the original post was over three years ago, I don't remember all the details from back then. Here's my current setup (pretty much scroll through the AWS menu and add one or two of everything):
My current project is implementing scripts to ban IPs that are requesting too many pages or bad URLs (almost all of which are originating from Russia and Ukraine). These can be >1000 requests/minute and cause major problems on several levels. I'm looking at using something similar to this fail2ban to Network ACL project. Speaking of malicious requests...ALWAYS move your admin folder to a different location that /admin/ and set up /downloader/.htaccess to deny all but your IP. Those are the two main URLs that these guys hit and the server load of them nailing those pages can cripple your server set up fast.
I feel that this site is not really small, but nowhere near large. There are components that should be different depending on catalog size and traffic levels and the probability of traffic spikes. The nice part is that you can set something like this up for around $100/mo, then scale quickly and easily by looking at loads from different services and add capacity where it is needed.
You've been advised poorly.
AWS is nothing more than a set of virtual servers, running in a contended, shared environment. Coupled with the fact it runs older generation hardware, that has its resources crippled by the AWS hypervisor - its performance is less than stellar if speed is important.
What you need to understand is that AWS is an entirely multi-tenant environment - from the physical hardware, to the internal networking, to the external networking. The performance you see will be heavily influenced by activity from other users of AWS. So you could have moments of "normal" performance, followed by catastrophically slow performance. You could opt for enhanced/clustered networking - but it is still effectively a public network performing a balancing act with QoS/CoS to try and improve latency/bandwidth to some users whilst limiting it for others.
Also consider that you really have no idea what hardware your different instances are sat on. Some may benefit from newer generation CPU's, others significantly older - it is effectively pot luck as to what you get - and computationally how one instance may perform compared to another.
In terms of handling high traffic, you are talking scalability, and AWS is constrained there too. You can only effectively scale horizontally. The capacity to scale vertically is limited to 32 cores - which is going to cripple any efforts to scale vertically.
So at this point, horizontal scaling is your only mechanism. Once you start scaling horizontally, you've got to add in considerations for
... and worrying about the fact that with each additional instance you run, the risk of failure is amplifying by as much. So high availability must then be built into your service.
When scaling horizontally, you are also back to the common bottleneck of contented networking. So whilst you can spin up infinite instances for load balancing, web servers, db servers and cache servers - your baseline performance will be dominated by the low communication speed between said instances.
Once you dig into the costs of "the cloud", you'll suddenly realise that you're actually paying close to 2x or 3x as much you actually would for a dedicated server. Now this is for a reason, you can create/destroy instances on a whim - so you are paying a premium for the flexibility of reducing cost.
But your, and many others', main reason for looking to use cloud hosting is to have infrastructure that will support your growing business, this means that your business is growing (or plans to),
Ie. the plans are to outgrow the original VPS and you want infrastructure to support it
So now, the original premise of cost savings (but running only what you need) is effectively gone - and once your business has grown, and is using the extra resources that you wished for - you'll suddenly realise that you are actually paying about 3x as much as you would have been if you just opted for a dedicated box to begin with - not to mention, have significantly worse performance.
The very fact you are asking about setting up AWS is evidence enough that you shouldn't attempt this endeavour at all. At best on AWS, you'll get an average performance Magento store - at worst, you'll have a terrifically slow, unreliable store.
Don't expect any support from Amazon - not even email support. You are limited to community forums for help (which, if the worst is happening and you are losing sales (£££) by the minute, is costing your business significantly). You can opt for paid support, if you can afford the costs (read: not cheap) - but don't expect the person on the other end to be able to offer any application specific help for Magento.
There are plenty of capable Magento hosts out there that run specialised infrastructure designed to scale - and plenty of which who have experience with both high volume stores and large catalogue stores.
You would be mad to go it alone, especially if you think that your magic bullet resides in AWS's promise of infinite scalability.
Best Answer
I was hosting Magento on AWS in 2011 until 2013. Many of the things you gain from using cloud infrastructure rather than traditional dedicated or shared hosting are more relevantly described under the topic of DevOps, which are not exclusive to AWS but are more easily enabled through its use.
Minimal footprint will be 1 ELB, 2 EC2 webservers in seperate AZs, 1 multi-az RDS, Route53 hosted zone for the domain. Initially you can use sticky sessions on the ELB to keep session management simpler, but as your traffic increases you'll want to move media into a CDN (S3 & CloudFront) and sessions off of the individual machines.
Areas I haven't looked but still are promising: CloudFormation scripts for easier deployment of a Magento stack, offloading order creation via DynamoDB and worker queues for greater checkout throughput (someone has already started a project to do this via MongoDB at one of the hackathons recently), and setting up a multi-region presence via latency based routing with Route53.
I guess I'm kind of an evangelist for cloud. Specific to AWS, c3.large are a decent instance size for production webservers, but I'd start off with the smallest of each instance class and measure performance and scale up or optimize code as you see fit, which is why I refer everyone to xhgui constantly.