Linux – Should I use ec2 as a file server

amazon ec2amazon s3file-serverlinuxrsync

I need to be able to share user uploaded content across multiple EC2 application servers. I have looked at rsync, mounted NFS, and S3 as potential options of being able to share this data in almost real-time. The uploaded and downloaded user files are almost always between 1-10MB. Some are accessed a lot and some just once and then deleted.

My newest approach involves launching an EC2 instance strictly as a file server, separate from the application servers. With this option, for a user to download a file, they are connected to one of the application servers which queries the database with data about the file they wish to download. The user is then prompted to download which connects them to the file server for download.

I feel like this option will be faster than my other options. The only downside that I see is that I can't autoscale up/down file servers. I can however scale up and make a column in the database that says which file server the file is located on.

Is this a good approach or am I missing something?
Also, what is a good way to determine how many concurrent uploads/downloads can occur on the file server based on server specs and with files being between 1-10MB or is that something best determined from load testing?

Also in terms of scaling, will it be a problem if 1 particular file located on just 1 file server becomes extremely popular? Would using a CDN solve this problem?

Best Answer

A CDN would be the better option for you, using S3 with CloudFront would. My recommendation would be to decentralized the user generated content from the application server(s), keeping your servers volatile when scaling up or down within you architecture is a good design practice.

Related Topic