In a multi server system, e.g. a load balancer, multiple web servers and a database server, where do you store the files / images users upload when each web server will need access?
Before now I've just used a single web server, or maybe one web server and a MySQL database server.
But if you have multiple web servers I guess you have a range of options for making sure resources like images and other user files are shared, such as:
- Storing them on the database server
- Or a server specifically for them
- Maybe mounting the same directory
- Or using a tool like rsync to synchronize the user uploads directories on each web server
Or maybe you have a different solution? or there's a de-facto way of doing this that I'm not aware of?
Best Answer
I would consider a few options, depending on several factors:
If it's not a huge volume of images, then a single fileserver accessible by all the webservers is the easiest route.
If the images are very small (much less than a megabyte...), a Key-value storage can be considered. (but note that a file system is a great and efficient key-value when the values are more than a few kilobytes)
If the content is read much more than modified (at least 100 or 1000 times more frequently), then replicated (via rsync or unison) can reduce lag time. But this advantage is unlikely to be significant. A fileserver is still a great option.
If the amount of images is more than what's convenient to store on a single server, then a distributed storage system allows for near-infinite growth. (ceph RADOS, moosefs, glusterFS, etc.)