Magento – Magento media folder and multiple web servers

it seems these guys find a solution: ( http://thinkglobal.co/resources/moving-the-magento-media-directory-to-s3/ )

There are 2 possible approaches, one being to have Magento push all media assets to S3 as they are created and the other being to simply mount the media directory to an S3 bucket and let Magento think that it’s writing to the local disk. We chose the latter.

We ended up using a tool called s3fs, which uses a broader technology called fuse. With some fairly straightforward configuration (described below), you can get s3fs up and running. Be careful not to hardcode S3 bucket names or API access credentials if you are using AWS or other PAAS provider.

Make sure you replace the following variables with values for your specific configuration:

• {{S3_BUCKET}} (name of your S3 bucket)
• {{API_PUBLIC_ACCESS_KEY}}(provided by AWS)
• {{API_SECRET_ACCESS_KEY}} (provided by AWS)
• {{UID}}(user id for nginx/apache user)
• {{GID}} (group id for nginx/apache user)
• {{MOUNTED_DIRECTORY}} (path to your magento media directory)

Here is the setup:

yum install -y gcc libstdc++-devel gcc-c++ curl curl* curl-devel libxml2 libxml2* libxml2-devel openssl-devel mailcap

cd /usr/local/src
wget http://downloads.sourceforge.net/project/fuse/fuse-2.X/2.9.3/fuse-2.9.3.tar.gz
tar -xzf fuse-2.9.3.tar.gz
rm -f fuse-2.9.3.tar.gz
mv fuse-2.9.3 fuse
cd fuse/
./configure –prefix=/usr
make
make install
export PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/lib64/pkgconfig/
ldconfig
modprobe fuse
pkg-config –modversion fuse

wget http://s3fs.googlecode.com/files/s3fs-1.74.tar.gz
tar -xzvf s3fs-1.74.tar.gz
rm -f s3fs-1.74.tar.gz
mv s3fs-1.74 s3fs
cd s3fs
./configure –prefix=/usr
make
make install

echo “{{S3_BUCKET}}:{{API_PUBLIC_ACCESS_KEY}}:{{API_SECRET_ACCESS_KEY}}” > ~/.passwd-s3fs
chmod 600 ~/.passwd-s3fs

echo “user_allow_other” > /etc/fuse.conf

echo “s3fs#{{S3_BUCKET}} {{MOUNTED_DIRECTORY}} fuse use_cache=/tmp,allow_other,uid={{UID}},gid={{GID}} 0 0″ >> /etc/fstab

s3fs -o allow_other -o uid={{UID}} -o gid={{GID}} -ouse_cache=/tmp {{S3_BUCKET}} {{MOUNTED_DIRECTORY}}

Having a quick look at the extension here are the benefits of using the extension rather than setting up CDN through web urls config. I've removed the marketing stuff...:

1. The extension comes with a tool for refreshing the files on the CDN instantaneously. Without our extension, you have to use a tool on the AWS website called "Invalidation Manager" to make the CDN pull updated files from your server. The invalidation manager is very slow and somewhat unreliable. The invalidation manager also only allows you to refresh 3 files every fifteen minutes... so it is quite time consuming to use it.

2. It prevents the CDN from being used on Javascript that requires same-origin loading. This is relevant to Javascript that is used for uploading files.

Having said that there also is a remark concerning S3 for this extension:

Currently it does not support S3 as the origin.

That being said, it uses your server as the origin in the same way that S3 works.

Basically, the extension caches your files on CloudFront and then serves them directly to your users from CloudFront. Every two hours the extension checks your server for a new version of the files and updates CloudFront accordingly.