Linux distributed file system over WAN advice

distributed-filesystemslinuxrhel6

I have a fairly simple (not really) requirement but I've looked at a few solutions and can't find a good solution. I've got a Red Hat EL 6 server environment in my co-location and office, and some linux and windows xp and win7 workstations my office.

The co-location and office do not have a high-bandwidth link. Let's say it's 5Mbps.

I have a collection of files which are processed by batch jobs at my co-location, but are viewed and edited by workstation users, i.e. the files need to be accessible at both locations in a read-write fashion.

Additionally, some individual files are huge – too big to wait for the file to transfer from the "other" location if needed on-demand. So a copy of each file should be immediately "synced" to the other location if created/changed.

I've looked at AFS, which is almost perfect, except that the distributed "replica" is always read-only, and re-synchronizing the read-only replicas happens after an administrative command, AFAICT.

I'd prefer, also, for each client (e.g. workstation) to not have to have a large cache, but rather have a server at each site which provides local (subnet) access to the shared FS (maybe re-shares the content using local NFS or samba…)

Is there any such fs available for linux rhel 6?

Best Answer

Based on the information above I think you have a similar situation to us. We have a main data center with a few remote offices that need offsite mirroring.

We are using rSync. It works very well.

RSync is a free software computer program for Unix and Linux like systems which synchronizes files and directories from one location to another while minimizing data transfer using delta encoding when appropriate. An important feature of rsync not found in most similar programs/protocols is that the mirroring takes place with only one transmission in each direction.

Here is a quick overview and guide http://www.cyberciti.biz/tips/linux-use-rsync-transfer-mirror-files-directories.html

You may also want to looking into what ubuntu is doing with their orchestra server. not a direct comparison for RHEL however it is just a compilation of other services rolled into a pre-installed package.

Thomas