I have an app farm which generates thumbnails and makes them available to a web server farm using nfs. However the performance is so poor so I am going to make a local copy of the files on each web node.
I looked at unison and lsync, but there doesn't seem to be a rpm available from the centOS5.5 repo for either.
which one should I use, or some other strategy. I critically need the sync to be immediate as the user needs to display the thumbnail straight away on a web page, so inotify based tools are good!
Edit: when I say immediate, I mean pushed out rather than pulled on some schedule.
In the end, lsync have released some more versions so I used that, and lsync works well once it is running (ie syncs in a few seconds) however it needs to enumerate each directory beforehand and for a very large file-system takes many hours..
Best Answer
So you've no control over server affinity in your load balancing? Time to get a new load balancer.
Previously I did this by building replication into the application - after all, it's the first thing to know when new content needs to be replicated - it was just a few lines of PHP to publish a notice to the other nodes that content needed fetching - when a node has been offline for a while, I ran rsync before bringing it completely back online (this approach makes handling offline nodes much simpler and non-intrusive). However if I were to implement such a system today then I do it using overlay filesystems - perhaps an NFS at the bottom of the stack and copy up on access.
See also this post
HTH
C.