Linux – Synchronizing files between Linux servers, through FTP

ftplinuxPHPsynchronization

I have the following configuration of servers:

  • 1 central linux server, a VPS
  • 8 satellite linux servers, "crappy shared hostings"

I have a bunch of files that I need to have in all servers. Right now i'm copying them everywhere manually, but I want to be able to copy them to the central server, and then have a scheduled process that runs every now and then and synchronizes them (only outwardly, no need to try to find "new" files in the satellite servers).

There are a couple of catches though:

  • I can't have any custom software in the satellite servers, or do strange command line things that'll auto connect to them and send the files directly. I know this is the way these kinds of things are normally done, but the satellite servers are crappy shared hosting ones where I have absolutely no control over anything. I need to send the files over FTP

  • I also need to have, in my central server, a list of the files that are available in each of the satellite servers, to make sure they are ready before I send traffic to them.

If I were to do this manually, the steps would be:

  1. get the list of files in a satellite server
  2. compare to my own, and send the files that are missing
  3. get the list of files again, and store it in my central database.

I'd like to know what tools are out there that can alleviate as much of this as possible, first the syncing, and then the "getting the list of files available in the other server".
I'm going to be doing everything from PHP, not sure if there are good tools to "use FTP from PHP", which i'm pretty sure i'll have to do for step 3 at least.

Thanks in advance for any ideas!
Daniel

Best Answer

I think you want to look into mirror mode of lftp.

Rsync doesn't work over ftp (that would be the normal 'default' admin choice).

I strongly recommend you migrate to shared hosting that supports ssh terminal access which you can then use rsync over. By rolling out your own solution, you are only digging your hole deeper into this crappy (Your words) setup. I imagine the same amount of effort and time (if not less) to migrate to a better shared hosting provider.

Related Topic