Linux to windows ssh/scp/sftp differential backup solutions

backuplinuxscpsshwindows

Here is the setup:

Large amounts of large files (many over 4gb, multiple TBs in total) in a simple tree structure that need to be backed up without compression from a handful of linux boxes to a windows backup server. Every day a few hundred gigs are generated and backups are currently done by hand with WinSCP. My goal is to automate the process and have good logging for monitoring. Write once, read rarely. (offsite will be LTO-4 or 5)

Here is the issue though, almost every solution I have tried has had a showstopping problem.

WinSCP scripting – does not handle symlinks properly, resulting in endless file transfer loops. Manual GUI works ok, (with some ignorable erros)

rdiff-backup for win – does not properly support existing files already on backup server, and doing over 20TB of backups from scratch is not a time/resource feasible option. (tried every stackexchange listed trick to recognize existing data without taking 8 years to generate checksums)

rsync via cygwin – Due to lack of differential, space requirements would quickly outpace our current capacity.

My next thought is to put an ssh server on the backup server (or FTP) and do the automation on the linux side through scp etc…though I would really like checksums and differential support.

Another option could be samba from the nix boxes to the backup server.

I would really appreciate any input or guidance on the matter. I'm open to suggestions, but am trying hard to avoid proprietary and closed source solutions. I'm not used to such mixed environments…

FOLLOW UP EDIT: So after messing around with every option under the sun, mounting windows via cifs and then using the backup program of your choice seems to be the most reliable way to go if you are going from nix to windows. (with a few caveats) That being said, I am only doing so as a stopgap measure until I can implement a ZFS based NAS. With NFS/CIFS/rsync all supported, there will be no problems moving from windows, mac, or linux to the NAS.

Best Answer

I use Backuppc on both Linux and Windows systems for differential incremental backups. For Windows, we just go the CIFS route, but one could do rsync via Cygwin/Gow.