I do have a 300TB Freenas server to backup several linux nodes. Backup works with daily snapshot and rsync tasks.
The user often move big datasets (2-5TB) between the server – so often big files are getting backed up several times on several server.
Online Deduplication would be too expensive (1,5TB RAM….), so is there any offline deduplication software out there?
I mean, the files do have the same name and often the same access times – fdupes would recognize them as identical with minimal effort…
Best Answer
Try BorgBackup - but you will have to use it instead of rsync.
Here: https://www.borgbackup.org/