I have a wireless network setup to provide about 100 users with internet access. There is no sharing of files or any communication between the machines. They want to have an automated backup of files on their machines to a central area where if they lost files they could recover them. The backups would need to be automated within the system to know when to go to the next user's machine once the prevous one is completed. The machines will be Windows or Mac. We could have two different systems if they are needed to be separate. What options are out there for this? I am not looking for a full machine backup but just files on their machines.
Backup files on 100 user machines to a central machine
backup
Related Solutions
It looks like the setpriority API on Mac OS X is supposed to be able to alter IO scheduling (see http://developer.apple.com/mac/library/documentation/Darwin/Reference/ManPages/man2/setpriority.2.html). I don't have any MacOS handy so I can't test that nice actually changes the IO priority.
On the Linux side, ionice is what you're looking for.
You can do very well with Bacula/Amanda. Hitting your requirements:
Revisions (SVN-style): a file has to be backed up each time it gets modified (and multiples versions of the same file can exist on the server, in fact they must)
Bacula and Amanda will grab a file each time it changes.
Scalability: if I attach an USB drive to the computer, I want it's data to be backed up as well (well... That on linux might be quite easy, simply backup all /media/ except cds and dvds, but for windows?)
Not bad on Unix (Just back up everything under / and it will grab the media), but probably not possible on Windows -- I believe you need to specify the drives you want to grab because the filesystem isn't a tree hierarchy under a specific root (there's a root for each drive).
That said, it's probably NOT a good idea (What if you attach a full 1TB drive to a machine being backed up? Your backups just ballooned).
Near real-time (~ 5 minutes at max) file backup: I lost a latex report and was hard to reconstruct it from scratch
Not happening -- You CAN specify a 5 minute backup window, but your logs will be filled with jobs being killed because there's already a duplicate running.
You can schedule nightly backups, or even every 12 hours without much trouble.
(Even Apple's Time Machine only does hourly backups... think about the largest file that may change and have to be shoved over the wire...)
No-Duplication: for instance if I backup the USB's disk content from 2 different computers, I do not want the data to be backed up twice (symlink instead of hard copy in worst case)
Bacula doesn't have deduplication at this time. Not sure about Amanda.
Manual restore / automatic restore: it's the same for me (simply not like described here below)
Restores are (and should be) a manual process. I have no idea what an "automatic restore" would look like (the backup server decides on its own to restore a file? :)
Maybe ability to remove / exclude large files from backups
You can include or exclude specific parts of the filesystem (down to file-level granularity) in Bacula.
Good logs
Database-backed lists of jobs and results, with the ability to write to log files, email, etc. in the event of errors.
BackupPC may also be able to hit these requirements (not certain - haven't used it) - other commercial backup solutions almost certainly can as well.
You may also want to consider tarsnap, though I'm not sure how the Windows support is.
Best Answer
You could probably accomplish this with CrashPlan's peer-to-peer backup feature:
http://www.crashplan.com/blog/consumer/thank-you-for-being-a-backup-friend
http://www.crashplan.com/consumer/compare.html
Put CrashPlan on all the PC's, including the PC that will contain the backups (it would need a large hard drive). Set the PC's to all back up to the backup machine.
It would be completely automatic.
The free version might do everything you need.