Since Windows Server 2008 R2 (up to and including Server 2019, as far as I know), Windows Server Backup performs automatic management of full and incremental backup:
Automatic management of full and incremental backups. You no longer need to manage full and incremental backups. Instead, Windows Server Backup will, by default, create an incremental backup that behaves like a full backup. You can recover any item from a single backup, but the backup will only occupy space needed for an incremental backup. In addition, Windows Server Backup does not require user intervention to periodically delete older backups to free up disk space for newer backups—older backups are deleted automatically.
This sounds like a nice feature.
We, however, use two backup disks: One is always attached to the server for daily backups, and one is always in off-site storage. Every week we switch those disks, to ensure that we always have an off-site server backup that is as most a week old.
How do those incremental backups work with alternating disks?
Obviously, this would be fine (Option 1):
Day 1: Full backup on Disk A.
Day 2: Incremental backup (w.r.t. Day 1 backup) on Disk A.
Day 3: Incremental backup (w.r.t. Day 1+2 backup) on Disk A.
...
Day 8: Full backup on Disk B.
Day 9: Incremental backup (w.r.t. Day 8 backup) on Disk B.
Day 10: Incremental backup (w.r.t. Day 8+9 backup) on Disk B.
...
Day 15: Incremental backup (w.r.t. Day 1-7 backup) on Disk A.
Day 16: Incremental backup (w.r.t. Day 1-7+15 backup) on Disk A.
But this would not be fine (Option 2):
Day 1: Full backup on Disk A.
Day 2: Incremental backup (w.r.t. Day 1 backup) on Disk A.
Day 3: Incremental backup (w.r.t. Day 1-2 backup) on Disk A.
...
Day 8: Incremental backup (w.r.t. Day 1-7 backup) on Disk B.
Day 9: Incremental backup (w.r.t. Day 1-8 backup) on Disk B.
Day 10: Incremental backup (w.r.t. Day 1-9 backup) on Disk B.
...
Day 15: Incremental backup (w.r.t. Day 1-14 backup) on Disk A.
Day 16: Incremental backup (w.r.t. Day 1-15 backup) on Disk A.
because it would require both disks to restore (and, thus, defeat the purpose of having two disks).
Does Windows Server Backup use Option 1 or Option 2? And were is this documented?
(To clarify: The question is the previous paragraph in bold. It's not "how do I add a second disk to my backup set", nor is it "how do incremental backups work in general".)
Best Answer
Does Windows Server's automatic incremental backup “work well” with multiple backup disks?
In my opinion, yes.
The following are the arguments I base my answer on, which allows additional discussion. There are still some aspects of how Windows Backup behaves under certain conditions I'm not that sure about and people might want to clarify or correct me. Nevertheless, I think they are worth mentioning here.
Does Windows Server Backup use Option 1 or Option 2? And were is this documented?
First of all, in general I agree with you and am pretty sure that
wbadmin/wbengineimplements option 1, but I don't have a definitive statement proving this from Microsoft as well. I additionally am somewhat sure that this is not related to Windows Server only, but works the same for non-Server editions of Windows. In fact, I'm using exactly your mentioned setup with 3 different USB-disks since Windows 7 already to create image based backups. 1 disk is off-site, two change regularly within one week.What I see is that the VHD(X) files contained on the USB-disks are updated always and how long backups take and what files are transferred at all really depends on which files have changed since the USB-disk currently used as backup target has last been used. That's the incremental part mentioned in your docs, only backup differences, but those differences are always written to the existing files in the VHD(X).
Windows Image Backup DOES NOT manage incremental history of backups on its own, it ALWAYS overwrites the existing files in the currently used VHD(X) as backup target. Therefore there's NEVER an incremental history needed to restore from the currently available VHD(X). In case of NTFS-formatted USB-disks, the history is implemented using Volume Shadow Copies: Before doing a new backup, a shadow copy is created in the backup target, only afterwards
wbengineopens the VHD(X) and replaces files within that as needed. Replacing files is done IN-PLACE, BLOCK-BY-BLOCK,wbenginereally reads the changed source files, compares read blocks to the same blocks in the backup target and only overwrites in case of changes. Because there's a shadow copy in the backup target, VSS recognizes those changed blocks, which are actually changed blocks of the VHD(X) in the end, and copies things away before overwriting. So all shadow copies in the backup target always contain a fully functional VHD(X) of the formerly backed up system and those shadow copies create an incremental history in the end. Because all shadow copies contain a fully functional VHD(X), one doesn't need any additional incremental data, but can simply mount some snapshot, the VHD(X) within that snapshot and restore as needed. VSS will handle the details of collecting the associated blocks.The following are the points I'm not that sure about:
How does Windows Image Backup decide which files to backup?
Windows/NTFS manages change journals on each volume, keeping track of which file has changed, how it has changed and because those are part of all NTFS volumes by default, they are available in the VHD(X) of the backup target as well. From my understanding,
wbenginesimply compares those journals to know which files need backup. This makes it easy to support different backup targets without caring about archive bits of files or stuff like that, because those change journals are bound to volume-unique file-IDs and those IDs are exactly the same in the backup target:In the above example,
C:\is my current system volume whileD:\is the mounted last backup on one of the two available backup targets. Even file sizes, timestamps etc. are all the same:By using this approach, backup can decide at any time with any older VHD(X) which files need to be backed up, as long as the current journal and the one in the image have something in common, which are entry-IDs in my understanding. Without such a shared ID, e.g. because too many I/O happend productive and the backup is too old,
wbenginewould simply do a full backup instead of an incremental one.Using those journals makes it as well pretty fast to know which files to backup, because one doesn't need to iterate the whole tree of files. That's what one actually sees in practice as well: Backup seems to know pretty soon which files to backup and starts processing those instead of iterating a file tree.
Behaviour in case of network shares as backup target for Windows Server 2008 R2.
In case of backing up to network shares, what
wbenginedoes seems to depend on the version of Windows used, but in general the formerly described approach of only always having one VHD(X) per backed up volume in the backup target seems to hold as well. The main difference seems to be how that is achieved, either by overwriting existing VHD(X) files, deleting and recreating them or, pretty much like is the case with USB-disks, by opening and modifying them in-place.Here's what some docs and other people say to that topic:
https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/cc742130(v=ws.10)
https://lennox-it.uk/a-complete-guide-to-wbadmin-windows-backups
https://administrator.de/forum/verständnisfragen-windows-server-backup-2012-294395.html
Looking at my own tests using Windows 2008 R2, "overwritten" seems to be a bit misleading here, because it seems that really new files get created, as not only names of the images change, but their inodes as well:
That is different to what I see on my USB-disks, where images keep their names and file-IDs(/inodes) and only most of the XML files get new UUIDs. On the USB-disks the parent directory of the VHD(X) changes as well, but that is simply a rename and hence doesn't influence the child files in any way. At one point during the tests I managed that
wbenginedecided to keep names of VHD files, but their inode changed always. Didn't invoke with any new command line, though, but simply subsequently:I don't know why they implemented things this way in case of using shares, as it breaks e.g. underlying BTRFS-snapshots. But that's exactly what I see: All the snapshots my NAS creates for the folder where I store those backups almost have all associated storage exclusively instead of sharing large parts of the data. Additionally, according to log file sizes and runtime lengths of
wbengine, all backups almost take the same amount of time, even though files in the backed up source don't change too much.Behaviour in case of network shares as backup target for Windows Server 2012+.
Things seem to be a little bit different with newer versions of Windows: I'm somewhat sure that one customer of mine ran into troubles with
wbadminand Windows Server 2012 and during debugging those using Process Monitor, I verified that existing VHDX files were kept instead of deleted and recreated. I've tested this right now with Windows Server 2019 again and multiple invocations ofwbadminled to successful backups while KEEPING inodes of VHDX files the same:So in theory this should allow incremental backups replacing files in-place and efficient snapshots of e.g. underlying BTRFS or ZFS at the same time. Looking at storage of the snapshots of the tested Windows Server 2019, at least some of those really share some space:
It's not as much as I would expect, but that might have other reasons, as this system is running into trouble backing up because of too few storage. That's actually why I'm investigating this topic again and came to this question. Looking at snapshots of other Windows 10 clients using image based backups as well, those mostly share much more. But those use the ZIP-based backup as well and the BTRFS subvolumes contain all backup-related directories per user, so the numbers might be a bit misleading.
The thing is that it might be that not much space is shared even though VHDX files are reused, because when I subsequently invoke backing up
C:of that server, backup doesn't seem to get faster. The first backup might take longer as much data has changed, but after that is finished and with the server doing nothing, a second backup only a few minutes later should be a lot faster, because of only backing up differences of the files. But that doesn't seem to be the case, instead it seems to take the same time like before. Additionally, while BTRFS can share differences in created snapshots between different invocations ofwbadminwith the exact same command line, those are much smaller than expected when really only backing up changed files:That is different to what I see when backing up to my USB-disks, subsequent backups are much faster if nothing has changed. What is interesting is that others seem to be not so sure about how things behave on network shares as well:
https://www.ubackup.com/windows-server/windows-server-backup-differential-4348.html
While the same people write the following, which doesn't make much sense:
https://www.ubackup.com/articles/wbadmin-incremental-backup-5740.html
If VHD is recreated always, like seems to be the case for older versions of Windows, in case of backing up to shares, one doesn't get incremental backups. But even though those are kept and incremental backups like with USB-disks would be possible, changing
-vssCopytovssFulldoesn't seem to change anything for me. Runtime of backups seems to be around the same in both cases.Influence of -vssFull and -vssCopy.
There's some discussion on the web once in a while about what
-vssFulland-vssCopydoes regarding full, incremental and differential backups. In my opinion both arguments simply DON'T influence howwbenginedecides which files to backup at all, because change journals are used always. What seems confusing regarding-vssCopyis especially the following:https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/cc742130(v=ws.10)
Because of how image based backups work in my opinion and described above, I have no idea what MS means with that warning. I strongly guess that this sentence is NOT related to
wbadminitself at all, but 3rd party products instead and under that assumption the warning would be in line with what is document for-vssFullEven though
-vssFullresets archive bits of files, I'm still convinced that those bits are NOT used bywbenginefor its decision which files to backup in any setup. Instead, that argument tells other applications only if to do additional backup-related stuff:Not sure if that influences change journals, though. I guess not because both arguments have the following statement in common:
Other explanations of those arguments seem to focus on 3rd party software as well:
https://techcommunity.microsoft.com/t5/Storage-at-Microsoft/What-is-the-difference-between-VSS-Full-Backup-and-VSS-Copy/ba-p/423575
Those logs are those of Exchange or databases or whatever most likely, I guess not those change journals NTFS manages on its own. Same source like above:
So in my opinion it's clear that
-vssFulland-vssCopyonly influence how things behave AFTER files have been backed up and DOES NOT influence which files to backup or how those get backed up. Executing the exact same command line forwbadminwith only-vssFullvs.-vssCopyto a network share doesn't change anything regarding VHD(X) as well for me.