I have a situation where I need to move over 2 TB of data from one server to another server in another data center. There are over 6 million files to copy.
The plan is to move data in phases. For example, as the first phase, copy data older than a certain date, compress them, transfer using a USB device, then copy to the new server.
After the final cutover, there will not be any changes happening in source server.
So I need to take checksum of the root folders in this server, and take one in the new server, and somehow compare both of them, and verify that they have the same contents. And this needs to be done in the least amount of time. There is no way to connect the two data centers. Source server runs on Windows 2008 and destination server runs on Windows 2016.
How can I do this ?
Best Answer
You should use an rsync like tool, which would allow you to copy and verify on the fly, while allowing you to stop the transfer and resume at any point.
A few alternatives that come to mind, Windows compatible:
All of this tools allow for data verification during (or after) the copy (checksums or integrity), so you can avoid corrupted files, they usually include parallel process for faster transfer speeds over the default Windows commands.
They shouldn't take too long to copy 2TB of files if you have a good external hard-drive and USB 3.0 support on both the server and the drive.
Disclaimer: All that said, even though I use rsync constantly on Linux, I have not tried the alternatives for Windows. The tools listed are from open-source alternatives I have come across or are free.