Linux – How to Merge Two Directory Trees Without Copying

bashfileslinux

I have two directory trees with similar layouts, i.e.

.
 |-- dir1
 |   |-- a
 |   |   |-- file1.txt
 |   |   `-- file2.txt
 |   |-- b
 |   |   `-- file3.txt
 |   `-- c
 |       `-- file4.txt
 `-- dir2
     |-- a
     |   |-- file5.txt
     |   `-- file6.txt
     |-- b
     |   |-- file7.txt
     |   `-- file8.txt
     `-- c
         |-- file10.txt
         `-- file9.txt

I would like to merge the the dir1 and dir2 directory trees to create:

 merged/
 |-- a
 |   |-- file1.txt
 |   |-- file2.txt
 |   |-- file5.txt
 |   `-- file6.txt
 |-- b
 |   |-- file3.txt
 |   |-- file7.txt
 |   `-- file8.txt
 `-- c
     |-- file10.txt
     |-- file4.txt
     `-- file9.txt

I know that I can do this using the "cp" command, but I want to move the files instead of copying, because the actual directories I want to merge are really large and contain lots of files (millions). If I use "mv" I get the "File exists" error because of conflicting directory names.

UPDATE: You can assume that there are no duplicate files between the two directory trees.

Best Answer

rsync -ax --link-dest=dir1/ dir1/ merged/
rsync -ax --link-dest=dir2/ dir2/ merged/

This would create hardlinks rather than moving them, you can verify that they were moved correctly, then, remove dir1/ and dir2/.

Related Topic