Why rsync still tries to copy and complains not enough space

rsync

I am using rsync to copy one big file (> 4 GB) from hard disk to usb drive with "-av –progress" option. Before that, the file already exists on the usb and is the same as the one on disk. So basically I am doing a test. My assumption is, if rsync is smart enough it shouldn't do the real copy.

However, it still tries to copy (at least from the progress and usb blinking). This is the first thing that I don't understand.

Then it complains not enough space (the free space on usb is 2 GB). This shouldn't happen because it only needs to replace the file which takes same amount of space, right? I even use "–delete-before" and it still complains.

Thank you for the hint.

Best Answer

To your first issue, what is the filesystem on the USB stick? It may not contain enough information about the file for rsync to make determinations about timestamps, permissions, etc.

As to your second point, the --partial option causes rsync to create a temporary file while copying which then gets renamed once the transfer is finished.

For instance, I'm running this command right now on on a Ubuntu 14.04 system:

/usr/bin/rsync -avzh --partial --progress 192.168.42.40:/Users/Shared/bsd /srv/share/distros

This is producing the following output:

receiving file list ... 
36 files to consider
bsd/
bsd/FreeBSD-10.0-RELEASE-i386-dvd1.iso
        738.41M  31%   11.20MB/s    0:02:17

But the file being created is:

15:28:57 file01 lnelson:/srv/share/distros/bsd> ls -al
drwxr-xr-x 3 lnelson lnelson       4096 Sep 17 15:28 .
drwxr-xr-x 4 lnelson lnelson       4096 Sep 14 23:12 ..
-rw------- 1 lnelson lnelson  242745344 Sep 17 15:28 .FreeBSD-10.0-RELEASE-i386-dvd1.iso.Xo0LH8

When the operation completed, I got:

15:28:58 file01 lnelson:/srv/share/distros/bsd> ls -al
drwxr-xr-x 3 lnelson lnelson       4096 Sep 17 15:31 .
drwxr-xr-x 4 lnelson lnelson       4096 Sep 14 23:12 ..
-rw-r----- 1 lnelson lnelson 2317000704 Sep 15 23:36 FreeBSD-10.0-RELEASE-i386-dvd1.iso

If rsync is indeed recopying the file and you are using the --partial flag, then it is creating a temporary new file that has a . as the first character so you'll only see it if you do an ls -a or ls -al while the rsync is in progress to see the temporary file. This means you need enough free space on the USB drive to handle the exiting 4GB copy plus the new 4GB temporary file until the copy finishes, at which time the temporary will replace the original and you'll be back to only using 4GB on the USB drive. During the rsync, you'll need 8GB of storage for the operation to complete.

Related Topic