Linux – rsync – Exclude files that are over a certain size

file-transferlinuxrsyncunix

I am doing a backup of my desktop to a remote machine. I'm basically doing rsync -a ~ example.com:backup/ However there are loads of large files, e.g. wikipedia dumps etc. Most of the files I care a lot about a small, such as firefox cookie files, or .bashrc. Is there some invocation to rsync that will exclude files that are over a certain size? That way I could copy all files that are less than 10MB first, then do all files. That way I can do a fast backup of the most important files, then a longer backup of everything else.

Best Answer

There is a max-size option:

--max-size=SIZE         don't transfer any file larger than SIZE

So:

# rsync -rv --max-size=1.5m root@tss01:/tmp/dm

Will send only files less than 1.5m.

Regarding sizes from man: The suffixes are as follows: "K" (or "KiB") is a kibibyte (1024), "M" (or "MiB") is a mebibyte (1024*1024), and "G" (or "GiB") is a gibibyte (1024*1024*1024). If you want the multiplier to be 1000 instead of 1024, use "KB", "MB", or "GB". (Note: lower-case is also accepted for all values.) Finally, if the suffix ends in either "+1" or "-1", the value will be offset by one byte in the indicated direction*

Related Solutions

Ftp – How to use rsync over FTP

You don't. rsync can't do that for you, it is a protocol of its own and doesn't work over FTP.

You might, however, want to try csync. IIRC it provides rsync-like behaviour over HTTP. I can't comment on whether it works over FTP, you'll have to try it.

Linux – How to symlink folders and exclude certain files

I assume you really mean "symlink everything inside the folder"...

Conceptually, you want something like:

for each file in base folder not in exclude list
    create symlink in target folder pointing back to base folder
done

bash logic to achieve this goes something like

#! /bin/bash

exclude=( "foo.conf" "bar.conf" )

for file in *; do
    for (( index = 0; index < ${#exclude[@]}; index++ )); do
        if [[ ${file} != ${exclude[${index}]} ]]; then
            ln -s ${file} ${target}/${file}
        fi  
    done
done

Assuming you don't know already, the Advanced Bash-Scripting Guide should give you enough to work out what globbing you want (perhaps find) and how to convert that to a function (perhaps a recursive one).

Best Answer

Related Solutions

Ftp – How to use rsync over FTP

Linux – How to symlink folders and exclude certain files

Related Topic