Bash – rsync include/exclude filters on subdirectories

bashrsync

I've read dozens of other answers for this but none of them are working.

I have a number of directories like this

Demultiplexing/Run1/analysis/
Demultiplexing/Run1/output/
Demultiplexing/Run2/analysis/
Demultiplexing/Run2/output/
OtherProjects/Run1/analysis/
OtherProjects/Run1/output/
OtherProjects/Run2/analysis/
OtherProjects/Run2/output/

I want to copy the following:

  • only Demultiplexing directories

  • only output subdirectories

  • exclude files with : in the name

I have been trying variations on the following command:

remote_dir="/home/$(whoami)/production"
remote_server="server.org"
production_dir="/data/production"
rsync --dry-run -vrthP -e ssh "${production_dir}/" "$(whoami)"@"${remote_server}":"${remote_dir}/" \
--include="Demultiplexing" \
--include="Demultiplexing/*/output" \
--exclude="*:*" \
--exclude="*"

However, this is not working, its only matching the top level Demultiplexing directory and nothing else. I have tried a lot of other combinations and they all either include everything, or exclude everything.

Best Answer

Figured it out:

rsync -vrthP -e ssh "${production_dir}/" "$(whoami)"@"${remote_server}":"${remote_dir}/" \
    --include="Demultiplexing" \
    --include="Demultiplexing/*" \
    --include="Demultiplexing/*/output/***" \
    --exclude="*:*" \
    --exclude="*"
Related Topic