How to rsync only files from different subdirs that contain a particular string in the filename

rsync is a powerful tool, but it is, unfortunately, strangely picky about some of its pathnames.

If $DATA is an absolute path (i.e. it begins with a /), then the correct command-line to use is:

rsync -a --link-dest=$DATA $DATA/ $DATA/../upgrade_tmp

[Now, just a brief aside about rsync's strangeness. Note the trailing / added to the source argument. This tells rsync to work with the contents of the source directory, rather than with the source directory itself. (I'm assuming that $DATA doesn't already contain a trailing /.) In this case, we want to work with the contents, so we add the trailing /.]

If, on the other hand, $DATA is a relative path (i.e. it does not begin with a /), then Sean R's comment about --link-dest is bang on: The link-dest path is interpreted relative to the destination path, so you would use the following:

rsync -a --link-dest=../`basename $DATA` $DATA/ $DATA/../upgrade_tmp

EDIT

One final note, it turns out that the second rsync command-line I gave should work regardless of whether $DATA is an absolute path, since basename doesn't care whether a path is absolute or relative.

You are probably running rsync like this:

rsync -a dir/ remote:/dir/

The -a option according to the documentation is equivalent to: -rlptgoD

      -a, --archive    archive mode; equals -rlptgoD (no -H,-A,-X)

You probably want to remove the -o and -g options:

      -o, --owner                 preserve owner (super-user only)
      -g, --group                 preserve group

So instead your rsync command should look something like this:

rsync -rlptD dir/ remote:/dir/

Or as @glglgl points out:

rsync -a --no-o --no-g dir/ remote:/dir/

The remaining options in use are:

      -r, --recursive             recurse into directories
      -l, --links                 copy symlinks as symlinks
      -p, --perms                 preserve permissions
      -t, --times                 preserve modification times
      -D                          same as --devices --specials
          --devices               preserve device files (super-user only)
          --specials              preserve special files