Linux – Using rsync to rename files during copying with –files-from

bashlinuxrenamersync

Using rsync, how can I rename files when copying with the –files-from argument? I have about 190,000 files, each of which need to be renamed when copying from source to destination. I plan to have the list of files in a text file to pass to the –files-from argument.

Best Answer

Unfortunately I don't think rsync has any built-in renaming capability. The first workaround that comes to mind is that you could create a local mirror directory, with (renamed) symlinks to the actual data. Then you could use rsync on the mirror directory, adding the -L option ("transform symlink into referent file/dir"). You'd want to write a quick helper script to automate updating the mirrored structure, but it doesn't sound too scary.

Related Solutions

Linux – Copying a large directory tree locally? cp or rsync

I would use rsync as it means that if it is interrupted for any reason, then you can restart it easily with very little cost. And being rsync, it can even restart part way through a large file. As others mention, it can exclude files easily. The simplest way to preserve most things is to use the -a flag – ‘archive.’ So:

rsync -a source dest

Although UID/GID and symlinks are preserved by -a (see -lpgo), your question implies you might want a full copy of the filesystem information; and -a doesn't include hard-links, extended attributes, or ACLs (on Linux) or the above nor resource forks (on OS X.) Thus, for a robust copy of a filesystem, you'll need to include those flags:

rsync -aHAX source dest # Linux
rsync -aHE source dest  # OS X

The default cp will start again, though the -u flag will "copy only when the SOURCE file is newer than the destination file or when the destination file is missing". And the -a (archive) flag will be recursive, not recopy files if you have to restart and preserve permissions. So:

cp -au source dest

Windows – Ignore windows hidden files when copying with rsync

To get the DOS attributes into Linux you will need to edit your smb.conf file.

There are two different ways to import Windows attributes.

The first option is to set map hidden = yes which will mean that hidden files on Windows will have the world execute bit set on Linux. For this to work you must also have at least 001 in your create mask.

The other option is to set store dos attributes = yes which will mean that the Windows attributes will be stored in an extended attribute in the Linux file system called user.DOSATTRIB.

rsync doesn't have the ability to filter files based on either normal or extended attributes but it can exclude a list of files that you have prepared in some other way. You can use the find command to create this list based on Unix attributes. The version of find I have doesn't seem to support extended attributes but it might still be possible to use the -exec option in find to get extended attributes of all your files and filter on them.

Since find can filter on normal Unix permissions, if you chose the world execute bit option then find . -perm -001 will find all of the hidden files in your mounted Windows filesystem. You can put this list in a file and then use rsync --exclude-from=FILE to exclude those files from your rsync.

Best Answer

Related Solutions

Linux – Copying a large directory tree locally? cp or rsync

Windows – Ignore windows hidden files when copying with rsync

Related Topic