Bash – Variable directory names over SCP

backupbashscp

We have a backup routine that previously ran from one disk to another on the same server, but have recently moved the source data to a remote server and are trying to replicate the job via scp.

We need to run the script on the target server, and we've set up key-based scp (no username/password required) between the two servers. Using scp to copy specific files and directories works perfectly:

scp -r -p -B buser@10.0.0.1:/mnt/disk1/bsource/filename.txt /mnt/disk2/btarget/

However, our previous routine iterates through directories on the source disk to determine which files to copy, then runs them individually through gpg encryption. Is there any way to do this only by using scp?

Again, this script needs to run from the target server, and the user the job runs under only has scp (no ssh) access to the target system.

The old job would look something like this:

#Change to source dir
cd /mnt/disk1

#Create variable to store 
# directories named by date YYYYMMDD
j="20000101/"

#Iterate though directories in the current dir
# to get the most recent folder name
for i in $(ls -d */);
do
 if [ "$j" \< "$i" ]; then
  j=${i%/*}
 fi
done

#Encrypt individual files from $j to target directory
cd ./${j%%}/bsource/
for k in $(ls -p | grep -v /$);
do
 sudo /usr/bin/gpg -e -r "Backup Key" --batch --no-tty -o "/mnt/disk2/btarget/$k.gpg" "$/mnt/disk1/$j/bsource/$k"
done

Can anyone suggest how to do this via scp from the target system? Thanks in advance.

EDIT: With all the bash one-liners I see flying around here, I was half-expecting within a few seconds to see some epic thing pop up with cat, grep, and maybe a '>>' or two between a few pipes. Ah, well. 😉

Best Answer

Run a similar version of the earlier script on the source server using cron, where it archives and encrypts all the files to a directory. At an appropriate interval following that, have a cron on the target server rsync over SSH that location. You could scp but rsync has many advantages.

Otherwise, I'd scrap your requirements and run an on-demand hybrid solution from the target using SSH pipes.