Bash – Detecting upload success/failure in a scripted command-line SFTP session

bashscriptingsftp

I am writing a BASH shell script to upload all the files in a directory to a remote server and then delete them. It'll run every few hours via a CRON job.

My complete script is below. The basic problem is that the part that's supposed to figure out whether the file uploaded successfully or not doesn't work. The SFTP command's exit status is always "0" regardless of whether the upload actually succeeded or not.

How can I figure out whether a file uploaded correctly or not so that I can know whether to delete it or let it be?

#!/bin/bash

# First, save the folder path containing the files.
FILES=/home/bob/theses/*

# Initialize a blank variable to hold messages.
MESSAGES=""
ERRORS=""


# These are for notifications of file totals.
COUNT=0
ERRORCOUNT=0

# Loop through the files.
for f in $FILES
do
    # Get the base filename
    BASE=`basename $f`

    # Build the SFTP command. Note space in folder name.
    CMD='cd "Destination Folder"\n'
    CMD="${CMD}put ${f}\nquit\n"


    # Execute it.
    echo -e $CMD | sftp -oIdentityFile /home/bob/.ssh/id_rsa bob@ftp.example.edu

    # On success, make a note, then delete the local copy of the file.
    if [ $? == "0" ]; then
        MESSAGES="${MESSAGES}\tNew file: ${BASE}\n"
        (( COUNT=$COUNT+1 ))

        # Next line commented out for ease of testing
        #rm $f
    fi

    # On failure, add an error message.
    if [ $? != "0" ]; then
        ERRORS="${ERRORS}\tFailed to upload file ${BASE}\n"
        (( ERRORCOUNT=$ERRORCOUNT+1 ))
    fi

done

SUBJECT="New Theses"

BODY="There were ${COUNT} files and ${ERRORCOUNT} errors in the latest batch.\n\n"

if [ "$MESSAGES" != "" ]; then
    BODY="${BODY}New files:\n\n${MESSAGES}\n\n"
fi

if [ "$ERRORS" != "" ]; then
    BODY="${BODY}Problem files:\n\n${ERRORS}"
fi

# Send a notification. 
echo -e $BODY | mail -s $SUBJECT bob@example.edu

Due to some operational considerations that make my head hurt, I cannot use SCP. The remote server is using WinSSHD on windows, and does not have EXEC privileges, so any SCP commands fail with the message "Exec request failed on channel 0". The uploading therefore has to be done via the interactive SFTP command.

Best Answer

If you batch your SFTP session, the exit status $? will tell you if it's a failed transfer or not.

echo "put testfile1.txt" > batchfile.txt
sftp -b batchfile.txt user@host
if [[ $? != 0 ]]; then
  echo "Transfer failed!"
  exit 1
else
  echo "Transfer complete."
fi

Edited to add: This is something we use in a production script at work, so I'm sure it's valid. We can't use scp because some of our customers are on SCO Unix.