Unix – Syncing local and remote directories using rsync+ssh+public key as a user different to the ssh key owner

Your command line should look like this:

rsync -rvz -e 'ssh -p 2222' --progress ./dir user@host:/path

this works fine - I use it all the time without needing any new firewall rules - just note the SSH command itself is enclosed in quotes.

When I set this same thing up in an application of ours, I also ran into 255 errors, and found that they can mean a variety of things; it's not a particularly helpful error code. In fact, even now, after the solution's been running smoothly for well over a year, I still see an occasional 255 show up in the logs.

I will also say that getting it to work can be a bit of a pain, since there are several variables involved. One thing I found extremely helpful was to construct the rsync command in a variable and then log it. This way, I can grab the exact rsync command being used and try to run it manually. You can even su to the apache user and run it from the same directory as your script (or whatever your script is setting as the cwd), which will make it act the same as when it's run programmatically; this makes it far simpler to debug the rsync command since you're not having to deal with the web server. Also, when you run it manually, if it's failing for some unstated reason, add in verbosity flags to crank up the error output.

Below is the code we're using, slightly edited for security. Note that our code actually supports rsync'ing to both local and remote servers, since the destination is fully configurable to allow easy test installations.

try {
    if ('' != $config['publishSsh']['to']['remote']):
    //we're syncing with a remote server
        $rsyncToRemote = escapeshellarg($config['publishSsh']['to']['remote']) . ':';
        $rsyncTo = $rsyncToRemote . escapeshellarg($config['publishThemes']['to']['path']);
        $keyFile = $config['publishSsh']['to']['keyfile'];
        $rsyncSshCommand = "-e \"ssh -o 'BatchMode yes' -o 'StrictHostKeyChecking no' -q -i '$keyFile' -c arcfour\"";
    else:
    //we're syncing with the local machine
        $rsyncTo = escapeshellarg($config['publishThemes']['to']['path']);
        $rsyncSshCommand = '';
    endif;

    $log->Message("Apache running as user: " . trim(`whoami`), GLCLogger::level_debug);
    $deployCommand = "
        cd /my/themes/folder/; \
        rsync \
        --verbose \
        --delete \
        --recursive \
        --exclude '.DS_Store' \
        --exclude '.svn/' \
        --log-format='Action: %o %n' \
        --stats \
        $rsyncSshCommand \
        ./ \
        $rsyncTo \
         2>&1
    "; //deployCommand
    $log->Message("Deploying with command: \n" . $deployCommand, GLCLogger::level_debug);
    exec($deployCommand, $rsyncOutputLines, $returnValue);
    $log->Message("rsync status code: $returnValue", GLCLogger::level_debug);
    $log->Message("rsync output: \n" . implode("\n", $rsyncOutputLines), GLCLogger::level_debug);
    if (0 != $returnValue):
        $log->Message("Encountered error while publishing themes: <<<$returnValue>>>");
        throw new Exception('rsync error');
    endif;

    /* ... process output ... */
} catch (Exception $e) {
    /* ... handle errors ... */
}

A couple of things to notice about the code:

• I'm using exec() so that I can capture the output in a variable. I then parse it so that I can log and report the results in terms of how many files were added, modified, and removed.

• I'm combining rsync's standard output and standard error streams and returning both. I'm also capturing and checking the return result code.

• I'm logging everything when in debug mode, including the user Apache is running as, the rsync command itself, and the output of the rsync command. This way, it's trivially easy to run the same command as the same user with just a quick copy-and-paste, as I mention above.

• If you're having problems with the rsync command, you can adjust its verbosity without impact and see the output in the log.

In my case, I was able to simply point to an appropriate key file and not be overly concerned about security. That said, some thoughts on how to handle this:

• Giving Apache access to the file doesn't mean that it has to be in the web directory; you can put the file anywhere that's accessible by the Apache user, even on another network machine. Depending on your other layers of security, this may or may not be an acceptable compromise.

• Depending on the data you're copying, you may be able to heavily lock down the permissions of the ssh user on the other machine, so that if someone unscrupulous managed to get in there, they'd have minimal ability to do damage.

• You can use suEXEC to run the script as a user other than the Apache user, allowing you to lock access to your key to that other user. Thus, someone compromising the Apache user simply would not have access to the key.

I hope this helps.