Encrypt backups with GPG to multiple tapes

autoloadergpgtape

Currently, I use tar to write my backups (ntbackup files) to a tape drive fed by an autoloader.

Ex:
tar -F /root/advancetape -cvf /dev/st0 *.bkf
(/root/advancetape just has the logic to advance to the next tape if there is one available or notify to swap the tapes out)

I was recently handed the requirement to encrypt our tape backups. I can easily encrypt the data with no problems using GPG. The problem I'm having is how do I write this to multiple tapes with the same logic that tar uses to advance the tapes once the current one is filled? I cannot write the encrypted file to disk first (2+TB). As far as I can tell, tar will not accept binary input from stdin (it's looking for file names). Any ideas? 🙁

Best Answer

I'm using this script:

#!/bin/sh

TAPE="/dev/nst0"
mt-st -f $TAPE setblk 0
mt-st -f $TAPE status
totalsize=$(du -csb . | tail -1 | cut -f1)
tar cf - . | \
        gpg --encrypt --recipient target@key.here --compress-algo none | \
        pipemeter -s $totalsize -a -b 256K -l | \
        mbuffer -m 3G -P 95% -s 256k -f -o $TAPE \
                -A "echo next tape; mt-st -f $TAPE eject ; read a < /dev/tty"

To adapt it for your needs, here are the main points:

  • tar reads from the current directory and outputs to stdout. This way tar doesn't deal with changing tapes or encryption.
  • gpg has compression switched off as this slows the process considerably (100MB/sec+ down to 5MB/sec)
  • pipemeter is used to monitor the process and give an estimated time until all the data has been written to tape - this can be removed if it is not needed
  • mbuffer buffers the data into memory - this example uses a 3GB buffer, adjust as needed - to allow the tape drive to run for longer before running out of data, reducing "shoe shining" of the tape.
  • The -A option of mbuffer handles multiple tapes by ejecting a tape once the end has been reached and waiting for the Enter key to be pressed after the next tape has been loaded. This is where your /root/advancetape script can go.

One issue to be aware of when using this with LTO tapes:

  • The tape block size is set to variable, and mbuffer writes in 256k blocks. This works well for me with an LTO3 drive, however tar likes to use a different block size. This, combined with the fact that mbuffer handles the spanning across tapes rather than tar, means you will need to read the data off the tape again through mbuffer and then pass it through gpg and on to tar. If you try to extract it directly off the tape with tar (even if you skipped encryption) it will likely not work, and will certainly break once it reaches the end of the first tape, without giving you a chance to change to the next tape.
Related Topic