Docker Backup – Backup Large Docker Volumes vs Bind Mounts

backupdockerowncloud

I some kind of "inherited" a dockerized server environment and now I have to implement a backup concept – especially for the Owncloud container, which stores the user files inside a docker volume. The docer docs [1] says that "volumes have several advantages over bind mounts" and therefore "are the preffered mechanism".
One of the listed advantages is that "volumes are easier to back up" and the docs suggests a command such as the following to create a tar backup of a volume:

$ docker run --rm --volumes-from dbstore -v $(pwd):/backup ubuntu tar cvf /backup/backup.tar /dbdata

However, especially for backups (I use borg backup to sync host files to an offsite repo) I see several disadvantages this way:

For each backup you have to copy the whole data locally, which needs twice the disk space and the time for copy.
Each backup creates a new BLOB – unchanged files have to be copied again.
To restore a single Owncloud file you have to restore the whole tar from the backup (untaring before syncing to backup would tripple the needed disk space).
…

I'm relatively new to docker, so I maybe have not yet the full overview of the docker ecosystem. But to my current knowledge, for this situation I think execptionally a bind mount would be the best solution. Also I have no benefit from the other advantages of volumes, anyway. Or am I missing something? Are there any disadavantages using a bind mount?

What about backing up /var/lib/docker/volumes/ directly?

Through my search, I came up with this blog post from owncloud [2] which suggests backing up /var/lib/docker/volumes/owncloud_files/_data directly. For me, this does not seem the cleanest way.

However, dealing with the current situation I'm thinking to go this way for the moment. Is it safe? (Of course with respect to the needed precautions for consitstency, e.g. shutdown database container, owncloud maintanance mode, etc. – which I think are needed in any way for the mentioned solutions here!? )

[1] https://docs.docker.com/storage/volumes/

[2] https://owncloud.org/news/docker-series-pt-3-automatically-easily-backup-restore-dockerized-owncloud/

Best Answer

Also I have no benefit from the other advantages of volumes, anyway. Or am I missing something? Are there any disadavantages using a bind mount?

I seem to understand that Docker wants you to forget a host even exists; and in fact, sometimes you won't have access to it, so volumes are the most obvious way to persist data.

The bind mount is a path on the host, so you have to "manage" it (allocate space, mount a partition...) in some way outside of Docker. Maybe you have already a system in place for that, for example an Ansible playbook that installs Docker, docker-compose and uploads the docker-compose.yml to the server. But if you don't, or you just point your docker-compose to a remote Docker daemon, well, you'll want to use just that, and nothing else: then you'll want data in volumes.

What about backing up /var/lib/docker/volumes/ directly?

I agree it is not a good idea: that is Docker's own space, I wouldn't touch it just like I would not copy from /var/lib/postgresql on a running instance.

Or am I missing something?

Why don't you run borg instead of tar inside your temporary container?

Related Solutions

Docker – How to Backup and Restore Docker Volumes

You're right. Since you can have multiple containers with volumes on their own, you need to keep track which volume corresponds to which container. How to do that depends on your setup: I use the name -data for the data container, so it's obvious to which container a image belongs. That way it can be backed up like this:

VOLUME=`docker inspect $NAME-data | jq '.[0].Volumes["/path/in/container"]'`
tar -C $VOLUME . -czvf $NAME.tar.gz

Now you just need to rebuild your image and recreate your data container:

cat $NAME.tar.gz | docker run -name $NAME-data -v /path/in/container \
                              -i busybox tar -C /path/int/container -xzf -

So this means you need to backup:

Dockerfile
volume
volume path in container
name of the container the volume belongs to

Update: In the meanwhile I created a tool to backup containers and their volume(s) (container(s)): https://github.com/discordianfish/docker-backup and a backup image that can create backups and push them to s3: https://github.com/discordianfish/docker-lloyd

Docker – Difference Between Creating a Volume or a Mount in Containers

There are actually three types of volumes:

Host Volume: what you refer to as a mount in a container, the more common term is a bind mount.
Named Volume: any volume managed by docker which you give a name.
Anonymous Volume: any volume without a source, docker will create this as a local volume with a long unique id, and it behaves as a named volume.

Volumes have a source and a target. The source identifies the type of volume, so a path (including the leading slash) to a file/directory results in a host volume. If you do not provide a source, you get the anonymous volumes. If you define a volume inside a Dockerfile, you cannot specify a source there, so by default docker will create anonymous volumes unless you direct it otherwise at runtime.

For each type, here are the pros/cons:

Host:
- Pro: easy to access the underlying files from the host
- Con: uid/gid permission issues occur when container user's uid does not match the host gid
- Con: data is not initialized
Named:
- Pro: easy to create an reuse between different containers/images. If you only give it a name with no other settings, the local driver will default to storing your data in /var/lib/docker/volumes which should only be accessible by root from outside of docker.
- Pro: initializes content to the image contents when it is empty/new and the container is created. This initialization includes file owners and permissions from the image, which can resolve most uid/gid issues.
- Pro: Can connect to anything that a mount command can, including a bind mount or NFS mount, with a local driver. Other drivers let you reference data in even more locations (e.g. cloud providers).
- Con: managing content should be done via a container.
Anonymous:
- Pro: requires no planning to use
- Con: data typically goes here to be lost since there is no mapping from the volume back to the container/image that created it. This is the worst way to store volumes in my opinion, and the reason that no one should ever define a volume inside their Dockerfile.

When possible, I use a named volume. The initialization of data and better handling of uid/gid issues trump the convenience of a host volume. If I really need access outside of docker directly to the data, then I try to use a named volume that points to a bind mount instead of the default local driver settings. A simple example of this is:

$ docker volume create --driver local \
  --opt type=none \
  --opt device=/home/user/test \
  --opt o=bind \
  test_vol

For defining my volumes, since you do not want to do this in a Dockerfile, I use a docker-compose.yml and define my volumes in there. If it's deployed with swarm mode, I'll point to a NFS server with a named volume to allow the data to be reached as the containers migrate to different hosts. Otherwise it's a local named volume that can be easily used with docker-compose.

Best Answer

Related Solutions

Docker – How to Backup and Restore Docker Volumes

Docker – Difference Between Creating a Volume or a Mount in Containers

Related Topic