Docker – Why does docker-machine clear data on restart

bootdisk-volumedockermac-osxstorage

I'm using Docker Toolbox on OSX.

I've created a data volume container for storing persistent data: https://docs.docker.com/userguide/dockervolumes/#creating-and-mounting-a-data-volume-container.

I checked that this data is indeed stored on the boot2docker VM (created by docker-machine) and not on the container, so that it will persist.
However "docker-machine restart " clears out this custom data on the vm.

I can't find documentation on what is happening. I found one forum post mentioning that data in /var/lib/docker will be preserved, but I could not find any official docs stating that and it also seems strange considering the persistent storage guide above doesn't use this path or even mention that your data will be deleted.

Is this expected and if so is there any official documentation on the correct path to store persistent data?

Edit: Adding example of failing senario

$ docker-machine ssh alt
docker@alt:~$ docker run -v /data:/var/lib/mysql --name mydata busybox sh -c "echo 'hello' > /var/lib/mysql/hello"
docker@alt:~$ docker run --rm --volumes-from mydata busybox sh -c "cat /var/lib/mysql/hello"
hello
docker@alt:~$ exit
$ docker-machine restart alt
Starting VM...
$ docker-machine ssh alt
docker@alt:~$ docker run --rm --volumes-from mydata busybox sh -c "cat /var/lib/mysql/hello"
cat: can't open '/var/lib/mysql/hello': No such file or directory

Best Answer

This definitely should work:

$ docker-machine ssh default
docker@default:~$ docker run -v /data --name mydata busybox true
docker@default:~$ docker run --volumes-from mydata busybox sh -c "echo hello >/data/hello"
docker@default:~$ docker run --volumes-from mydata busybox cat /data/hello
hello
docker@default:~$ exit
$ docker-machine restart default
Starting VM...
$ docker-machine ssh default
docker@default:~$ docker run --volumes-from mydata busybox cat /data/hello
hello

Can you elaborate more on the steps to reproduce your problem?

boot2docker has a read-only filesystem (will get wiped on reboot) with the exception of:

Containers and their data (volumes) -- this is what you read about /var/lib/docker
Docker images
Docker configuration (e.g. /var/lib/boot2docker/profile where the daemon flags can be tweaked)

Related Solutions

Docker – Stop postgresql container

Whether or not the database stops "gracefully" you will not have corrupt data. PostgreSQL is crash-safe unless you configure it otherwise, so long as the underlying file system and disk storage respect disk flush requests (fsync). You lose in-progress transactions, but nothing is corrupted or left half-done. This is true even if you SIGKILL (kill -9) the server, though that doesn't make it a good idea to do that routinely.

The main issue you'll have is that if the PostgreSQL server is shut down abruptly it might take longer to start up next time because it has to do more work to recover work that's committed but not yet fully applied. (oversimplification).

If there's no init running in the container (sysvinit/systemd/upstart) then most likely no graceful shutdown is performed, unless you've given docker some kind of shutdown script explicitly. (I haven't worked with docker yet). Some quick searches suggest that it just sends a SIGTERM to the process(es) in the container. This is fine for PostgreSQL and is in fact what pg_ctl -m smart does. It might take a while to stop, so you may choose to use SIGINT to force transaction aborts and shut down faster.

Docker – How to deploy a docker container and associated data container, including contents

You got an assumption wrong about how volumes work in docker. I'll try to explain how volumes relates to docker containers and docker images and hopefully differences between data volumes and data volume containers will become clear.

First let's recall a few definitions

Docker images

Docker images are essentially a union filesystem + metadata. You can inspect the content of docker image union filesystem with the docker export command, and you can inspect a docker image metadata with the docker inspect command.

Data volumes

from the Docker user guide:

A data volume is a specially-designated directory within one or more containers that bypasses the Union File System to provide several useful features for persistent or shared data.

It is important to note here that a given volume (as the directory or file that contains data) is reusable only if it exists at least one docker container using it. Docker images don't have volumes, they only have metadata which eventually tells where volumes would be mounted on the union filesystem. Data volumes aren't either part of docker containers union filesystem, so where are they? under /var/lib/docker/volumes on the docker host (while containers are stored under /var/lib/docker/containers).

Data volume containers

That special type of container has nothing special. They are just stopped containers using a data volume with the sole and unique goal of having at least one container using that data volume. Remember, as soon as the last container (running or stopped) using a given data volume is deleted, that volume will become unreachable through the docker run --volumes-from option.

Working with data volume containers

How to create a data volume container

The image used to create a data volume container has no importance as such a container can remain stopped and still fill its purpose. So to create a data container named datatest_data for a volume in /datafolder you only need to run:

docker run --name datatest_data --volume /datafolder busybox true

Here base is the image name (a conveniently small one) and true is a command we provide just to avoid seeing the docker daemon complain about a missing command. Anyway after you have a stopped container named datatest_data with the sole purpose of allowing you to reach that volume with the --volumes-from option of the docker run command.

How to read from a data volume container

I know two ways of reading a data volume: the first one is through a container. If you cannot have a shell into an existing container to access that data volume, you can run a new container with the --volumes-from option for the sole purpose of reading that data.

For instance:

docker run --rm --volumes-from datatest_data busybox cat /datafolder/data.txt

The other way is to copy the volume from the /var/lib/docker/volumes folder. You can discover the name of the volume in that folder by inspecting the metadata of one of the container using the volume. See this answer for details.

Working with volumes (since Docker 1.9.0)

How to create a volume (since Docker 1.9.0)

Docker 1.9.0 introduced a new command docker volume which allows to create volumes :

docker volume create --name hello

How to read from a volume (since Docker 1.9.0)

Let say you created a volume named hello with docker volume create --name hello, you can mount it in a container with the -v option :

docker run -v hello:/data busybox ls /data

About committing & pushing containers

It should now be clear that since data volumes aren't part of a container (the union filesystem), committing a container to produce a new docker image won't persist any data that would be in a data volume.

Making backups of data volumes

The docker user guide has a nice article about making backups of data volumes.

Good article reagarding volumes: http://container42.com/2014/11/03/docker-indepth-volumes/