For some use cases I'd like to have the possibility to create a volume with docker volume create
, and fill it with data. Then I want to create a new volume that is only a copy of the first one, but no data needs to be copied, only changed files should be written.
This is what Docker already does with images and containers using aufs or overlayfs. Do you know any strategy to solve this for volumes? If it does not work with docker volumes right now, can you tell me how to do something like this on the Docker host? (Then I could mount those folders from the host.)
By now I have two use cases for it, maybe you have a completely different idea to solve it:
-
Very big Git repository (around 10GB): I need to run multiple containers with this repository, some of them also commit changes. So it must be independent from other containers, push commits to the remote repository and then the copied volume may be deleted.
-
Very big mysql databases (around 130GB): For testing purposes and other migration tasks I'd like to not have to copy the whole database.
Best Answer
Given the lack of other answers to this, I'll chime in. The best way to achieve this today is to do this yourself with
overlayfs
. It's included in the Linux kernel nowadays and so you can get going with no additional packages or modules (provided you're on 3.18 or newer).Using
overlayfs
requires a working directory, an 'upper' directory and a mountpoint in addition to your existing directory that you don't want to change.Assuming I have data in
myreadonlydir
which I don't want containers to change, I can create two overlays like this:Now I have two mountpoints,
mountpoint1
andmountpoint2
which I can attach to my containers, e.g.And each container will see the contents of
myreadonlydir
inside/data
. When they write to anything in/data
(delete files, append to files, create files, whatever),overlayfs
will only 'write' to theupperdir
, and the data inmyreadonlydir
is left alone.Since the containers are using different mountpoints, each container's modifications to
/data
do not affect the other container.If you want to be really safe, you can mount your data
-o ro
to protect it from any writes.You could quite easily throw together a script to do this for each container you spin up in your test environment.
Note though, I've no idea how well
overlayfs
would perform when using it with a big MySQL database... under the hood it performs 'copy-up' operations inside theworkdir
when a file is written. Ideally yourworkdir
should be on fast storage - SSD or atmpfs
if you've enough RAM.More info @ https://wiki.archlinux.org/index.php/Overlay_filesystem.