Docker – Share disks through NFS on a CoreOS cluster

coreosdockernfs

Is it possible for a CoreOS cluster to share disk space, e.g. using NFS? If so, how would one go about this? (e.g. in the scenario where one node has a lot of disk space). This would be useful to avoid having each node have to download and store it's own library of docker images, for instance, or to share home directory space across nodes.

Because we cannot install additional software directly in CoreOS, I imagine one would have to write a container just to install NFS (e.g. nfs-kernel-server on a Ubuntu-based container).

I have no idea if this is possible, but am hoping there's some established way to go about sharing disk space across a CoreOS cluster (after all, seems like a common expectation for a cluster, and perhaps my proposal below is more convoluted than necessary). Just to provide some fodder for feedback, here's what I'm thinking so far:

Providing the host-side of NFS seems like a reasonable docker task, e.g. I imagine a Dockerfile like:

FROM ubuntu:14.04
ENV CLIENT_IP 11.111.111.111          
RUN apt-get update && apt-get install -y nfs-kernel-server supervisor
RUN mkdir /var/nfs && chown nobody:nogroup /var/nfs
RUN echo "/home       ${CLIENT_IP}(rw,sync,no_root_squash,no_subtree_check)" >> /etc/exports
RUN echo "/var/nfs    ${CLIENT_IP}(rw,sync,no_subtree_check)" >> /etc/exports
RUN exportfs -a
CMD service nfs-kernel-server start

Where CLIENT_IP was filled out appropriately (and perhaps we need to replace the CMD with a call to supervisord or similar to make this persistent, but you get the idea)

So, how would we link volumes appropriately when running this container? Which volume would we link from the CoreOS host? Or do I need to add something like --net="host" to make the client available?

docker run -v /home:/home nfs-server

It's not at all clear to me how we could implement the client side of things, since once again we would need a container to provide nfs-common, and somehow figure out how other containers could then share that resource (perhaps some appropriate use of --volumes-from?) I'd love to see an outline of how to go about this, or why it's not possible & if there are better alternatives to address this use case. Thanks!

Best Answer

You should be able to mount an NFS volume on the CoreOS host. If there is no userland tools to export a filesystem from CoreOS, you can use Fedora toolbox container provided by the toolbox command.