Docker – google-cloud storage bucket mounted to docker service

dockergoogle-cloud-storage

I am trying to achieve the following:

I want to mount a google-storage bucket to my docker swarm workers locally as a file-system. All swarm-workers have the necessary access-rights to the google storage API.

The bucket is mounted on instance-creation.

Afterwards, when deploying a service, this service should be able to write to the storage-bucket.

Goal:

The service should be able to run on any of the workers within the tag-group.
R/W-locking is not an issue. The service only runs once.

I would prefer not to mount the bucket from within the container, as I want to minimize the overhead and container-size.

The issue:

I can access, read and write to the filesystem locally just fine.

However, when the container tries to write to the mounted filesystem, I can see from the logs, that it can't (permission denied)

I already tried to check if it's an issue with docker swarm or not being able to run the containers in privileged mode, but the same behaviour rises, when I run the container locally in privileged mode.

I'd like to know if there is a way to achieve my goal this way, or if I need to rethink not mounting the bucket from directly within the container.

Docker-Version
17.03.1-ce

Docker info

Server Version: 17.03.1-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: active
 NodeID: wcq01hi8zfzofs6elnxdfswm9
 Is Manager: false
 Node Address: 10.132.16.15
 Manager Addresses:
  10.132.16.2:2377
  10.132.16.3:2377
  10.132.16.4:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 4ab9917febca54791c5f071a9d1f404867857fcc
runc version: 54296cf40ad8143b62dbcaa1d90e520a2136ddfe
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.8.0-32-generic
Operating System: Ubuntu 16.10
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.755 GiB
Name: persistent-worker-8rrl
ID: 4H6O:BBPD:ZMSE:OKWE:QZQS:P5BT:CXFH:VAG5:MNDU:A5ZM:V2O5:TKQH
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Service create parameter

docker service create \
--constraint 'node.labels.availability == persistent' \
--mount type=bind,source=/mnt/storage-bucket/jenkins,destination=/var/jenkins_home \
--name jenkins \
--replicas=1 \
jenkinsci/jenkins

Run command

sudo docker run \
--privileged \
--volume /mnt/storage-bucket/jenkins:/var/jenkins_home \
-d \
jenkinsci/jenkins

Best Answer

I figured it out myself and thought I'd share in case somebody faces the same or a similar issue.

The user that is used in the jenkins-container (aka jenkins) can't access the locally mounted bucket. I found two possible solutions:

  1. Use the root user inside the jenkins-container (not pretty)
  2. Mount the bucket directly inside the container

For now, I'm not sure which solution I will use, since I'm not satisfied with either one. Obviously running jenkins as root is not a good solution, but I also don't want to provide the Google-API access-key in a container that's exposed to the internet.

Related Topic