I am trying to achieve the following:
I want to mount a google-storage bucket to my docker swarm workers locally as a file-system. All swarm-workers have the necessary access-rights to the google storage API.
The bucket is mounted on instance-creation.
Afterwards, when deploying a service, this service should be able to write to the storage-bucket.
Goal:
The service should be able to run on any of the workers within the tag-group.
R/W-locking is not an issue. The service only runs once.
I would prefer not to mount the bucket from within the container, as I want to minimize the overhead and container-size.
The issue:
I can access, read and write to the filesystem locally just fine.
However, when the container tries to write to the mounted filesystem, I can see from the logs, that it can't (permission denied)
I already tried to check if it's an issue with docker swarm or not being able to run the containers in privileged mode, but the same behaviour rises, when I run the container locally in privileged mode.
I'd like to know if there is a way to achieve my goal this way, or if I need to rethink not mounting the bucket from directly within the container.
Docker-Version
17.03.1-ce
Docker info
Server Version: 17.03.1-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: active
NodeID: wcq01hi8zfzofs6elnxdfswm9
Is Manager: false
Node Address: 10.132.16.15
Manager Addresses:
10.132.16.2:2377
10.132.16.3:2377
10.132.16.4:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 4ab9917febca54791c5f071a9d1f404867857fcc
runc version: 54296cf40ad8143b62dbcaa1d90e520a2136ddfe
init version: 949e6fa
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.8.0-32-generic
Operating System: Ubuntu 16.10
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.755 GiB
Name: persistent-worker-8rrl
ID: 4H6O:BBPD:ZMSE:OKWE:QZQS:P5BT:CXFH:VAG5:MNDU:A5ZM:V2O5:TKQH
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Service create parameter
docker service create \
--constraint 'node.labels.availability == persistent' \
--mount type=bind,source=/mnt/storage-bucket/jenkins,destination=/var/jenkins_home \
--name jenkins \
--replicas=1 \
jenkinsci/jenkins
Run command
sudo docker run \
--privileged \
--volume /mnt/storage-bucket/jenkins:/var/jenkins_home \
-d \
jenkinsci/jenkins
Best Answer
I figured it out myself and thought I'd share in case somebody faces the same or a similar issue.
The user that is used in the jenkins-container (aka jenkins) can't access the locally mounted bucket. I found two possible solutions:
For now, I'm not sure which solution I will use, since I'm not satisfied with either one. Obviously running jenkins as root is not a good solution, but I also don't want to provide the Google-API access-key in a container that's exposed to the internet.