Docker – gitlab-runner process in container can’t find gitlab container when using docker-compose

dockerdocker-composegitlab

Update:
I did not resolve my problem but I know why it does not work. If you are using docker executor, when you launch a job, gitlab-runner binary will start a special container. This container is gitlab-runner-helper and will manage git, caches, etc.
The container is started by calling the Docker Engine API running on the host (localhost, the physical machine). But, since it's started "manually", it is not linked to any bridge network. Or, at least not linked to the docker-compose network. So, the helper does not even know that that the gitlab container exists.

The problem:

I just want to have gitlab and gitlab-runner (using docker executor) working on my localhost. I want to install and manage them with docker and docker-compose.

docker-compose.yml : 

gitlab:
  container_name: my-container-gitlab
  image: gitlab/gitlab-ce:latest
  ports:
    - "443:443"
    - "9090:80"
    - "22:22"
  environment:
    GITLAB_OMNIBUS_CONFIG: |
      external_url 'http://gitlab'
  volumes:
    - ./gitlab/config:/etc/gitlab
    - ./gitlab/logs:/var/log/gitlab
    - ./gitlab/data:/var/opt/gitlab

gitlab-runner:
  container_name: my-container-gitlab-runner
  image: gitlab/gitlab-runner:latest
  volumes:
     - ./gitlab-runner/config:/etc/gitlab-runner
     - /var/run/docker.sock:/var/run/docker.sock

Gitlab works like a charm.
Runner registration works also : 

docker-compose run gitlab-runner register -n \
    --url http://gitlab/ \
    --registration-token xxxxxxxxxx \  
    --executor docker \
    --docker-image alpine \ 
    --description "My Docker Runner"

But when launching a job from Gitlab web UI, I get this :

Running with gitlab-runner 11.11.2 (ac2a293c)
      on My Docker Runner sBqMfFys
    Using Docker executor with image alpine ...
    Pulling docker image alpine ...
    Using docker image sha256:055936d3920576da37aa9bc460d70c5f212028bda1c08c0879aedf03d7a66ea1 for alpine ...
    Running on runner-sBqMfFys-project-1-concurrent-0 via 881cd3e0423c...
    Initialized empty Git repository in /builds/root/bertrand-malvaux/.git/
    Fetching changes...
    Created fresh repository.
    fatal: unable to access 'http://gitlab-ci-token:[MASKED]@gitlab/root/bertrand-malvaux.git/': 
Could not resolve host: gitlab
    ERROR: Job failed: exit code 1

For what I investigated so far :

But, as you can see in the above error, the binary does not "see" the gitlab hostname. I modified the image to check if it can see the gitlab container outside the binary only with dumb-init, and the answer is yes.

Do you have any idea how to make this works ?

Best Answer

So, since I have come across this issue myself, I thought I would reply here with my fix and an explanation:

My home lab has git-lab docker setup, and I use docker-compose to deploy both the gitlab and gitlab-runner servers. What this does is create a network that links the two together, allowing for hostname recognition within the containers...but not from outside of the created network.

Gitlab-runner, I have found by default will startup a container when running a test and add it to the "bridge" docker network. If you use dockstation you can see this by checking the info of the created container before it stops, or run docker inspect against it: docker inspect --format='{{.NetworkSettings.Networks}}' <runner temp container_id>

I had another container running on bridge and tested whether 'gitlab' would resolve. It would not.

Soooooooo.....there is a setting you can pass in the gitlab-runner's config.toml:

[[runners]]
...
  [runners.docker]
    network_mode= <----- THIS

This setting will tell gitlab-runner which network to place the container in when it gets spun up, so if you know the name of the network your gitlab and gitlab-containers run on:

docker inspect --format='{{.NetworkSettings.Networks}}' <gitlab container id>

then add that as the network mode (my network name is git-lab_default):

[[runners]]
...
  [runners.docker]
    network mode= "git-lab_default"

Then voila! Your test containers will be in the same network and everything is peachy.

Hope this helps anyone with the issue.

Related Topic