Nginx – Gitlab showing 404 while running behind nginx reverse proxy, all within a docker network

dockergitlabnginxreverse-proxy

As the title says, I'm trying to serve Gitlab through an nginx reverse proxy, with both programs being run in separate docker containers connected through a docker network. A picture as an example:

Linux Host
 ____________________________
|                            |
| Docker                     |
|  __________________________|
| |                          |
| | Docker network (test-net)|
| |  ________________________|
| | |                        |
| | | nginx        gitlab    | Only nginx has a port bound to the host (443).
| | | |   |        |   |     | TLS is terminated at nginx as well.
| | | |   |   -->  |   |     | in my test, I have nginx running as localhost.
| | | |___|        |___|     | To access gitlab, hit https://localhost/git/
| | |________________________|
| |__________________________|
|____________________________|

nginx runs with this docker command:

docker run -dit --network=test-net --name=nginx -p 443:443 -v "$PWD/conf":/etc/nginx:ro nginx:alpine && docker logs -f nginx

nginx.conf

<Removed unnecessary config from here, very basic setup>
http {
    keepalive_timeout 65;
    server {
        listen 443 ssl;
        server_name localhost;
        ssl_certificate localhost.crt;
        ssl_certificate_key localhost.key;
        ssl_protocols   TLSv1.2;
        ssl_ciphers HIGH:!aNULL:!MD5;
        location /git/ {
            proxy_pass http://test/;
        }
    }
}

gitlab.rb

<only relevant parts added here>
external_url 'https://localhost'
nginx['listen_port'] = 80
nginx['listen_https'] = false
nginx['proxy_set_headers'] = {
 "Host" => "$http_host_with_default",
 "X-Real-IP" => "$remote_addr",
 "X-Forwarded-For" => "$proxy_add_x_forwarded_for",
 "X-Forwarded-Proto" => "http",
 "Upgrade" => "$http_upgrade",
 "X-Forwarded-Ssl" => "on",
 "Connection" => "$connection_upgrade"
}
nginx['custom_error_pages'] = {
  '404' => {
    'title' => '404',
    'header' => 'You\'ve been hit by !! You\'ve been struck by ! A false URL.',
    'message' => 'Double check that URL! Is it correct?'
  }
}

docker-compose.yml for gitlab:

version: '3.7'
services:
  gitlab:
    image: 'internal-docker-repo:1234/gitlab/gitlab-ce:11.8.3-ce.0'
    restart: always
    hostname: 'test'
    container_name: test
    volumes:
      - './config:/etc/gitlab:rw'
    networks:
      - net
networks:
  net:
    external: true
    name: test-net

Internally (to docker networks) nginx is known as nginx and gitlab is known as test. I have confirmed I can ping each container from inside the other, using their container names.

As it is now, it almost works. When I go to https://localhost/git/ on my linux host I get a 404 error page from gitlab, but no login screen.

the 404 screen. Custom, so I know gitlab is running and picked up the configuration

I'm obviously missing something but I'm not sure what it is. It's hard for me to tell if it's an NGinx configuration issue or a Gitlab configuration issue.

Log output when I hit https://localhost/git/

nginx log output:

172.19.0.1 - - [07/Jan/2020:21:28:35 +0000] "GET /git/ HTTP/1.1" 404 2289 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0"

gitlab log output:

test      | ==> /var/log/gitlab/nginx/gitlab_access.log <==
test      | 172.19.0.3 - - [07/Jan/2020:21:28:35 +0000] "GET / HTTP/1.0" 404 2289 "" "Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0"
test      | 
test      | ==> /var/log/gitlab/gitlab-workhorse/current <==
test      | 2020-01-07_21:28:35.10649 test 127.0.0.1:0 - - [2020/01/07:21:28:35 +0000] "GET / HTTP/1.1" 404 3108 "" "Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0" 0.001
test      |

Best Answer

I think at least one of the problems (if not the most important) is the fact that gitlab expects https://localhost as the URL in your browser. Unfortunately, you're not giving it that, but https://localhost/git, although the request obviously reaches the backend (which is the gitlab container).

Changing the external url directive on the inner nginx to "https://localhost/git" might work as a quick fix. Obviously you need to reconfigure gitlab afterwards:

sudo gitlab-ctl reconfig

If I were you, though, I would use a dedicated domain for gitlab (so a separate server block in nginx), as this would be cleaner to my mind. Something to the effect of:

server {
        listen 443 ssl;
        server_name gitlab.example.com;
        ssl_certificate localhost.crt;
        ssl_certificate_key localhost.key;
        ssl_protocols   TLSv1.2;
        ssl_ciphers HIGH:!aNULL:!MD5;
        location / {
            proxy_pass http://test/;
              }
        }

And then change the external_url, of course, to gitlab.example.com.

Since you're using the localhost anyway, I don't think it would be much bother if you added this new domain in /etc/hosts : 127.0.0.1 gitlab.example.com

Related Topic