Ubuntu – Cannot apt-get update from inside the docker container if connected to bridge network

aptdebiandockernetworkingUbuntu

I am trying to understand, why I am unable to run apt-get update from within my docker containers, either debian:latest or ubuntu:latest.

  • I am able to ping 8.8.8.8 or other address from within the container
  • I am able to ping google.com or other domains from within the container
  • apt-get update works if I start the container with --network host
  • apt-get update does not work if the container is connected to the default bridge network or any other user defined network with adapter set to bridge
  • The host is a virtual machine managed by openstack
  • Everything works as expected on my computer (not openstack, not virtual)

apt-get update will time out with:

root@66230c3e7572:/# apt update
Err:1 http://deb.debian.org/debian buster InRelease                           
  Connection failed [IP: 199.232.138.132 80]
Err:2 http://security.debian.org/debian-security buster/updates InRelease
  Connection failed [IP: 151.101.194.132 80]
Err:3 http://deb.debian.org/debian buster-updates InRelease              
  Connection failed [IP: 199.232.138.132 80]
Reading package lists... Done
Building dependency tree       
Reading state information... Done
All packages are up to date.
W: Failed to fetch http://deb.debian.org/debian/dists/buster/InRelease  Connection failed [IP: 199.232.138.132 80]
W: Failed to fetch http://security.debian.org/debian-security/dists/buster/updates/InRelease  Connection failed [IP: 151.101.194.132 80]
W: Failed to fetch http://deb.debian.org/debian/dists/buster-updates/InRelease  Connection failed [IP: 199.232.138.132 80]
W: Some index files failed to download. They have been ignored, or old ones used instead.

I would like to find out why apt-get udpate is not working while I'm connected to any bridge network on the virtual machine. So any hints on how to debug this problem are very appreciated.

Best Answer

After hours and hours I was able to solve the problem

The MTU of the docker's bridge network has to match to the MTU of host's the network adapter

In my case the MTU of eth0 (host) was set to 1450 while MTU of docker0 was set to 1500

You can change the MTU by either

In case you don't have a /etc/docker/daemon.json just create one:

# /etc/docker/daemon.json
# adjust the MTU accordingly to the hosts network adapter

{
    "mtu":1450
}

Don't forget to restart docker.service: systemctl restart docker.service

Further details:

If you wan't to check the settings use ip and compare the mtu value

$ ip a

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> **mtu 1450** qdisc fq_codel state UP group default qlen 1000
    ...
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> **mtu 1500** qdisc noqueue state DOWN group default 
    ...

Note, that docker0 states always 1500 and changes it's value only if a container ist connected to that network

$ ip a
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc fq_codel state UP group default qlen 1000
    ...
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default 
    ...
17: vethe4b452f@if16: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master docker0 state UP group default 
    ...

Custom Network

I also tried to just create a custom network with a defined MTU instead of setting the MTU via /etc/docker/daemon.json. This did not work and I don't know why

docker network create --opt com.docker.network.mtu=1450 CustomMTU

Related Topic