Docker Container Loses Connection Through Host VPN Interface – Fix

dockerlinux-networkingwireguard

So I have the following example compose file, with a custom defined network:

version: '3.7'
services:
  ubuntu:
    image: ubuntu
    restart: unless-stopped
    tty: true
    networks:
      wireguard:
        ipv4_address: 172.18.0.2
networks:
  wireguard:
    driver: bridge
    driver_opts:
      com.docker.network.bridge.name: 'wireguard'
    ipam:
      driver: default
      config:
        - subnet: 172.18.0.0/16

And the following Wireguard config defined as wg0.conf:

[Interface]
Table = off
SaveConfig = false
PrivateKey = {Client Private Key}
Address = 10.8.0.21/32
DNS = 10.8.0.1

# And create routing manually only for WG related communication.
PostUp = iptables -t mangle -I PREROUTING 1 -i wg0 -m conntrack --ctstate NEW -j CONNMARK --set-mark 2
PostUp = iptables -t mangle -I OUTPUT 1 -m connmark --mark 2 -j CONNMARK --restore-mark
PostUp = iptables -t mangle -I PREROUTING 2 -m connmark --mark 2 -j CONNMARK --restore-mark
PostUp = ip rule add fwmark 2 lookup 2
PostUp = ip rule add table main suppress_prefixlength 0
PostUp = ip route add default dev wg0 table 2
PostUp = sysctl -q net.ipv4.conf.all.src_valid_mark=1
PostUp = sysctl -w net.ipv4.conf.wg0.rp_filter=2

# Removal when `wg0` goes down
PostDown = iptables -t mangle -D PREROUTING -i wg0 -m conntrack --ctstate NEW -j CONNMARK --set-mark 2
PostDown = iptables -t mangle -D OUTPUT -m connmark --mark 2 -j CONNMARK --restore-mark
PostDown = iptables -t mangle -D PREROUTING -m connmark --mark 2 -j CONNMARK --restore-mark
PostDown = ip rule del table main suppress_prefixlength 0
PostDown = ip rule flush table 2

[Peer]
PublicKey = {VPN Server Public Key}
AllowedIPs = 10.8.0.1/32,0.0.0.0/0
Endpoint = {VPN Server Connection Endpoint}
PersistentKeepalive = 25

Now the wireguard connection is setup to route incoming connections back through the wg0 interface, to avoid all of the client connections going through the VPN.

So now my objective is to have specific docker containers routed through the VPN connection for their connections out. So what I do is rune the following rule and route in the host shell to add to the existing table:

# ip rule add from 172.18.0.0/16 lookup 2
# ip route add 172.18.0.0/16 dev wg0 table 2

And if I enter the docker shell and check the connection it works perfectly and shows the IP of the VPN server:
ping and ip inside docker container

But after around 30 second to up to 5 minutes, I can't get a connection out and any connection attempts just timeout:
no connection out docker container

And if I tcpdump the bridge network it just keeps trying to make ARP requests for 172.18.0.1:
tcpdump container bridge

I have no idea why this happening or how to fix this. I have seen several posts and forums of people doing the same for routing a docker network connection through a specific interface, so I don't know if I've done something wrong elsewhere.

Best Answer

I think the right way to do is running the VPN in a seperate docker container and then connect the other docker containers through that VPN. That's at least the best practice.

A example docker compose:

---
version: "3.7"
services:
  wireguard:
    image: lscr.io/linuxserver/wireguard:latest
    container_name: wireguard
    cap_add:
      - NET_ADMIN
      - SYS_MODULE #optional
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Etc/UTC
      - SERVERURL=wireguard.domain.com #optional
      - SERVERPORT=51820 #optional
      - PEERS=1 #optional
      - PEERDNS=auto #optional
      - INTERNAL_SUBNET=10.13.13.0 #optional
      - ALLOWEDIPS=0.0.0.0/0 #optional
      - PERSISTENTKEEPALIVE_PEERS= #optional
      - LOG_CONFS=true #optional
    volumes:
      - /path/to/appdata/config:/config
      - /lib/modules:/lib/modules #optional
    ports:
      - 51820:51820/udp
    sysctls:
      - net.ipv4.conf.all.src_valid_mark=1
    restart: unless-stopped
   

  ubuntu:
    image: ubuntu
    restart: unless-stopped
    tty: true
    network_mode: service:wireguard

With network_mode: service the Ubuntu container will connect through the WireGuard container with the vpn client. service: This allows a container to share the network namespace with another service. You specify the service name, and the containers will use the same network stack.

You can also use network_mode: host instead of networks

Related Topic