Docker – Allow Docker containers to connect to OpenVPN clients on the host tunnel interface

dockeropenvpn

I have the following setup :

A CentOS host running the docker service
A user defined docker bridge network
2 Docker containers connected to that user defined bridge network
An OpenVPN installation (currently running on the host. Can also run in a docker container)
Some clients connected to the OpenVPN

How can I allow for docker containers on the docker bridge network to communicate with the openvpn clients on the tun0 network ?

I would like to be able to have tcp based communication between docker1 (10.10.0.3) and clients connected to the vpn (172.19.0.x range) in a transparent way.

What do I need to setup on docker (networking / iptables / …) side and on the host (iptables ?)

Best Answer

Context

I have been using the very good Docker container from Kyle Manna (https://github.com/kylemanna/docker-openvpn). I'm using the so-called "paranoid" documentation to set-up my OpenVPN server, but in my view this should be the standard way and not the paranoid way.

Configuration

In order to allow bi-directional connection between selected Docker containers and the VPN clients, you need to create a Docker network on which you are going to attach container which should be allowed to be accessed by the VPN clients. The VPN server is going to be one of those containers.

The VPN server should have the client-to-client, topology subnet, dev tun0 (or other tun device) and push "route <docker net IP> <docker net mask>" configured.

The host of the VPN server should be configured to support forwarding of IP packets from one subnet to another. This means setting the sysctl ip_forward to 1 (it should be the case if you have Docker install), allowing packets from the tun device to go through the iptables FORWARD chain and setting proper routing. This can be summarise with these commands:

$ sudo sysctl -w net.ipv4.ip_forward=1
$ sudo iptables -A FORWARD -i tun+ -j ACCEPT
$ sudo ip route add 192.168.255.0/24 via <IP address of OpenVPN server container>

Anyway, here are the options I've used to set-up the server:

$ docker run --rm --net=none -it -v $PWD/files/openvpn:/etc/openvpn kylemanna/openvpn:2.4 ovpn_genconfig -u udp://<FQDN> -N -d -c -p "route <docker net IP> <docker net range>" -e "topology subnet"

This should generate a server config file similar to:

server 192.168.255.0 255.255.255.0
verb 3
key /etc/openvpn/pki/private/vpn.example.com.key
ca /etc/openvpn/pki/ca.crt
cert /etc/openvpn/pki/issued/vpn.example.com.crt
dh /etc/openvpn/pki/dh.pem
tls-auth /etc/openvpn/pki/ta.key
key-direction 0
keepalive 10 60
persist-key
persist-tun

proto udp
# Rely on Docker to do port mapping, internally always 1194
port 1194
dev tun0
status /tmp/openvpn-status.log

user nobody
group nogroup
client-to-client

### Push Configurations Below
push "dhcp-option DNS 8.8.8.8"
push "route 172.20.20.0 255.255.255.0"

### Extra Configurations Below
topology subnet

Concrete example

I will now take a concrete example. In this example, I will run the above mention OpenVPN server inside Docker on host vpn.example.com. This container is attached to the Docker network docker-net-vpn. Here are the commands (in this example I generate the server configuration directly on the server and I skip the CA generation, please follow the paranoid documentation of the above mention project instead):

$ docker network create --attachable=true --driver=bridge --subnet=172.20.20.0/24 --gateway=172.20.20.1 docker-net-vpn
$ docker run --rm --net=none -it -v $PWD/files/openvpn:/etc/openvpn kylemanna/openvpn:2.4 ovpn_genconfig -u udp://vpn.example.com -N -d -c -p "route 172.20.20.0 255.255.255.0" -e "topology subnet"
$ docker run --detach --name openvpn -v $PWD/files/openvpn:/etc/openvpn --net=docker-net-vpn --ip=172.20.20.2 -p 1194:1194/udp --cap-add=NET_ADMIN kylemanna/openvpn:2.4
$ sudo sysctl -w net.ipv4.ip_forward=1
$ sudo iptables -A FORWARD -i tun+ -j ACCEPT
$ sudo ip route add 192.168.255.0/24 via 172.20.20.2

The first command creates a dedicated new Docker network which define a new subnet. We will attach the OpenVPN server to this network.

The second one creates the OpenVPN configuration using the same subnet as defined in the 1st command.

The third one creates the OpenVPN server. It is attached to the newly created Docker network and uses a fix IP.

The fourth and fifth commands configure IP forwarding.

The last command adds a new route towards the VPN client configuration via the OpenVPN container fixed IP.

Note

I haven't tried it, but it should be possible to restrict the FORWARD rule for iptables. The Docker network creation created a new bridge device. This bridge is named br-<ID> with ID being the first 12 characters of the Docker network ID. This ID can be obtained with docker network inspect -f '{{.Id}}' docker-net-vpn | cut -b-12. Therefore the following command is maybe more restrictive (so better security-wise) but should still allow our traffic to be routed:

$ NET_VPN_BRIDGE="br-$(docker network inspect -f '{{.Id}}' docker-net-vpn | cut -b-12)"
$ sudo iptables -A FORWARD -i tun+ -o ${NET_VPN_BRIDGE} -j ACCEPT

Related Solutions

Docker – Routing from docker containers using a different physical network interface and default gateway

A friend and I ran into this exact problem where we wanted to have docker support multiple network interfaces servicing requests. We were specifically working with the AWS EC2 service where we were also attaching/configuring/bringing up the additional interfaces. In this project, there is more than what you need so I will try to only include what you need here.

First, what we did was create a separate route table for eth1:

ip route add default via 192.168.1.2 dev eth1 table 1001

Next we configured the mangle table to set some connection marks coming in from eth1:

iptables -t mangle -A PREROUTING -i eth1 -j MARK --set-xmark 0x1001/0xffffffff
iptables -t mangle -A PREROUTING -i eth1 -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff

Finally we add this rule for all fwmarks to use the new table we created.

ip rule add from all fwmark 0x1001 lookup 1001

The below iptables command will restore the connection mark and then allow the routing rule to use the correct routing table.

iptables -w -t mangle -A PREROUTING -i docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff

I believe this is all that is needed from our more complex example where (like I said) our project was attaching/configuring/bringing up the eth1 interface at boot time.

Now this example would not stop connections from eth0 from servicing requests through to docker0 but I believe you could add a routing rule to prevent that.

Iptables – Allow Docker containers to use IPSEC VPN on host

I wrote a python program to wrap ipsec that installs iptables entries to allow docker containers to talk to the VPN tunnel:

https://github.com/cbrichford/docker-ipsec

Instead of doing: ipsec up

You do: docker-ipsec up

This script might need some work for use with the latest docker networking features, but worked with the old default docker0 bridge.

If you don't want to use my script to edit iptables here is how you construct the iptables command:

Figure out the virtual IP address of the host in the VPN. Call this virtualIP.
Figure out the CIDR block for IP address in the VPN. If IP address in the VPN are always of the form 10.10.X.X then your CIDR block would be 10.10.0.0/16. Call this vpnSubnet
Figure out the default route interface. This will be something like "eth0". This is the command I use to find it: sudo ip route show | grep -e "^default" | awk -- "{ print \$5 }". Call this defaultRouteInterface
Figure out the CIDR block for the docker network you want to grant access to your VPN. I this command to find it: sudo ip route show | grep -e "[[:space:]]dev[[:space:]]docker" | awk -- "{ print \$1 }". Call this dockerSubnet.

The iptables command you need to run is: sudo \ iptables \ -j SNAT \ -t nat \ -I POSTROUTING 1 \ -o ${defaultRouteInterface} \ -d "${vpnSubnet}" \ -s "${dockerSubnet}" \ --to-source "${virtualIP}"