Iptables – Route outgoing connections from a docker container through a specific IP

dockeriptables

I have a docker host machine with a single physical network interface (eth0) and multiple IP addresses assigned to it using IP aliasing (eth0:1, eth0:2, eth0:3).

I would like to run several docker containers so that each of them is using it's own IP address for outgoing calls to the internet. Preferably they would be also reachable on the same IP when connecting from container to container.

How do I setup docker and iptables for this to work? And with what parameters I need to run each of the containers afterwards?

Best Answer

eth0 -> 192.168.0.1

eth0:1 -> 192.168.0.2

eth0:2 -> 192.168.0.3

eth0:3 -> 192.168.0.4

docker run --name=web01 -p 192.168.0.1:80:80 ....

docker run --name=web02 -p 192.168.0.2:80:80 ....

docker run --name=web03 -p 192.168.0.3:80:80 ....

docker run --name=web04 -p 192.168.0.4:80:80 ....

They will be created DNAT rules:

Chain DOCKER (2 references) pkts bytes target prot opt in out source destination
0 0 DNAT tcp -- !docker0 * 0.0.0.0/0 192.168.0.1 tcp dpt:80 to:172.17.0.1:80

pkts bytes target prot opt in out source destination
0 0 DNAT tcp -- !docker0 * 0.0.0.0/0 192.168.0.2 tcp dpt:80 to:172.17.0.2:80

...

traffic generated container falls under the rule:

Chain POSTROUTING (policy ACCEPT 430M packets, 26G bytes) pkts bytes target prot opt in out source destination
0 0 MASQUERADE all -- * !docker0 172.17.0.0/16 0.0.0.0/0

p.s.

In linux we can use many ip without alias

ip a a 192.168.0.1/24 dev eth0

ip a a 192.168.0.2/24 dev eth0 ....

Related Solutions

Limit Outside Connections to Docker Container with Iptables

Two things to bear in mind when working with docker's firewall rules:

To avoid your rules being clobbered by docker, use the DOCKER-USER chain
Docker does the port-mapping in the PREROUTING chain of the nat table. This happens before the filter rules, so --dest and --dport will see the internal IP and port of the container. To access the original destination, you can use -m conntrack --ctorigdstport.

For example:

iptables -A DOCKER-USER -i eth0 -s 8.8.8.8 -p tcp -m conntrack --ctorigdstport 3306 --ctdir ORIGINAL -j ACCEPT
iptables -A DOCKER-USER -i eth0 -s 4.4.4.4 -p tcp -m conntrack --ctorigdstport 3306 --ctdir ORIGINAL -j ACCEPT
iptables -A DOCKER-USER -i eth0 -p tcp -m conntrack --ctorigdstport 3306 --ctdir ORIGINAL -j DROP

NOTE: Without --ctdir ORIGINAL, this would also match the reply packets coming back for a connection from the container to port 3306 on some other server, which is almost certainly not what you want! You don't strictly need this if like me your first rule is -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT, as that will deal with all the reply packets, but it would be safer to still use --ctdir ORIGINAL anyway.

Iptables – Whitelisting outgoing traffic from docker containers

The trick is to get iptables to redirect only the connections from the DEV Env containers. We can do this by adding a rule to accept all connections from the Reverse Proxy. So the IP table rules will now become:

-A PREROUTING -i docker0 -s 172.17.0.2/32 -j ACCEPT
-A PREROUTING -i docker0 -s 172.17.0.1/32 -j ACCEPT
-A PREROUTING -i docker0 -p tcp -d 0/0 -j REDIRECT --to-port 3128

Since docker dynamically allocates IPs. The IPs used will need to be updated if the docker containers are rerun or the server is restarted. I also added the rule for 172.17.0.1 which is the docker0 ip.

These rules mean that all other traffic originating from the docker0 interface, other than the reverse proxy container and docker host itself, get redirected to squid.

Within squid we can whitelist domain as we like by using the following lines

acl allowed_domain dstdomain google.com
http_access allow allowed_domain

full iptables rules are:

# Generated by iptables-save v1.4.21 on Fri Nov  6 18:54:09 2015
*nat
:PREROUTING ACCEPT [30:1796]
:INPUT ACCEPT [28:1680]
:OUTPUT ACCEPT [37:2388]
:POSTROUTING ACCEPT [46:2964]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A PREROUTING -i docker0 -s 172.17.0.2/32 -j ACCEPT
-A PREROUTING -i docker0 -s 172.17.0.1/32 -j ACCEPT
-A PREROUTING -i docker0 -p tcp -d 0/0 -j REDIRECT --to-port 3128
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.17.0.3/32 -d 172.17.0.3/32 -p tcp -m tcp --dport 8000 -j MASQUERADE
-A POSTROUTING -s 172.17.0.3/32 -d 172.17.0.3/32 -p tcp -m tcp --dport 80 -j MASQUERADE
-A POSTROUTING -s 172.17.0.2/32 -d 172.17.0.2/32 -p tcp -m tcp --dport 80 -j MASQUERADE
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 9000 -j DNAT --to-destination 172.17.0.3:8000
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 8192 -j DNAT --to-destination 172.17.0.3:80
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 80 -j DNAT --to-destination 172.17.0.2:80
COMMIT
# Completed on Fri Nov  6 18:54:09 2015
# Generated by iptables-save v1.4.21 on Fri Nov  6 18:54:09 2015
*filter
:INPUT ACCEPT [1891:3910112]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [1500:1500230]
:DOCKER - [0:0]
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8000 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER -d 172.17.0.2/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 80 -j ACCEPT
COMMIT
# Completed on Fri Nov  6 18:54:09 2015

full squid rules are:

acl SSL_ports port 443
acl Safe_ports port 80      # http
acl Safe_ports port 21      # ftp
acl Safe_ports port 443     # https
acl Safe_ports port 70      # gopher
acl Safe_ports port 210     # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280     # http-mgmt
acl Safe_ports port 488     # gss-http
acl Safe_ports port 591     # filemaker
acl Safe_ports port 777     # multiling http
acl CONNECT method CONNECT
http_access deny !Safe_ports
http_access allow localhost manager
http_access deny manager
acl allowed_domain dstdomain google.com
http_access allow allowed_domain
http_access allow localhost
http_access deny all
http_port 3128 accel vhost allow-direct
coredump_dir /var/spool/squid3
refresh_pattern ^ftp:       1440    20% 10080
refresh_pattern ^gopher:    1440    0%  1440
refresh_pattern -i (/cgi-bin/|\?) 0 0%  0
refresh_pattern (Release|Packages(.gz)*)$      0       20%     2880
refresh_pattern .       0   20% 4320

Best Answer

Related Solutions

Limit Outside Connections to Docker Container with Iptables

Iptables – Whitelisting outgoing traffic from docker containers

Related Topic