Docker – Isolating Docker Bridge Network with VLANs

dockerdocker-networking

My network is isolated into a few different VLANs but I am struggling a little with the best way to achieve isolation of some docker containers running inside a custom bridge network. For the sake of simplicity lets say the network looks like this and there are two VLANs (Trusted traffic is untagged and non-trusted traffic is on VLAN 10). Untagged traffic is on a 192.168.x.0/24 subnet and VLAN 10 is on a 10.x.x.0/24 subnet.

Network Overview
I am running docker on Unraid and have set it up to allow VLANs and it gets an IP on both the untagged and VLAN 10 subnets.
Unraid is configured so that is just docker that is accessible on VLAN 10 and with some containers running in a custom bridge network (10.99.99.0/24) everything works one way. When mapping ports to containers I use <VLAN 10 IP>:Port so that the port is only accessible via the VLAN 10 IP. This allows me to have some containers running inside the docker network that are purely internal to that network and not accessible outside.

The issue I have is that the containers themselves can communicate back out to the untagged network. I guess this makes sense as the host has a route to the untagged network and the docker traffic isn't tagged with a VLAN ID.

I have tried creating a iptables rule to drop traffic between the docker 10.99.99.0/24 interface and br0

iptables -A FORWARD -i br-<MAC> -o br0 -j DROP

but that doesn't seem to work and traffic can still get through.

How can I add outbound isolation to this setup? (ie. I only want to allow traffic from this docker network out through the br.10 interface)

I don't want to just use a MACVLAN network and put the devices on VLAN 10 as their are ports on containers I don't want to be accessible to other devices and there are some containers that I simply don't want to be seen at all on that subnet.

Ideally it would be nice to do all of this at the host level but the only other option I can see is to create a MACVLAN network on a new VLAN and apply lots of firewall rules on the router (which is a bit of a pain to maintain as I will then need to fix IPs of the containers etc.)

Best Answer

An iptables rule should be sufficient. It would need to be at the top of your FORWARD chain; something like:

iptables -A FORWARD -s 10.99.99.0/24 -d 192.168.x.0/24 -j DROP

It needs to be at the top of the chain because otherwise it becomes a no-op -- there are rules added by Docker that will explicitly ACCEPT the traffic.

You could instead add the rule to the DOCKER-USER chain; this is a chain that Docker arranges to be called before any Docker-managed rules. The FORWARD chain on my local system looks like:

-P FORWARD ACCEPT
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
.
.
.

Alternately, you could add a policy routing rule so that traffic from your containers wouldn't have a route to the untagged network. Your default routing policy looks like this:

# ip rule show
0:      from all lookup local
32766:  from all lookup main
32767:  from all lookup default

Where the main table is what you see when you run ip route. You can add a rule that uses an alternate lookup for traffic originating from your containers:

ip rule add priority 1000 from 10.99.99.0/24 lookup 1000

That should result in:

# ip rule show
0:      from all lookup local
1000:   from 10.99.99.0/24 lookup 1000
32766:  from all lookup main
32767:  from all lookup default

And then add routing entries to table 1000:

ip route add table 1000 default via 10.x.x.1

Now connections from the containers will only have a default route, and would only be able to access the untagged network if the router at 10.x.x.1 provided an appropriate route.

Related Topic