I'm trying to secure containers on my homelab.
The main goal is:
- isolating nginxA and nginxB, so they are not able to talk to each other via
172.17.0.1
(eg. preventing nginxA from reaching nginxB via172.17.0.1:5001
) - isolating nginxA and nginxB from localhost, making them exclusively accessible via traefik
- Allowing nginxA to talk to nginxA_DB, which are in the same stack, but without allowing other containers nor traefik to talk to nginxA_DB
Each docker stack has a nginx/apache service with it's port exposed in docker-compose in the following manner:
docker-compose-nginxA.yml:
networks:
internal:
ipam:
config:
- subnet: 10.0.0.0/29
...SNIP...
nginxA:
networks:
internal:
ipv4_address: 10.0.0.2
ports:
- "172.17.0.1:5000:80"
docker-compose-nginxB.yml:
networks:
internal:
ipam:
config:
- subnet: 10.0.0.8/29
...SNIP...
nginxB:
networks:
internal:
ipv4_address: 10.0.0.10
ports:
- "172.17.0.1:5001:80"
Making these services only accessible via localhost (172.17.0.1) or traefik,
Traefik being also on the same machine with the following configuration:
http:
routers:
nginxA:
entryPoints:
- web
service: nginxA
nginxB:
entryPoints:
- web
service: nginxB
...SNIP...
services:
nginxA:
loadBalancer:
servers:
- url: http://172.17.0.1:5000
nginxB:
loadBalancer:
servers:
- url: http://172.17.0.1:5001
An idea was to connect all the containers in a same /24
network with traefik, without exposing any ports via compose, but a single traefik network wouldn't isolate the containers from each other, only from the outside.
Another idea was to create a /31
network for each container and merge all the networks traefik, but I'm unsure if this will do the isolation as intended. eg.
nginxA:
networks:
internal:
ipv4_address: 10.0.0.2 # /29
nginxA-traefik:
ipv4_address: 10.50.0.1 # /31
traefik:
networks:
nginxA-traefik:
nginxB-traefik:
...
Thanks for reading this far! Do you have any ideas how this can be done?
Best Answer
You're on the right track. As in your last example, you want to create a separate network for each communication path you want between two containers. These can all be
/31
's or/29
s or whatever you want as long as the subnets are different.Example:
nginxA-traefik
connectingnginxA
andtraefik
nginxB-traefik
connectingnginxB
andtraefik
nginxA-nginxA_DB
connectingnginxA
andnginxA_DB
You should not do any port mapping on the
nginx
containers sincetraefik
will communicate with them directly over the docker networks. This will allow eachnginx
container to communicate withtraefik
(and vice versa) but they can't communicate with each other.However! This does not scale very well because you'll have to create/manage one network for every service connected to traefik which will quickly get unwieldy. A better way would be to create a single network for all traefik-linked services (e.g.
traefik_public
), connect all of your service containers (i.e.nginx_A
,nginx_B
and anything else) to this network, and add some rules toiptables
to only allow traffic to/fromtraefik
.I wrote a pretty simple container firewall that will take care of the
iptables
rules for you automatically: https://github.com/kaysond/trafficjam You just specify the network you want to protect (traefik_public
) and the container to allow communication to/from (e.g.ancestor=traefik:latest
) and it takes care of the rest!