Background
I have a debian server that has 3 network interfaces which are:
- eno1 (10.0.0.35/24)
- eno1.10 (10.0.10.65/24)
- eno1.40 (10.0.40.40/24)
Between those interfaces is a firewall. The multiple routes on the server lead to asymetrical routing which was blocked by firewall as invalid traffic.
Because of that I added some policy-based rules so the destination/source IP address stay the same. I accomplished this by editing my /etc/network/interfaces
like this:
# The primary network interface
allow-hotplug eno1
iface eno1 inet dhcp
post-up ip route add 10.0.0.0/24 dev eno1 table 1
post-up ip route add default via 10.0.0.1 table 1
post-up ip rule add from 10.0.0.35/32 table 1 priority 100
post-up ip route flush cache
pre-down ip rule del from 10.0.0.35/32 table 1 priority 100
pre-down ip route flush table 1
pre-down ip route flush cache
# VLANS
auto eno1.10
iface eno1.10 inet dhcp
post-up ip route add 10.0.10.0/24 dev eno1.10 table 2
post-up ip route add default via 10.0.10.1 table 2
post-up ip rule add from 10.0.10.65/32 table 2 priority 110
post-up ip route flush cache
pre-down ip rule del from 10.0.10.65/32 table 2 priority 110
pre-down ip route flush table 2
pre-down ip route flush cache
auto eno1.40
iface eno1.40 inet dhcp
post-up ip route add 10.0.40.0/24 dev eno1.40 table 3
post-up ip route add default via 10.0.40.1 table 3
post-up ip rule add from 10.0.40.40/32 table 3 priority 120
post-up ip route flush cache
pre-down ip rule del from 10.0.40.40/32 table 3 priority 120
pre-down ip route flush table 3
pre-down ip route flush cache
All the services running on the server were now working as they should be.
Additionally I have a docker host running on the server that hosts some containers which are bound to the different interfaces on the server.
Problem
Now the problem is that the rules I created apparently don't apply to traffic coming from the docker containers and I can't access them because the traffic is being blocked as invalid.
What would I need to do here for the docker containers to know which route to use according to the source IP?
Best Answer
The quick solution:
-i
match or match by source address, otherwise you need add directly connected routes into additional tables.DNAT
.tcpdump
and theconntrack
tool to troubleshoot issues.rp_filter
. It can drop the packets in some cases. Better set it into theloose
mode (sysctl -w net.ipv4.conf.all.rp_filter=2
).Update
After some tests in the lab I've found a perfect rule set. It requires only one mark value and one additional routing rule per uplink. It also handle complex cases, when you use public addresses on several interfaces.