Iptables – How to allow iptables to properly use a tunnel rather than NAT

iptablesnat;site-to-site-vpn

I have two NAT networks connected via a point-to-point tunnel as follows:

  | Machine A  |                                         | Machine B  |
  | 1.2.3.4    |<-10.101.3.2-------tun------10.101.3.1-> | 4.5.6.7    |
  | 10.110.0.1 |                                         | 10.100.1.1 |
  -----+--------                                         --------+-----
       |                                                         |
 [ 10.110.1.0/24 net ]                                   [ 10.100.0.0/16 net]

Machine A has a public IP on eth0 and a private IP 10.110.1.1 on vmbr0, and is connected over the internet via a p-t-p VPN with Machine B, which has a public IP on en0 and a private IP on en1 (and a number of other private networks)

There's clients on both of the LANs using NAT with thw two public IPs. This works fine. (It's done using iptables on Machine A and pfSense on Machine B)

Routing is set up such that Machine A can ping Machine B's private ip 10.100.1.1 and can access all hosts on network B 10.100.0.0/16. Likewise, Machine B (and all hosts on network B) can access Machine A, and all hosts on Network A 10.110.1.0/16

However, hosts on Network A can't establish connections with hosts on network B (E.G. 10.110.1.2 can't ping nor SSH to 10.100.10.50)

I suspect this has something to do with the iptables setup on A, because SSH from 10.100.10.50 to 10.110.1.2 works but I can't then SSH in the reverse direction. That says to me that my iptables FILTER rule for RELATED,ESTABLISHED works but somehow I suspect that machine A is trying to NAT and use eth0 for new connections.

I added ACCEPT destination 10.100.0.0/16 iptables rules everywhere I thought they were need, and then even in places I didn't think they were needed when things didn't work, and still can't get outgoing connections from network A to network B working.

How can I properly connect two NAT'ed private networks via a point-to-point tunnel using iptables?

Additional info:

iptables on Machine A:

Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             10.100.0.0/16       
DNAT       tcp  --  anywhere             anywhere             tcp dpt:http to:10.110.1.3:80
DNAT       tcp  --  anywhere             anywhere             tcp dpt:https to:10.110.1.3:443
DNAT       tcp  --  anywhere             anywhere             tcp dpt:xmpp-client to:10.110.1.4:5222
DNAT       tcp  --  anywhere             anywhere             tcp dpt:5281 to:10.110.1.4:5281
DNAT       tcp  --  anywhere             anywhere             tcp dpt:64738 to:10.110.1.4:64738
DNAT       udp  --  anywhere             anywhere             udp dpt:64738 to:10.110.1.4:64738
DNAT       udp  --  anywhere             anywhere             udp multiport dports isakmp,ipsec-nat-t to:10.110.1.10:1194

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             10.100.0.0/16       
ACCEPT     all  --  anywhere             10.101.0.0/16       
MASQUERADE  all  --  10.110.1.0/24        anywhere            
MASQUERADE  all  --  10.110.1.0/24        anywhere            
MASQUERADE  all  --  10.110.1.0/24        anywhere

Best Answer

Sooo, this was a bonehead mistake on my end! Everything on Machine A was set up properly, it was actually Machine B's firewall rules blocking this. Machine B had a rule allowing traffic from 10.101.3.2 but was missing the rule allowing traffic from 10.110.1.0/24. Adding that rule corrected the issue.

I discovered the problem using tcpdump and nc as follows:

  1. On 10.100.10.50 I executed nc -kl 60321 to listen for connections on port 60321 (I chose a random port that I knew wasn't used by services we run in order to minimize noise in tcpdump)
  2. On Machines A and B I executed tcpdump -s0 -X -i $dev port 60321 where I set $dev to either the NAT'ed networks' interfaces (vmbr0 or vmx0) or to the p-t-p interfaces (tun0)
  3. On 10.110.1.2 I sent data to 10.100.10.50 using: echo -e "test1\ntest2\ntest3" | nc 10.100.10.50 60321
  4. I saw the traffic on Machine A's LAN, both Machine A and B's tun0 interface, but not on Machine B's LAN, so that meant the problem was with routing / firewall on Machine B
  5. I facepalmed and added the missing firewall rule
Related Topic