IPTables – Allow L2TP/IPSEC VPN Behind Firewall

I asked about this on the Smoothwall Express forums but they are understandably reluctant to provide to much help. After all, they really want to sell the commercial version.

I need to provide an L2TP/IPSEC VPN for remote support of some new machinery we're getting soon (no choice about that part). As the remote user also needs to be authenticated against Active Directory I need to run the VPN on our Windows (2003) server, rather than directly on the firewall.

Because Smoothwall Express has no provision for forwarding protocols from it's normal management interface this needs to be done by editing the iptables settings in rc.firewall.up. Herein lies my problem. I don't know iptables and feel that even if I take the time to read up about it the risks of me screwing it up are significant.

The following section currently exists in rc.firewall.up:

# IPSEC
/sbin/iptables -N secin
/sbin/iptables -A secin -i ipsec0 -j ACCEPT
/sbin/iptables -A INPUT -j secin

/sbin/iptables -N secout
/sbin/iptables -A secout -i ipsec0 -j ACCEPT
/sbin/iptables -A FORWARD -j secout

[some unrelated stuff here]

# IPSEC
/sbin/iptables -N ipsec
/sbin/iptables -A ipsec -p udp --destination-port 500 -j ACCEPT
/sbin/iptables -A ipsec -p udp --destination-port 4500 -j ACCEPT
/sbin/iptables -A ipsec -p 50 -j ACCEPT
/sbin/iptables -A ipsec -p 51 -j ACCEPT
/sbin/iptables -A block -i ppp0 -j ipsec
/sbin/iptables -A block -i ippp0 -j ipsec
if [ "$RED_DEV" != "" ]; then
    /sbin/iptables -A block -i $RED_DEV -j ipsec
fi

Unless I'm mistaken that second bock will cause the traffic I want forwarded to be swallowed instead. I tried commenting out that second block and adding this one, which is adapted from the lines required for PPTP passthrough (which does work) and what I've been able to obtain from various sources:

# L2TP/IPSEC
/sbin/iptables -N l2tp
/sbin/iptables -A l2tp -p udp --destination-port 500 --dst 192.169.0.7 -j ACCEPT
/sbin/iptables -A l2tp -p udp --destination-port 1701 --dst 192.169.0.7 -j ACCEPT
/sbin/iptables -A l2tp -p udp --destination-port 4500 --dst 192.169.0.7 -j ACCEPT
/sbin/iptables -A l2tp -p 50 --dst 192.169.0.7 -j ACCEPT
/sbin/iptables -A l2tp -p 51 --dst 192.169.0.7 -j ACCEPT
/sbin/iptables -I FORWARD -j l2tp
/sbin/iptables -t nat -N l2tp
/sbin/iptables -t nat -A l2tp -i $RED_DEV -p tcp --dport 500 -j DNAT --to 192.169.0.7:500
/sbin/iptables -t nat -A l2tp -i $RED_DEV -p tcp --dport 1701 -j DNAT --to 192.169.0.7:1701
/sbin/iptables -t nat -A l2tp -i $RED_DEV -p tcp --dport 4500 -j DNAT --to 192.169.0.7:4500
/sbin/iptables -t nat -A l2tp -i $RED_DEV -p 50 -j DNAT --to 192.169.0.7
/sbin/iptables -t nat -A l2tp -i $RED_DEV -p 51 -j DNAT --to 192.169.0.7
/sbin/iptables -t nat -A PREROUTING -j l2tp

.. but that doesn't work. Where did I stuff up?

Incidentally, $RED_DEV translates to the public facing interface and obviously 192.168.0.7 is my VPN server.

Update:

It turns out that the above settings do work – sort of. I'm testing this from home, where I have a MacBook and a several of Windows XP machines. I can connect just fine from the Mac but cannot connect using any of the Windows machines. Unfortunately, the machinery company that requires this VPN for remote support uses only Windows XP. 🙁