Setup
____________________________ ____________________________
| Host | | Client |
| Public IP: 66.66.66.66 | | |
| Internal IP: 10.0.3.1 | <---------> | Internal IP: 10.0.3.192 |
---------------------------- ----------------------------
- Host with public IP (=66.66.66.66)
- Linux Container (LXC), called client in the rest of the post, started with
veth
network option on which a webserver is running. - On the host, the firewall is configured with
ufw
and the respective ports are opened and forwarded to the client, this works.
iptables
port forwarding in the /etc/ufw/before.rules
:
# nat Table rules
*nat
:PREROUTING ACCEPT [0:0]
-A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to-destination 10.0.3.192:80
-A PREROUTING -i eth0 -p tcp --dport 443 -j DNAT --to-destination 10.0.3.192:443
# don't delete the 'COMMIT' line or these rules won't be processed
COMMIT
/etc/network/interfaces
on host:
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
auto eth0
iface eth0 inet static
address 66.66.66.66
netmask 255.255.255.0
network 66.66.66.0
broadcast 66.66.66.255
gateway 66.66.66.1
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 8.8.8.8
/etc/network/interfaces
on client:
# The loopback network interface
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address 10.0.3.192
netmask 255.255.255.0
gateway 10.0.3.1
broadcast 10.0.3.255
Port scan from client
Starting Nmap 5.21 ( http://nmap.org ) at 2014-04-18 18:46 CEST
Nmap scan report for 66.66.66.66
Host is up (0.00011s latency).
PORT STATE SERVICE
80/tcp closed http
443/tcp closed https
Port scan from "outside"
Starting Nmap 6.40 ( http://nmap.org ) at 2014-04-18 19:54 CEST
Nmap scan report for 66.66.66.66
Host is up (0.41s latency).
PORT STATE SERVICE
80/tcp open http
443/tcp open https
Nmap done: 1 IP address (1 host up) scanned in 1.28 seconds
iptables configuration:
$ iptables -t nat -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
DNAT tcp -- anywhere anywhere tcp dpt:http to:10.0.3.192:80
DNAT tcp -- anywhere anywhere tcp dpt:https to:10.0.3.192:443
DNAT tcp -- anywhere anywhere tcp dpt:43211 to:10.0.3.192:22
DNAT tcp -- anywhere anywhere tcp dpt:http-alt to:10.0.3.192:8080
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
MASQUERADE tcp -- anywhere 10.0.3.192 tcp dpt:http
MASQUERADE tcp -- anywhere 10.0.3.192 tcp dpt:https
MASQUERADE all -- 10.0.3.0/24 !10.0.3.0/24
Versions
- Ubuntu 12.04 (host as well as client)
- LXC (1.0.3-0ubuntu1~ubuntu12.04.1~ppa1)
Desired result
I want to access the webserver on port 80 on the client from the client itself over the public IP, e.g.
$ curl http://66.66.66.66
curl: (7) couldn't connect to host
should give the same result as
$ curl http://10.0.3.192
<html>.....
Strangely $ ping 66.66.66.66
from the client inside the private network works though.
What I tried
I see that this problem is very much related to the so called Hairpin NAT / Loopback NAT, though I could not configure the masquerading with the following rules, such that it works (in /etc/ufw/before.rules
after the PREROUTING
entries):
# nat Table rules
*nat
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -o eth0 -d 10.0.3.192 -p tcp --dport 80 -j MASQUERADE
-A POSTROUTING -o eth0 -d 10.0.3.192 -p tcp --dport 443 -j MASQUERADE
# don't delete the 'COMMIT' line or these nat table rules won't be processed
COMMIT
My guess to the solution is that the bridged network or just my incapability of adapting the MASQUERADE rule correctly to my setup is the problem. Any suggestions and comments, also to the setup in general are appreciated.
Related questions or articles, but not helping
- LXC networking with public IP
- About different network modes of LXC: http://containerops.org/2013/11/19/lxc-networking/
- https://coderwall.com/p/k0gutq
- Using NAT for container with private IPs (explicitly written that it will not work from inside the private network): https://openvz.org/Using_NAT_for_container_with_private_IPs
- Clean LXC + NAT configuration: https://coderwall.com/p/k0gutq
- iptables – how to access local server using external ip (disadvise of doing so, suggest setting up DNS or editing host file, but based on FQDN): http://www.tomshardware.co.uk/forum/12532-42-iptables-access-local-server-external
- Is NAT Loopback on my router a security problem? (Gives SNAT rule,: https://security.stackexchange.com/questions/16343/is-nat-loopback-on-my-router-a-security-problem
- NAT loopback using iptables: http://for-invent.com/nat-loopback-using-iptables/
Best Answer
(I wrote first that's it's a routed network not a bridged network. Now I see it's using LXC, so I don't really know. but if PREROUTING works already, I sure hope what I wrote below will work)
You're using the PREROUTING chain, this chain alters (about to be) routed packets only, that is packets coming from somewhere else. Packets generated on Host itself are not routed (they're just well... output, as any host can do), so this chain never receives curl's packets. curl just tried to connect on Host as usual. There's an other chain to catch locally generated packets: OUTPUT.
So, you also duplicate the DNAT rules to the (-t nat) OUTPUT chain, with some modifications: OUTPUT doesn't want input interface. You replace
-i eth0
with-o lo ! -s 127.0.0.0/8
or just-d 66.66.66.66
or something else but you need some restriction else any web request to anywhere will go to Client. First example is independent of Host's IP(s), second one is shorter, as you like. It's not a typo, if you connect to 66.66.66.66 from itself, it's a local packet, so it goes through thelo
interface. But then because the 127.0.0.1-like destinations can't be rerouted (curl http://127.0.0.1/
would timeout instead of saying couldn't connect ) 127.0.0.0/8 is put as an exception.And that's it. Everything else is as usual handled by conntrack. You should remove the 2 port specific MASQUERADE rules. They're not needed and they may even interfere (I think if they work, the result is that Client will always see Host as source on the web server)
so, the short answer is (using -t nat):