Nginx Reverse Proxy Setup to Access Localhosts from the Internet

linuxnginxreverse-proxy

I have some troubles to configure Nginx as a reverse proxy in order to have my vm's with private ip addresses being reachable from the internet.

I drew a diagram with my current configuration.

My dedicated server runs on Proxmox hypervisor and has only 1 public ip address. A bridge has been installed on the hypervisor to get the vm’s access to the internet through their local ip address.

I made some research and found this is possible with a Nginx reverse proxy to reach the vm's with a private ip address (who are running websites) from internet but I have some troubles to set it up properly.

My config /etc/network/interfaces on the hypervisor:

root@ns568745:~# cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback


    # vmbr0: Bridging. Make sure to use only MAC adresses that were assigned to you.
    auto vmbr0
    iface vmbr0 inet static
            address 40.53.XX.XX/24
            gateway 40.53.XX.254
            bridge_ports eno1
            bridge_stp off
            bridge_fd 0





    auto vmbr1
    iface vmbr1 inet static
            address  192.168.4.254/24
            broadcast 192.168.4.255
            bridge_ports none
            bridge_stp off
            bridge_fd 0
            post-up echo 1 > /proc/sys/net/ipv4/ip_forward
            post-up iptables -t nat -A POSTROUTING -s '192.168.4.254/24' -o vmbr0 -j MASQUERADE
            post-down iptables -t nat -D POSTROUTING -s '192.168.4.254/24' -o vmbr0 -j MASQUERADE


    post-up   iptables -t nat -A PREROUTING -i vmbr1 -p tcp --dport 80 -j DNAT --to 192.168.4.2:80
    post-down iptables -t nat -D PREROUTING -i vmbr1 -p tcp --dport 80 -j DNAT --to 192.168.4.2:80
    post-up   iptables -t nat -A PREROUTING -i vmbr1 -p tcp --dport 443 -j DNAT --to 192.168.4.2:443
    post-down iptables -t nat -D PREROUTING -i vmbr1 -p tcp --dport 443 -j DNAT --to 192.168.4.2:443

Vm1 has the ip address 192.168.4.4 and run a website (with Apache ) that has the domain name "london.austria.com"

Vm2 has the ip address 192.168.4.5 and run a website (with Apache ) that has the domain name "manchester.austria.com"

I purchased a domain name ,let’s call it, “austria.com”.

In my registrar, I set up the A records for 2 subdomains :

london.austria.com with an A record pointing to 40.53.XX.XX

manchester.austria.com with an A record pointing to 40.53.XX.XX

VM1 config: 192.168.4.4 with subdomain "london.austria.com" running wwebiste with Apache.The firewall is disabled

VM2 config: 192.168.4.5 with subdomain "manchester.austria.com" running wwebiste with Apache.The firewall is disabled

Nginx has been installed in a LXC ubuntu container with the following steps:

systemctl start nginx

systemctl enable nginx

unlink /etc/nginx/sites-enabled/default

cd /etc/nginx/sites-available

vim reverse-proxy.conf

    server {
            listen 80;
            listen [::]:80;

            access_log /var/log/nginx/reverse-access.log;
            error_log /var/log/nginx/reverse-error.log;

            location / {
                        proxy_pass http://192.168.4.4:80;
      }
    }

ln -s /etc/nginx/sites-available/reverse-proxy.conf /etc/nginx/sites-enabled/reverse-proxy.conf

Tests I performed:

Inside the vm1:

I have access to the website by entering the ip address 192.168.4.4

I don't have access to the website by entering the domain name "london.austia.com" in a browser.

From home:

I can't access to the websites in the vm's.

I didn't even go further to test the vm2.

My new /etc/network/interfaces

root@ns568745:~# cat /etc/network/interfaces
    # This file describes the network interfaces available on your system
    # and how to activate them. For more information, see interfaces(5).

    # The loopback network interface
    auto lo
    iface lo inet loopback


    # vmbr0: Bridging. Make sure to use only MAC adresses that were assigned to you.
    auto vmbr0
    iface vmbr0 inet static
        address 40.53.XX.XX /24
        gateway 40.53.XX.XX .254
        bridge_ports eno1
        bridge_stp off
        bridge_fd 0





    auto vmbr1
    iface vmbr1 inet static
        address  192.168.4.254/24
        broadcast 192.168.4.255
        bridge_ports none
        bridge_stp off
        bridge_fd 0
        post-up echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up iptables -t nat -A POSTROUTING -s '192.168.4.254/24' -o vmbr0 -j MASQUERADE
        post-down iptables -t nat -D POSTROUTING -s '192.168.4.254/24' -o vmbr0 -j MASQUERADE



    iptables -t nat -A PREROUTING -d 40.53.XX.XX  -p tcp -m multiport --dports 80,443,22 -m comment --comment "nginx" -j DNAT --to-destination 192.168.4.2

    iptables -t nat -A POSTROUTING -s 192.168.4.0/24 -j SNAT --to-source 40.53.XX.XX

Edit 1 :

So,what I did:

In a shell, I entered:

iptables -t nat -F
iptables -t nat -A PREROUTING -d 40.53.XX.XX -p tcp -m multiport --dports 80,443,22 -m comment --comment "nginx" -j DNAT --to-destination 192.168.4.2
iptables -t nat -A POSTROUTING -s 192.168.4.0/24 -j SNAT --to-source 40.53.XX.XX

Then , from home, if I enter in a browser the url http://london.austria.com ,I have now access to this website.

But if I open another tab the url for the second website http://manchester.austria.com , the website displayed is still http://london.austria.com

Best Answer

Your DNAT rules are in the wrong direction (the packets coming from the Internet not your private network, i.e. interface vmbr0, need a translation of the destination address). Test them from a root shell, before you modify the post-up and post-down hooks:

iptables -t nat -A PREROUTING -i vmbr0 -p tcp -m multiport --dports 80,443 \
-m comment --comment "nginx" -j DNAT --to-destination 192.168.4.2

However, since you have a static IP address (40.53.XX.XX), I would rather replace both the MASQUERADE and DNAT rules with:

iptables -t nat -A PREROUTING -d 40.53.XX.XX -p tcp -m multiport --dports 80,443 \
-m comment --comment "nginx" -j DNAT --to-destination 192.168.4.2
iptables -t nat -A POSTROUTING -s 192.168.4.0/24 -j SNAT --to-source 40.53.XX.XX

since the SNAT target is faster (cf. netfilter manual). This set of rules will also provide a hairpin NAT configuration and allow you to connect from a VM to another using the server's public IP.

Edit: Since your network is working now, you can configure nginx with two server blocks:

server {
    listen 80 default;
    listen [::]:80 default;
    server_name manchester.example.com;

    location / {
        proxy_pass http://192.168.4.5:80;
    }
}

server {
    listen 80;
    listen [::]:80;
    server_name london.example.com;

    location / {
        proxy_pass http://192.168.4.4:80;
    }
}

Related Solutions

Nginx – How to Set Up as a Caching Reverse Proxy

I don't think that there is a way to explicitly invalidate cached items, but here is an example of how to do the rest. Update: As mentioned by Piotr in another answer, there is a cache purge module that you can use. You can also force a refresh of a cached item using nginx's proxy_cache_bypass - see Cherian's answer for more information.

In this configuration, items that aren't cached will be retrieved from example.net and stored. The cached versions will be served up to future clients until they are no longer valid (60 minutes).

Your Cache-Control and Expires HTTP headers will be honored, so if you want to explicitly set an expiration date, you can do that by setting the correct headers in whatever you are proxying to.

There are lots of parameters that you can tune - see the nginx Proxy module documentation for more information about all of this including details on the meaning of the different settings/parameters: http://nginx.org/r/proxy_cache_path

http {
  proxy_cache_path  /var/www/cache levels=1:2 keys_zone=my-cache:8m max_size=1000m inactive=600m;
  proxy_temp_path /var/www/cache/tmp; 


  server {
    location / {
      proxy_pass http://example.net;
      proxy_cache my-cache;
      proxy_cache_valid  200 302  60m;
      proxy_cache_valid  404      1m;
    }
  }
}

Linux – Issue with multiple bridging for KVM hosts

You have multiple interfaces with addresses on the same subnet with similar netmasks. The odds are a different route is being used for different packets, causing confusion. If you look at the output of

ip route show

You've probably got multiple default gateways as a result. For each set of IPs on the same subnet, give one of them the netmask you have, and the rest a netmask of 255.255.255.255. Those other interfaces will then only be used for packets originating or destined for that address and not others on the subnet.

Best Answer

Related Solutions

Nginx – How to Set Up as a Caching Reverse Proxy

Linux – Issue with multiple bridging for KVM hosts

Related Topic