WireGuard – Fix IP Routing Issue When Chaining Servers

iptablesnetworkingroutingvpnwireguard

I am attempting a chained/double-hop VPN setup where all client traffic passes through 2 servers before reaching the internet:

Client → Server1 → Server2 → Public Internet

All peers are on these private address blocks: 10.103.213.0/24 (IPv4) and fd6f:9403:2887:9cd6:10:103:213:0/112 (IPv6). Below are the configurations of the peers and the IPTables rules in place. (Please also note the names of the configuration files, in case it matters.)

Server2 Configuration

File: /etc/wireguard/wg0.conf

# Server2

[Interface]
PrivateKey = SERVER2_PRIVATE_KEY
Address = 10.103.213.2/24, fd6f:9403:2887:9cd6:10:103:213:2/112
ListenPort = 53701
SaveConfig = false

# CLIENTS

[Peer] # Server1
PublicKey = SERVER1_PUBLIC_KEY
PresharedKey = SERVER1_PRESHARED_KEY
# ↓ to allow traffic from client (10.103.213.11/32) via Server1 (10.103.213.1/32), allow both
AllowedIPs = 10.103.213.0/24, fd6f:9403:2887:9cd6:10:103:213:0/112

Firewall config. commands:

ufw allow 53701/udp comment 'WireGuard VPN'

iptables -A FORWARD -i wg0 -j ACCEPT &&
iptables -A FORWARD -o wg0 -j ACCEPT &&
ip6tables -A FORWARD -i wg0 -j ACCEPT &&
ip6tables -A FORWARD -o wg0 -j ACCEPT

iptables -t nat -A POSTROUTING -s 10.103.213.0/24 -o enp8s0 -j MASQUERADE
ip6tables -t nat -A POSTROUTING -s fd6f:9403:2887:9cd6:10:103:213:0/112 -o enp8s0 -j MASQUERADE

Server1 Configuration

File: /etc/wireguard/wg0.conf

# Server1

[Interface]
PrivateKey = SERVER1_PRIVATE_KEY
Address = 10.103.213.1/24, fd6f:9403:2887:9cd6:10:103:213:1/112
ListenPort = 53701
SaveConfig = false

# CLIENTS

[Peer] # Server2
PublicKey = SERVER2_PUBLIC_KEY
PresharedKey = SERVER1_PRESHARED_KEY
Endpoint = SERVER2_PUBLIC_IP:53701
AllowedIPs = 10.103.213.2/32, fd6f:9403:2887:9cd6:10:103:213:2/128
#PersistentKeepalive = 25

[Peer] # PC
PublicKey = CLIENT_PUBLIC_KEY
PresharedKey = CLIENT_PRESHARED_KEY
AllowedIPs = 10.103.213.11/32, fd6f:9403:2887:9cd6:10:103:213:11/128

Firewall config. commands:

ufw allow 53701/udp comment 'WireGuard VPN'

iptables -A FORWARD -i wg0 -j ACCEPT &&
iptables -A FORWARD -o wg0 -j ACCEPT &&
ip6tables -A FORWARD -i wg0 -j ACCEPT &&
ip6tables -A FORWARD -o wg0 -j ACCEPT

Client Configuration

# CLIENT: PC

[Interface]
PrivateKey = CLIENT_PRIVATE_KEY
Address = 10.103.213.11/24, fd6f:9403:2887:9cd6:10:103:213:11/112
DNS = 10.103.213.1, fd6f:9403:2887:9cd6:10:103:213:1

[Peer] # Server1
PublicKey = SERVER1_PUBLIC_KEY
PresharedKey = CLIENT_PRESHARED_KEY
Endpoint = SERVER1_PUBLIC_IP:53701
AllowedIPs = 0.0.0.0/0, ::0/0
#PersistentKeepalive = 25

Now, like I said, I am able to ping Server1 (10.103.213.2) and Server2 (10.103.213.1) from the Client (10.103.213.11) on the private (WireGuard) network when the VPN is enabled on all peers, but I cannot access the internet.

I know I'm missing some much needed IP routes or IPTables rules, but despite trying to find a solution that I can understand for days, I've had no success. I see that there's little hope for me without actually reading a book on Linux networking/firewall.

For now, I am hoping someone can help me with a solution along with an explanation as to what we are doing and why, so I can understand better and take notes.

Thank you very much!

NOTES:

If I'm missing any useful information, please ask and I'll be happy to get it.
Client runs macOS. Server1 and Server2 run Debian 11 "Bullseye". Just an FYI, in case it's relevant.
Server1 and Server2 have Unbound installed and set up for local DNS resolution. That's why you see DNS = 10.103.213.1, fd6f:9403:2887:9cd6:10:103:213:1 in Client config. If not for that, I'd be using either CloudFlare or Google's IPs there.
Someone on #wireguard IRC suggested that I try adding Table = 123 under [Interface] in Server1's WireGuard config. and then running the command ip rule add iif wg0 table 123. That didn't work and I couldn't understand what it's supposed to do either (I couldn't make sense of the man pages or the technical details).
From my reading I came to the conclusion that iptables and ufw can be used together—except one has to be careful when using iptables-persistent. You need to run netfilter-persistent save even after running ufw commands for the firewall rules to be persistent across reboots even if ufw status says they are in place. If for some reason you reboot before saving, delete the UFW rules and add them again and then run netfilter-persistent save.

Best Answer

The kind folks over at #wireguard IRC channel on Libera.Chat helped me out!

The issues with my config. were as follows:

Server1's config has Server2 with AllowedIPs of just Server2's IP addresses. That wont permit "The Internet". It needs to be 0.0.0.0/0, ::0/0.
Server1's WireGuard interface must be configured to add routes (for all entries in AllowedIPs) to a custom IP routing table (let's call it wireguard2x) instead of the main table. Then add an IP policy rule that says that traffic with the input interface (iif) wg0 go to the custom table—which also means other kinds of traffic take the default route, per usual.

Corrected Configuration

Server2 Configuration

File: /etc/wireguard/wg0.conf

# Server2

[Interface]
PrivateKey = SERVER2_PRIVATE_KEY
Address = 10.103.213.2/24, fd6f:9403:2887:9cd6:10:103:213:2/112
ListenPort = 53701
SaveConfig = false

# CLIENTS

[Peer] # Server1
PublicKey = SERVER1_PUBLIC_KEY
PresharedKey = SERVER1_PRESHARED_KEY
# ↓ to allow traffic from client (10.103.213.11/32) via Server1 (10.103.213.1/32), allow both
AllowedIPs = 10.103.213.0/24, fd6f:9403:2887:9cd6:10:103:213:0/112

Firewall config. commands:

ufw allow 53701/udp comment 'WireGuard VPN'

iptables -A FORWARD -i wg0 -j ACCEPT &&
iptables -A FORWARD -o wg0 -j ACCEPT &&
ip6tables -A FORWARD -i wg0 -j ACCEPT &&
ip6tables -A FORWARD -o wg0 -j ACCEPT

iptables -t nat -A POSTROUTING -s 10.103.213.0/24 -o enp7s0 -j MASQUERADE
ip6tables -t nat -A POSTROUTING -s fd6f:9403:2887:9cd6:10:103:213:0/112 -o enp7s0 -j MASQUERADE

Server1 Configuration

File: /etc/wireguard/wg0.conf

# Server1

[Interface]
PrivateKey = SERVER1_PRIVATE_KEY
Address = 10.103.213.1/32, fd6f:9403:2887:9cd6:10:103:213:1/128
ListenPort = 53701
Table = wireguard2x
# ↓ should only be set if resolvconf or openresolv is installed on the system, otherwise let the system use defaults
# ↓ is unncessary if local DNS resolution is set up
#DNS = 1.1.1.1, 1.0.0.1, 2606:4700:4700::1111, 2606:4700:4700::1001
DNS = 10.103.213.1, fd6f:9403:2887:9cd6:10:103:213:1
SaveConfig = false

# CLIENTS

[Peer] # Server2
PublicKey = SERVER2_PUBLIC_KEY
PresharedKey = SERVER1_PRESHARED_KEY
Endpoint = SERVER2_PUBLIC_IP:53701
AllowedIPs = 0.0.0.0/0, ::0/0
#PersistentKeepalive = 25

[Peer] # PC
PublicKey = CLIENT_PUBLIC_KEY
PresharedKey = CLIENT_PRESHARED_KEY
AllowedIPs = 10.103.213.11/32, fd6f:9403:2887:9cd6:10:103:213:11/128

Firewall config. commands:

ufw allow 53701/udp comment 'WireGuard VPN'

iptables -A FORWARD -i wg0 -j ACCEPT &&
iptables -A FORWARD -o wg0 -j ACCEPT &&
ip6tables -A FORWARD -i wg0 -j ACCEPT &&
ip6tables -A FORWARD -o wg0 -j ACCEPT

echo 123 wireguard2x >> /etc/iproute2/rt_tables
ip rule add iif wg0 table wireguard2x

Client Configuration

# CLIENT: PC

[Interface]
PrivateKey = CLIENT_PRIVATE_KEY
Address = 10.103.213.11/32, fd6f:9403:2887:9cd6:10:103:213:11/128
DNS = 10.103.213.1, fd6f:9403:2887:9cd6:10:103:213:1

[Peer] # Server1
PublicKey = SERVER1_PUBLIC_KEY
PresharedKey = CLIENT_PRESHARED_KEY
Endpoint = SERVER1_PUBLIC_IP:53701
AllowedIPs = 0.0.0.0/0, ::0/0
#PersistentKeepalive = 25

NOTES:

You can see all the policy routing rules that are currently in effect using this command: ip rule list or ip rule
View the routing tables with, for e.g., ip route show table wireguard2x or ip route list table wireguard2x.
To flush the route cache: ip route flush cache
Further reading: Linux Advanced Routing & Traffic Control (LARTC) HOWTO
You can monitor Client's network traffic flow—if on Linux, using sudo iptraf-ng; on macOS using sudo iftop. (To be run on the client.)
- I found running that hard to parse due to too many live changes. So I was suggested trying route get 192.0.2.0 (which shows the interface being used, e.g., interface: utun2) and then running ifconfig <interface> (e.g., ifconfig utun2) on macOS. The latter should show the Client's private (WireGuard peer) IP address, e.g., 10.103.213.11, confirming that the traffic is being routed through WG interface. It's not much, but it's a start.
- UDPATE: traceroute is brilliant for this! For e.g., traceroute 8.8.8.8. (Hat tip to Chrispus Kamau.)
curl ipinfo.io to check your IP address with the VPN enabled. (To be run on the client.)

RELATED:

https://superuser.com/a/1651179

Related Solutions

VirtualBox Networking – How to Set Up Networking for Host and Guest

Try this:

Setup the virtualbox to use 2 adapters:
- The first adapter is set to NAT (that will give you the internet connection).
- The second adapter is set to host only.
Start the virtual machine and assign a static IP for the second adapter in Ubuntu (for instance 192.168.56.56). The host Windows will have 192.168.56.1 as IP for the internal network (VirtualBox Host-Only Network is the name in network connections in Windows). What this will give you is being able to access the apache server on ubuntu, from windows, by going to 192.168.56.56. Also, Ubuntu will have internet access, since the first adapter (set to NAT) will take care of that.
Now, to make the connection available both ways (accessing the windows host from the ubuntu guest) there's still one more step to be performed. Windows will automatically add the virtualbox host-only network to the list of public networks and that cannot be changed. This entails that the firewall will prevent proper access.
To overcome this and not make any security breaches in your setup:
- go to the windows firewall section, in control panel,
- click on advanced settings. In the page that pops up,
- click on inbound rules (left column), then on new rule (right column). Chose custom rule, set the rule to allow all programs, and any protocol. For the scope, add in the first box (local IP addresses) 192.168.56.1, and in the second box (remote IP) 192.168.56.56. Click next, select allow the connection, next, check all profiles, next, give it a name and save.

That's it, now you have 2 way communication, with apache/any other service available as well as internet. The final step is to setup a share. Do not use the shared folders feature in virtualbox, it's quite buggy especially with windows 7 (and 64 bit). Instead use samba shares - fast and efficient.

Follow this link for how to set that up: https://wiki.ubuntu.com/MountWindowsSharesPermanently

Iptables – http request via iptables –to-destination ip redirect results in no response

It sounds like you could have a routing problem. Check using tcpdump on both servers and pointing tcpdump with the -i option to the correct ethX device to check if packets are being sent through the right network device on server1 and these packets also arrive at server2.

tcpdump -i ethX host 1.1.1.1 and port 80

Could it be that the packets arrive at server2 but server2 doesn't know where to send the replies to? Does server2 have a correct default gateway configured?

Check it with

route -n

Server2 Configuration

Server1 Configuration

Client Configuration

Best Answer

Corrected Configuration

Server2 Configuration

Server1 Configuration

Client Configuration

Related Solutions

VirtualBox Networking – How to Set Up Networking for Host and Guest

Iptables – http request via iptables –to-destination ip redirect results in no response

Related Topic