I currently have my mesh setup like this:
With the wireguard config similar to this on every node:
[Interface]
Address = 10.1.0.1/32
PrivateKey =
ListenPort = 5888
[Peer] # example public node [1-3]
PublicKey =
AllowedIPs = 10.1.0.2/32
Endpoint = X.X.X.X:5888
PersistentKeepalive = 25
[Peer] # example node behind cgnat [4-6]
PublicKey =
AllowedIPs = 10.1.0.51/32
PersistentKeepalive = 25
This allows me to ping all the green lines in the graph above. But I can't ping any of the reds between the nodes in the CGNAT. How can I?
ATTEMPT 1 (without Node3
)
example CGNAT (Node4
)
[Interface] # NODE 4
Address = 10.3.0.3/32
PrivateKey =
[Peer] # NODE 1, 5 & 6
PublicKey =
AllowedIPs = 10.3.0.1/32,10.3.0.51/32,10.3.0.52/32
Endpoint = X.X.X.X:5888
PersistentKeepalive = 25
[Peer] # NODE 2
PublicKey =
AllowedIPs = 10.3.0.2/32
Endpoint = X.X.X.Y:5888
PersistentKeepalive = 25
example Public Endpoint (Node1
)
[Interface] # NODE 1
Address = 10.3.0.1/32
PrivateKey =
ListenPort = 5888
[Peer] # NODE 5
PublicKey =
AllowedIPs = 10.3.0.51/32
PersistentKeepalive = 25
[Peer] # NODE 6
PublicKey =
AllowedIPs = 10.3.0.52/32
PersistentKeepalive = 25
[Peer] # NODE 2
PublicKey =
AllowedIPs = 10.3.0.2/32
Endpoint = X.X.X.Y:5888
PersistentKeepalive = 25
[Peer] # NODE 4
PublicKey =
AllowedIPs = 10.3.0.3/32
PersistentKeepalive = 25
I have also ran on Node1
:
$ sysctl -w net.ipv4.ip_forward=1
$ sysctl -w net.ipv4.conf.maxnet.forwarding=1
where maxnet
is my wg name.
Best Answer
If nodes 4,5,6 are each behind their own CGNAT, then as usual, they can't manage to do proper communication between them. They have to rely on nodes 1,2,3 to relay traffic. So these CGNAT nodes will require extra AllowedIPs for some public peers in their settings to allow other CGNATs nodes through the public nodes, and the public nodes must be set as routers.
Example for node 4 assumed to have IP address 10.1.0.51/32 and using node 2 as routing node:
Other CGNAT nodes' IP addresses are expected to be routed through node 2 and should be added to that peer's AllowedIPs. This should also add them automatically to the routing table.
Node 2 must now also be a router, at least on its WireGuard interface which will be both ingress and egress. On Linux this would be done for example with (assuming an interface wg0):
Its firewall rules must also allow forwarded traffic on wg0.
Please note that there's no need to define the other CGNAT peers on the CGNAT nodes, and if defined they must not have other CGNAT nodes' IP addresses in AllowedIPs:
Nodes 5 and 6 must have a compatible configuration (also using node 2 as router). You could also imagine having instead:
[...]
In all cases, for any topology change even if due to a failure rather than an intended change, a way to synchronize configuration changes in all affected peers is needed and there's currently no dedicated tool to do this.
As a conclusion, here's a blog where BGP (that would be the missing tool) is used along WireGuard, with multiple addresses and also one interface per peer node with only that peer defined to sidestep cryptokey routing. There's probably something to be learnt from this but the topic is way too advanced for me.
Route-based VPN on Linux with WireGuard