Wireguard – How to Calculate Working AllowedIPs for Wireguard on Android

MASQUERADE/POSTROUTING rules do not change where certain traffics go. Routes do. The problem is that you have a default route (or what's equivalent) that leads traffics into the pia tunnel.

You will need to make use of policy routing for the replying traffics from the wireguard server:

# ip r add 192.168.1.1 dev eth0 table 123
# ip r add default via 192.168.1.1 table 123
# ip rule add iif lo ipproto udp sport 51820 lookup 123

The first command could be optional. Make sure you replace 192.168.1.1 and eth0 with the LAN IP of your router and the interface name of your Ethernet NIC correspondingly. (You can copy them from the output of ip r, i.e. routes in the main table.) The number 123 is arbitrary. iif lo limits the rule to UDP traffics with source port of 51820 from the host itself (but not such traffics from another host).

You should probably start with using Table=off in the wg-quick conf on both S and P. The value of AllowedIPs= will not cause changes to the routes / policy routing rules on them then.

EDIT: Actually it should be fine to leave Table= untouched on P unless you need AllowedIPs= of S on it to be 0.0.0.0/0 instead of 192.168.60.0/24 for some reasons, e.g. need traffics originates from itself to be routed S. You don't need to mess with the routes and routing rules on P yourself since even the prefix in Address=192.168.60.2/24 should get the necessary route configured. The next paragraph probably does not apply to what you need -- although it might gives you some extra insights on how things work.

And you should probably use an additional IP subnet for S and P, e.g. 192.168.59.0/30. This will save you the hassle of needing extra ip rule. Remember to add the subnet route for 192.168.60.0/24 on P though, as with Table=off, only prefix routes will be added by the kernel for the prefix(es) in the Address= field(s). Make good use of PostUp= (and PreDown=) btw.

I don't suppose you want to route traffics that originates from S itself to P, so you probably want the following ip rule instead:

# ip rule add iif wg0 from 192.168.60.0/24 lookup 200

If you really need want to route e.g. traffics other than the ssh and wireguard server replies to P, you can additionally have:

# ip rule add iif lo lookup 200
# ip rule add iif lo ipproto tcp sport 22 lookup main
# ip rule add iif lo ipproto udp sport 51820 lookup main

Note: you can't just match with from 192.168.60.1 added in the first rule and omit the other two, because for non-replying traffics, the source address is often (if not always) chosen based on the decided route -- it's not set yet at this point.

Note that the order of the command normally determines the priority, so make sure you add the "superset" rule before the "subset" rules, otherwise the latter will be overridden by the former.

Also it's best to keep table 200 empty until all the desire rules are in position, otherwise remote access of the host could be cut off.

Finally nexthop makes no sense in route to an L3 tunnel:

# ip route add default dev wg0 table 200

P.S. Make sure you didn't just allow IP forwarding in the firewall but also enable it with sysctl.