From: http://www.linuxhorizon.ro/iproute2.html
You could try something like this:
echo "1 admin" >> /etc/iproute2/rt_tables
ip route add 192.168.48.16/24 dev wlan0 src 192.168.48.16 table admin
ip route add default via 192.168.48.16 dev wlan0 table admin
ip rule add from xx.xx.xx.xx/32 table admin
ip rule add to xx.xx.xx.xx/32 table admin
ip rule add from 192.168.48.16/32 table admin
ip rule add to xx.xx.xx.xx/32 table admin
warning, untested. But what it should do is to make sure that traffic comming
in on interface X also leaves it.
I have solved my own issue...for the curious, here's the blow-by-blow.
First, make sure to carefully check your sysctls, as Ubuntu has some enabled by default you would not expect to be enabled by default, namely RP checks.
sysctl -a | grep net.ipv4.conf.*
You might be surprised at what you find in here.
On Ubuntu 16.04, set the following in /etc/sysctl.conf...
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.ens192.rp_filter = 0
net.ipv4.conf.vrrp150.rp_filter = 0
Due to the RP defaults being set in sysctl, you'll need to do it on the 'all' level as well as each interface (as shown here).
Perform the following to activate it live (or just reboot).
sysctl -p
Second thing is to make sure your DvSwitch settings are correct (again, as above, the port group needs to have promiscuous mode, mac address change, and forged transmits enabled). It need not be the same port group as the rest of your VMs, even if the VLAN they reside in is the same. This helps you keep maximum security on your other VMs, while only exposing additional traffic to VMs that absolutely need it.
If they're on the same host, even if they're in different VM portgroups on the DvSwitch, that traffic will be switched locally on the vSwitch within the host, never egressing an uplink port.
For the truly paranoid, you can spin up a completely different DvSwitch with different uplinks (if you have many to burn) and a separate VLAN for them (pruned on the physical switch). This ensures they are utterly containerized and will see no other traffic except the traffic they are meant to see.
If you are using default multicasting with keepalived and have multiple uplink ports on the DvSwitch they are assigned to (which is best practice), make sure to set...
Advanced Settings > Net > Net.ReversePathFwdCheckPromisc to 1 on each host
to prevent multicast traffic looping back to the host (similar to https://doc.pfsense.org/index.php/CARP_Configuration_Troubleshooting).
This can be omitted if using unicast_peer
in your keepalived configuration.
Best Answer
A little recap
Recently Ubuntu adopted systemd, which handles the naming of network interfaces when the system boots. The naming will be associated to something physical in the hardware (like the slot the card is inserted in), so you can add/remove/replace network hardware and the names don't suddenly change on you (as it was before, with the
ethX
scheme).Now, you did the upgrade and the network interface was renamed, so your configuration in
/etc/network/interfaces
needed to be fixed. But, being confused, you first tried removing the virtual NIC and then added a new one. That gave you a different mac address AND did not solve your connectivity problem. Then you realized the renaming has happened, you fixed your configuration and the connectivity was back again.The problem (to be solved):
Your VM now can talk to another machine (
192.168.1.82
) in the same LAN, but cannot talk to the gateway anymore.The fact the VM can't receive an
arp reply
from the gateway means they can't even communicate at the ethernet level.Possible causes (to be checked):
iptables
or something that adds iptables rules, probably on the hypervisor, not the VM itself) applied to the gateway NIC or the VM NIC, which might allow traffic with the old mac address but not the new one.