On my my VM host, I have two libvirt virtual networks:
- virbr0: 192.168.122.1/255.255.255.0
- virbr1: 192.168.130.1/255.255.255.0
I have a VM behind each of those virtual networks. Machines can see the host, and see the Internet. However, when I try to connect from one network to the other, I get "connection refused" even though I am connecting to a port I know is open.
I have disabled UFW (host is Ubuntu) and still fails. Nothing is logged, so I'm not sure which rule is blocking it. My FORWARD block (I assume created by virtlib) is:
Chain FORWARD (policy ACCEPT)
target prot opt source destination
DOCKER all -- anywhere anywhere
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere 192.168.122.0/24 ctstate RELATED,ESTABLISHED
ACCEPT all -- 192.168.122.0/24 anywhere
ACCEPT all -- anywhere anywhere
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
ACCEPT all -- anywhere 192.168.130.0/24 ctstate RELATED,ESTABLISHED
ACCEPT all -- 192.168.130.0/24 anywhere
ACCEPT all -- anywhere anywhere
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable
If I clear all the firewall rules, then the VMs on different virtual networks can talk to each other, but of course they can't talk off the VM hosts.
I found if I got rid of these rules:
-A FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -o virbr1 -j REJECT --reject-with icmp-port-unreachable
-A FORWARD -i virbr1 -j REJECT --reject-with icmp-port-unreachable
The VMs could then talk to each other. That's great, but how do I instruct libvirt to modify how it creates the forward rules so it doesn't prohibit traffic between the virtual networks?
Thanks for any tips, pointers, or URLs!
Best Answer
This is happening because you've configured your virtual networks as "nat" networks.
Such a network is set up so that it can only access:
In particular, access to any other virtual networks on the host is blocked, as you've discovered.
To solve this problem requires two steps:
Reconfigure each virtual network as a "routed" network, without NAT. In this case, libvirtd will not attempt to isolate the virtual networks from each other, but it also will not perform any NAT.
You will need to do this by editing the XML (with
virsh net-edit networkname
; the virt-manager GUI cannot make this change):should be changed to:
This change takes effect when you shutdown all VMs using the network, stop the network (
virsh net-stop networkname
), restart the network (virsh net-start networkname
), and restart all VMs using the network.You will then also have to insert your own masquerading rules, if you want the virtual machines to access the Internet.
For example, in the nat table section of
/etc/ufw/before.rules
:Alternately, you can forget all that, and create a new virtual network which is isolated, and give each VM a second virtual NIC which is connected to the isolated network. The VMs can then communicate via this network.