NFTables – How to Bypass NFTables Drop Rule

firewallnftables

tldr; bridge (see below) doesn't work if there is a matching drop in another table (like the default rules of firewalld).

Hello,
I'm building my own VM lib (kind of quickemu).
I have a problem with the firewall rules for bridging. I started with iptables-nft and it worked fine until I tested my scripts on fedora. No matter what I do, firewalld blocks everything. So I went with nftables to try to bypass firewalld rules directly, but even then I can't find a way to make it work without deleting firewalld rules.

It's my understanding that the priority only affects the order in which the rules are tested, if there is a drop, even at the end, the packet is dropped?

Is there a way to bypass this behavior that I don't know about? I tried looking into marks, but I'm not sure it's the right solution.

Here is the rules I use for the bridge (works perfectly is there is no other table in the ruleset):

#!/usr/bin/nft -f
# vim:set ts=2 sw=2 et:

table ip QEMU
delete table ip QEMU
table ip QEMU {
  chain input {
    type filter hook input priority filter - 1;

    ct state {established,related} iifname "virbr0" counter accept
  }

  chain forward {
    type filter hook forward priority filter - 1;

    ct state {established,related} iifname "virbr0" counter accept
  }

  chain postrouting {
    type nat hook postrouting priority srcnat;

    iifname "virbr0" counter masquerade
  }
}

Best Answer

Nitpicking: despite virbr0 being a bridge interface, this is about routing, not about bridging. The firewall happens on the routing side of the bridge: the bridge interface itself, not on the bridge ports (which would require a table bridge rather than a table ip firewall). So the word bridging won't be mentioned again below.

When a packet is dropped within netfilter, including nftables, it stays dropped. When a packet is accepted, it will then just continue to traverse additional hooks possibly making this packet dropped later, as reminded in nft(8):

accept

Terminate ruleset evaluation and accept the packet. The packet can still be dropped later by another hook, for instance accept in the forward hook still allows one to drop the packet later in the postrouting hook, or another forward base chain that has a higher priority number and is evaluated afterwards in the processing pipeline.

drop

Terminate ruleset evaluation and drop the packet. The drop occurs instantly, no further chains or hooks are evaluated. It is not possible to accept the packet in a later chain again, as those are not evaluated anymore for the packet.

This causes firewalling tools with a default drop behavior and accepting only some flows (which is a good behavior from a security standpoint) rather than tools with a default accept behavior and dropping some flows (which is not so good from a security standpoint) to occult the decision from other firewalling tools: they drop things that other tools would have chosen to accept.

So to have two tools co-exist, actions have usually to be done on both of them.

For this case, one can tell firewalld to ignore (ie: accept) flows related to one or more specific interface(s), so the handling of this specific interface remains in the control of an other tool. If such handling is not done elsewhere, this can possibly open a security hole instead, so this has to be considered properly.

This answer requires firewalld >= 0.9.0 to leverage firewalld policies as initially introduced:

Policies are applied to traffic flowing between zones in a stateful unidirectional manner. This allows different policies depending on the direction of traffic.

+----------+     policyA     +----------+
|          |  <------------  |          |
| libvirt  |                 |  public  |
|          |  ------------>  |          |
+----------+     policyB     +----------+

The zone is used between a remote side and the host, policies are used for routing between zones. They will reuse interfaces attached to zones.

Create a new zone:

firewall-cmd --permanent --new-zone=local-ignore

Have this zone accept anything by default:

firewall-cmd --permanent --zone=local-ignore --set-target=ACCEPT

Add the target interface to this zone:

firewall-cmd --permanent --zone=local-ignore --add-interface=virbr0

This will take care of traffic between VMs and host (once rules are reloaded).

Add firewalld policies that will use this zone, once for ingress and once egress, with ANY as other side and also set them to a default behavior of ACCEPT:

firewall-cmd --permanent --new-policy=local-ignore-from
firewall-cmd --permanent --policy=local-ignore-from --add-ingress-zone=local-ignore
firewall-cmd --permanent --policy=local-ignore-from --add-egress-zone=ANY
firewall-cmd --permanent --policy=local-ignore-from --set-target=ACCEPT

firewall-cmd --permanent --new-policy=local-ignore-to
firewall-cmd --permanent --policy=local-ignore-to --add-egress-zone=local-ignore
firewall-cmd --permanent --policy=local-ignore-to --add-ingress-zone=ANY
firewall-cmd --permanent --policy=local-ignore-to --set-target=ACCEPT

Finally reload the rules:

firewall-cmd --reload

This takes care of having traffic related to virbr0 unhindered by firewalld.

Now the (huge) nftables table inet firewalld should co-exist peacefully with OP's table ip QEMU. Whether firewalld is running or not (ie: its rules are removed) should not change the behavior for anything related to virbr0. All restrictions have to be handled in table ip QEMU.

Currently table ip QEMU doesn't restrict anything at all (there is no drop rule nor drop policy) so should be fixed properly. One should not use a drop policy or this will cause again the problem that has just been fixed, in the other direction. Just drop any unwanted traffic related to virbr0 (especially towards virbr0) in it.

Additional interfaces can be added to the ignore list, for example to add lxcbr0 (meaning: to leave LXC unhindered by firewalld):

firewall-cmd --permanent --zone=local-ignore --add-interface=lxcbr0
firewall-cmd --reload
Related Topic