Nat – Linux Virtual Server: NAT return path

I'm trying to set up a LVS load balancer in front of my Kubernetes: Calico is used as container network, so every pod has its own network interface in it's host. External IPs, routes and routing rules are manager using keepalived.
Every subnet assigned by my hosting provider has its own router.
The servers have two physical interfaces connected to the Internet. The external IPs are available on the second interface.

The configuration is this:
The first interface has a global IP address and a default Route.
The second interface has a private IP. It also has all additional IPs by my hosting provider. Because all packets from the additional IPs must be routed through a special default gateway I've added routing tables and rules.

Up to this point everything is working as expected.
To make Kubernetes services available in the IPs I've added a LVS on some nodes.

My problem is this: Packets from the internet arrive at my Kubernetes pods but their answers don't get back to the internet.
The packets disappear somewhere in my Kubernetes node.
The answer packets from my pods are using the pod IP as source and have to be rewritten by NAT.

When looking at the iptables rules I don't see the ones for LVS. Where are they? When are LVS NAT rules executed? Are they bound to an interface? Is the forwarding path and interface decided before NAT for the answer packets are done?