Linux – IPTables Multi-VLAN NAT to Multiple IPs

I have a slight Problem with my NAT-Setup.

What I'm trying to do:

I have 2 VLANs (in this example 100 and 200) and want that they can access the Network via separate IPs. This should be accomplished by a iptables NAT. I created this setup with pf (FreeBSD) but unfortunately the hardware I have to use is not compatible. I would love to that within our main router, but the CPU cannot handle such loads.
With IP-Tables etc. I'm just a noob 😉
Schema:

User on VLAN 100 (10.100.0.0/24) -> 10.100.0.1 NAT xxx.yyy.zzz.6 -> Network

User on VLAN 200 (10.200.0.0/24) -> 10.200.0.1 NAT xxx.yyy.zzz.7 -> Network

All of this on the same physical Interface, with VLAN Interfaces etc.

This is my current config:
/etc/network/interfaces

    lo ....

    iface ens2f3 inet static
        address xxx.yyy.zzz.6
        netmask 255.255.252.0

auto ens2f3.100
iface ens2f3.100 inet static
        address 10.100.0.1
        netmask 255.255.255.0
        vlan-raw-device ens2f3

iface ens2f3 inet static
        address xxx.yyy.zzz.7
        netmask 255.255.252.0

auto ens2f3.200
iface ens2f3.200 inet static
        address 10.200.0.1
        netmask 255.255.255.0
        vlan-raw-device ens2f3

/etc/iptables/rules.v4 (automatically loaded)

#NAT TABLE
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING -s 10.100.0.0/24 -o ens2f3 -j SNAT --to-source xxx.yyy.zzz.6
-A POSTROUTING -s 10.200.0.0/24 -o ens2f3 -j SNAT --to-source xxx.yyy.zzz.7

COMMIT

# General Table
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
COMMIT

# I know this setup is very insecure! I lockdown the server when I got NAT working

The Problem is that I can get with a 10.100.0.100 I can connect onto xxx.yyy.zzy.1 with exactly this IP (10.100.0.100; which shouldn't be possible) instead of the IP I should have obtained via NAT.

I've also tried marking the packages when they arrive at the interface and
have seperate routing tables for them. There the problem above doesn't exist, but no connection to anywhere is possible

Could you please provide me tips on how to realize this with a Linux-based OS. If necessary I have another 10G Interface available