We've got AT&T U-Verse internet service, which has an extremely boneheaded DSL gateway.
We have 5 IPs (netmask 248), but the gateway is incapable of doing anything other than a single IP -> single MAC address mapping.
We have a single firewall machine, and we redirect different IP/port combos to different places inside a DMZ.
Our solution so far is to have a VMWare virtual machine on the firewall with 4 additional NICs in it, to get the other 4 IP addresses… however we have a problem.
The gateway is basically doing an ARP ping to see if the IP is responding on the expected MAC. With 4 NICs all on the same LAN, linux is responding to ARP requests for ALL IPs using a single interface. That's not what the gateway is expecting, and it's messing up the 3 other NICs. The gateway refuses to route incoming traffic for IPs where the ARP ping results aren't the expected MAC.
How can we get the ARP replies for eth0's IP to go out eth0, eth1's IP to go out eth1, etc?
EDIT
Christopher Cashell's response does not work in this situation. I had great hopes reading it, but… nope.
EDIT 2
Solved! See my answer below.
Best Answer
Your chosen solution works, but there are alternatives that don't involve arptables. (Christopher Cashell was on the right track, originally, but he was off by a smidge.)
In short, you want to set these parameters:
These should be available when running a modern, 2.6 series Linux kernel. Check and make sure you that '/proc/sys/net/ipv4/conf//arp_announce' and /proc/sys/net/ipv4/conf//arp_ignore' exist on your system.
The 'arp_filter' parameter only works when your various IP addresses share a LAN segment but use different IP subnet. If they share the IP subnet, too, you need to use 'arp_ignore' and 'arp_announce', as above.
(I believe you may also need to set 'arp_filter' back to '0', too.)