Networking – Why Do ICMP Redirect Host Happen?

debianicmpiptablesnetworkingrouting

I'm setting up a Debian box as a router for 4 subnets. For that I have defined 4 virtual interfaces on the NIC where the LAN is connected (eth1).

eth1      Link encap:Ethernet  HWaddr 94:0c:6d:82:0d:98  
          inet addr:10.1.1.1  Bcast:10.1.1.255  Mask:255.255.255.0
          inet6 addr: fe80::960c:6dff:fe82:d98/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6026521 errors:0 dropped:0 overruns:0 frame:0
          TX packets:35331299 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:673201397 (642.0 MiB)  TX bytes:177276932 (169.0 MiB)
          Interrupt:19 Base address:0x6000 

eth1:0    Link encap:Ethernet  HWaddr 94:0c:6d:82:0d:98  
          inet addr:10.1.2.1  Bcast:10.1.2.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:19 Base address:0x6000 

eth1:1    Link encap:Ethernet  HWaddr 94:0c:6d:82:0d:98  
          inet addr:10.1.3.1  Bcast:10.1.3.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:19 Base address:0x6000 

eth1:2    Link encap:Ethernet  HWaddr 94:0c:6d:82:0d:98  
          inet addr:10.1.4.1  Bcast:10.1.4.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          Interrupt:19 Base address:0x6000 

eth2      Link encap:Ethernet  HWaddr 6c:f0:49:a4:47:38  
          inet addr:192.168.1.10  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::6ef0:49ff:fea4:4738/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:199809345 errors:0 dropped:0 overruns:0 frame:0
          TX packets:158362936 errors:0 dropped:0 overruns:0 carrier:1
          collisions:0 txqueuelen:1000 
          RX bytes:3656983762 (3.4 GiB)  TX bytes:1715848473 (1.5 GiB)
          Interrupt:27 

eth3      Link encap:Ethernet  HWaddr 94:0c:6d:82:c8:72  
          inet addr:192.168.2.5  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: fe80::960c:6dff:fe82:c872/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:110814 errors:0 dropped:0 overruns:0 frame:0
          TX packets:73386 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:16044901 (15.3 MiB)  TX bytes:42125647 (40.1 MiB)
          Interrupt:20 Base address:0x2000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:22351 errors:0 dropped:0 overruns:0 frame:0
          TX packets:22351 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:2625143 (2.5 MiB)  TX bytes:2625143 (2.5 MiB)

tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:10.8.0.1  P-t-P:10.8.0.2  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:41358924 errors:0 dropped:0 overruns:0 frame:0
          TX packets:23116350 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          RX bytes:3065505744 (2.8 GiB)  TX bytes:1324358330 (1.2 GiB)

I have two other computers connected to this network. One has IP 10.1.1.12 (subnet mask 255.255.255.0) and the other one 10.1.2.20 (subnet mask 255.255.255.0). I want to be able to reach 10.1.1.12 from 10.1.2.20.

Since packet forwarding is enabled in the router and the policy of the FORWARD chain is ACCEPT (and there are no other rules), I understand that there should be no problem to ping from 10.1.2.20 to 10.1.1.12 going through the router.

However, this is what I get:

$ ping -c15 10.1.1.12
PING 10.1.1.12 (10.1.1.12): 56 data bytes
Request timeout for icmp_seq 0
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 81d4   0 0000  3f  01 e2b3 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 1
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 899b   0 0000  3f  01 daec 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 2
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 78fe   0 0000  3f  01 eb89 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 3
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 14b8   0 0000  3f  01 4fd0 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 4
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 8ef7   0 0000  3f  01 d590 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 5
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 ec9d   0 0000  3f  01 77ea 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 6
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 70e6   0 0000  3f  01 f3a1 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 7
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 b0d2   0 0000  3f  01 b3b5 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 8
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 f8b4   0 0000  3f  01 6bd3 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 9
Request timeout for icmp_seq 10
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 1c95   0 0000  3f  01 47f3 10.1.2.20  10.1.1.12 

Request timeout for icmp_seq 11
Request timeout for icmp_seq 12
Request timeout for icmp_seq 13
92 bytes from router2.mydomain.com (10.1.2.1): Redirect Host(New addr: 10.1.1.12)
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 62bc   0 0000  3f  01 01cc 10.1.2.20  10.1.1.12

Why does this happen?

From what I've read the Redirect Host response has something to do with the fact that the two hosts are in the same network and there being a shorter route (or so I understood). They are in fact in the same physical network, but why would there be a better route if they are not on the same subnet (they can't see each other)?

What am I missing?

Some extra info you might want to see:

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.8.0.2        0.0.0.0         255.255.255.255 UH    0      0        0 tun0
127.0.0.1       0.0.0.0         255.255.255.255 UH    0      0        0 lo
192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0 eth3
10.8.0.0        10.8.0.2        255.255.255.0   UG    0      0        0 tun0
192.168.1.0     0.0.0.0         255.255.255.0   U     1      0        0 eth2
10.1.4.0        0.0.0.0         255.255.255.0   U     0      0        0 eth1
10.1.1.0        0.0.0.0         255.255.255.0   U     0      0        0 eth1
10.1.2.0        0.0.0.0         255.255.255.0   U     0      0        0 eth1
10.1.3.0        0.0.0.0         255.255.255.0   U     0      0        0 eth1
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth2
0.0.0.0         192.168.2.1     0.0.0.0         UG    100    0        0 eth3

# iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination   

# iptables -L -n -t nat
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MASQUERADE  all  -- !10.0.0.0/8           10.0.0.0/8          
MASQUERADE  all  --  10.0.0.0/8          !10.0.0.0/8          

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Best Answer

At first blush, it looks like Debian is stretching the boundaries for sending an ICMP redirect; quoting RFC 792 (Internet Protocol).

  The gateway sends a redirect message to a host in the following

  situation.  A gateway, G1, receives an internet datagram from a
  host on a network to which the gateway is attached.  The gateway,
  G1, checks its routing table and obtains the address of the next
  gateway, G2, on the route to the datagram's internet destination
  network, X.  If G2 and the host identified by the internet source
  address of the datagram are on the same network, a redirect
  message is sent to the host.  The redirect message advises the
  host to send its traffic for network X directly to gateway G2 as
  this is a shorter path to the destination.  The gateway forwards
  the original datagram's data to its internet destination.

In this case, G1 is 10.1.2.1 (eth1:0 above), X is 10.1.1.0/24 and G2 is 10.1.1.12, and the source is 10.1.2.20 (i.e. G2 and the host identified by the internet source address of the datagram are **NOT** on the same network). Maybe this has been historically interpreted differently in the case of interface aliases (or secondary addresses) on the same interface, but strictly speaking I'm not sure we should see Debian send that redirect.

Depending on your requirements, you might be able to solve this by making the subnet for eth1 something like 10.1.0.0/22 (host addresses from 10.1.0.1 - 10.1.3.254) instead of using interface aliases for individual /24 blocks (eth1, eth1:0, eth1:1, eth1:2); if you did this, you'll need to change the netmask of all hosts attached and you wouldn't be able to use 10.1.4.x unless you expanded to a /21.

EDIT

We're venturing a bit outside the scope of the original question, but I'll help work through the design/security issues mentioned in your comment.

If you want to isolate users in your office from each other, let's step back for a second and look at some security issues with what you have now:

You currently have four subnets in one ethernet broadcast domain. All users in one broadcast domain doesn't meet the security requirements you articulated in the comments (all machines will see broadcasts from other machines and could spontaneously send traffic to each other at Layer2, regardless of their default gateway being eth1, eth1:0, eth1:1 or eth1:2). There is nothing your Debian firewall can do to change this (or maybe I should say there is nothing your Debian firewall should do to change this :-).

  • You need to assign users into Vlans, based on security policy stated in the comments. A properly-configured Vlan will go a long way to fixing the issues mentioned above. If your ethernet switch doesn't support Vlans, you should get one that does.
  • With respect to multiple security domains accessing 10.1.1.12, you have a couple of options:
    • Option 1: Given the requirement for all users to access services on 10.1.1.12, you could put all users in one IP subnet and implement security policies with Private Vlans (RFC 5517), assuming your ethernet switch supports this. This option will not require iptables rules to limit intra-office traffic from crossing security boundaries (that is accomplished with private Vlans).
    • Option 2: You could put users into different subnets (corresponding to Vlans) and implement iptables rules to deploy your security policies
  • After you have secured your network at the Vlan level, set up source-based routing policies to send different users out your multiple uplinks.

FYI, if you have a router that supports VRFs, some of this gets even easier; IIRC, you have a Cisco IOS machine onsite. Depending on the model and software image you already have, that Cisco could do a fantastic job isolating your users from each other and implement source-based routing policies.

Related Topic