Static IP Route Not Clearing – Troubleshooting Tips

brocaderouting

I have a Brocade FESX648-PREM I set a static route with the following command:

ip route 123.123.123.123 255.255.255.255 124.124.124.124

Where 123.123.123.123 is my real IP address and 124.124.124.124 is an intentionally incorrect next hop IP.

Since, 124.124.124.124 is an invalid next hop IP, 123.123.123.123 stops pinging immediately.

However, when I clear the route with no ip route 123.123.123.123 255.255.255.255 124.124.124.124 the 123.123.123.123 still does not ping.

If I follow up with clear ip route 123.123.123.123 255.255.255.255 the IP again will continue to not ping.

If I run a show ip route 123.123.123.123, I can see that it is correctly routing to the default gateway.

However, if I write memory and reload the switch, then the IP will ping again.

It was suggested to me that I should verify that my ip-route and ip-cache entries are not exceeding their system-max values, but that checked out OK as well:

Total number of IP routes: 64382, avail: 15618 (out of max 80000)

Total number of cache entries: 303

Any ideas why this route is getting "stuck"?

EDIT: I found a workaround. 1. Remove the invalid static route, 2. manually set the correct static route, 3. clear the route set in the previous step. Strange, seems like this should not be necessary. If anyone has any thoughts or suggestions on resolving the core issue, I'd still be very happy to hear them.

Best Answer

Is there another existing routing protocol, default route, or static-route that would begin to take precedence once the less explicit route entry is removed?

Does placing a static-route that is correct (valid next-hop) allow pings to flow again?

Related Solutions

Routing – What happens when the ARP cache overflows

Edit 2:

As you mentioned...

ip route 10.1.0.0 255.255.0.0 iface0

Forces the Brocade to proxy-arp for every destination in 10.1.0.0/16 as if it was directly connected to iface0.

I can't respond about Brocade's ARP cache implementation, but I would simply point out the easy solution to your problem... configure your route differently:

ip route 10.1.0.0 255.255.0.0 CiscoNextHopIP

By doing this, you prevent the Brocade from ARP-ing for all of 10.1.0.0/16 (note, you might need to renumber the link between R1 and R2 to be outside 10.1.0.0/16, depending on Brocade's implementation of things).

Original answer:

I expect that in most, or even all, implementations, there is a hard limit on the capacity of the ARP table.

Cisco IOS CPU routers are only limited by the amount of DRAM in the router, but that is typically not going to be a limiting factor. Some switches (like Catalyst 6500) have a hard limitation on the adjacency table (which is correlated to the ARP table); Sup2T has 1 Million adjacencies.

So, what happens when the ARP cache is full and a packet is offered with a destination (or next-hop) that isn't cached?

Cisco IOS CPU routers don't run out of space in the ARP table, because those ARPs are stored in DRAM. Let's assume you're talking about Sup2T. Think of it like this, suppose you had a Cat6500 + Sup2T and you configured all Vlans possible, technically that is

4094 total Vlans - Vlan1002 - Vlan1003 - Vlan1004 - Vlan1005 = 4090 Vlans

Assume you make each Vlan a /24 (so that's 252 possible ARPs), and you pack every Vlan full... that is 1 Million ARP entries.

4094 * 252 = 1,030,680 ARP Entries

Every one of those ARPs would consume a certain amount of memory in the ARP table itself, plus the IOS adjacency table. I dont know what it is, but let's say the total ARP overhead is 10 Bytes...

That means you have now consumed 10MB for ARP overhead; it still isn't very much space... if you were that low on memory, you would see something like %SYS-2-MALLOCFAIL.

With that many ARPs and a four hour ARP timeout, you would have to service almost 70 ARPs per second on average; it's more likely that the maintenance on 1 million ARP entries would drain the CPU of the router (potentially CPUHOG messages).

At this point, you could start bouncing routing protocol adjacencies and have IPs that are just unreachable because the router CPU was too busy to ARP for the IP.

Cisco – Traffic from route-map to crypto-map

I have to give credit to @ron for this answer.

The policy map was never going to work the way it was previously. @ron suggested a gre tunnel, then protect that with ipsec.

interface Tunnel0
ip address 10.10.10.2 255.255.255.252
ip mtu 1420
tunnel source 1.1.1.1
tunnel destination 2.2.2.2
crypto map IOFVPN

and a route to point to the internal subnet on the remote side with a gateway of the remote side.

S    192.168.10.0/24 [1/0] via 10.10.10.1

I've never used gre before but I will now. Getting the tunnel up was pretty basic both on the cisco and linux ( openswan ) side. Once that was up and running I tested without ipsec.

On my fa0/0 interface, my internal I put:

ip policy route-map PROXY-REDIRECT

route-map proxy-redirect permit 100
 match ip address PROXY-REDIRECT
 set ip next-hop recursive 192.168.10.1

The matching ACL is:

ip access-list extended proxy-redirect
 deny   tcp host 192.168.30.13 any eq www
 permit tcp 192.168.30.0 0.0.0.255 any eq www
 permit tcp 192.168.30.0 0.0.0.255 any eq 443
 permit tcp 192.168.30.0 0.0.0.255 any eq irc
 permit tcp 192.168.30.0 0.0.0.255 any eq 6667
 deny   tcp 192.168.30.0 0.0.0.255 any eq 5938
 deny   tcp any any
 deny   udp any any
 deny   ip any any

As soon as I added that my traffic started passing the way I wanted it. I'll note here that this ACL could be shrunk. I added the implicit deny's because I was having the odd traffic pass over the link.

Once was verified working then I just needed to wrap it in ipsec. Configuring the Cisco side was easy.

crypto isakmp policy 1
 encr aes 192
 authentication pre-share
 group 2
 lifetime 43200
crypto isakmp key *********** address 2.2.2.2
!
!
crypto ipsec transform-set IOFSET2 esp-aes 192 esp-sha-hmac 
 mode transport
!
crypto map IOFVPN 1 ipsec-isakmp 
 description IOM
 set peer 2.2.2.2
 set transform-set IOFSET2 
 match address IPSEC-GRE-IOF

ip access-list extended IPSEC-GRE-IOF
 permit gre host 1.1.1.1 host 2.2.2.2

** Must use transport mode. It took me a bit to figure that one out.

Apply that crypto map to both f0/1 and tun0 and you have a tunnel.

The openswan side is what gave me trouble though this whole thing. Their config is a bit odd and it just took some getting used to.

At the end of the day all traffic that I want is peeled off and routed through an ipsec protected gre tunnel to a remote endpoint.

Enjoy.

Best Answer

Related Solutions

Routing – What happens when the ARP cache overflows

Cisco – Traffic from route-map to crypto-map

Related Topic