Nat – ASA 5510 NAT stops working

cisco-asanat;troubleshooting

I’ve been searching on this issue and can’t seem to find info about it.

I’ve inherited an older ASA5510 (8.2(1)) and unfortunately I’m pretty new to them. I have a set of overloaded NAT statements that will stop working seemingly at random. 

access-list 101 extended permit tcp x.x.x.88 255.255.255.248 host x.x.x.121 eq 3389
access-list 101 extended permit tcp x.x.x.88 255.255.255.248 host x.x.x.121 eq www
access-list 101 extended permit tcp x.x.x.88 255.255.255.248 host x.x.x.121 eq https
access-list 101 extended permit tcp any host x.x.x.121 eq 222
static (inside,outside) tcp x.x.x.121 www 10.15.60.43 www netmask 255.255.255.255
static (inside,outside) tcp x.x.x.121 https 10.15.60.43 https netmask 255.255.255.255
static (inside,outside) tcp x.x.x.121 3389 10.15.60.2 3389 netmask 255.255.255.255
static (inside,outside) tcp x.x.x.121 222 10.15.60.42 ssh netmask 255.255.255.255

If I clear xlate it doesn’t come back. If I show xlate for x.x.x.121 it shows them as still in there. However, if I remove the portions of the config listed above and add just static (inside,outside) x.x.x.121 10.15.60.2 then clear xlate then I’m able to connect to the server again.

If I then put the original config back in everything works fine for a day or two. Any thoughts or diagnostic commands that I should be running would be greatly appreciated.

Thanks!

Edit:

Per Ricky Beam's suggestion I checked the number of users in sh ver. Maximum Physical Interfaces set to Unlimited. Inside Hosts also set to Unlimited.

show local-host | inc host limit returns no results. I'm guessing since hosts are Unlimited.

Edit:

I came in this morning and it was not allowing connections to that IP again. I did a show conn count while it wasn't allowing connections and it shows 298 in use, 782 most used. I did a clear xlate then a clear conn all and it didn't start allowing connections. However, removing the static (inside,outside) statements and adding static (inside,outside) x.x.x.121 10.15.60.2 brought the connection up right away.

We did add a new building about 1 week before this started happening but has far as I can tell the asa is not actually running out of connections.

It does show Built dynamic UDP translation and Built outbound UDP connection in the log while it was not working though.

Edit:

Was down again. Verified that the other devices that are overloaded on that IP are down as well. I can ping 10.15.60.2 using the ping 10.15.60.2 command but I can't while sourced from the outside.

asa# ping  
Interface: outside  
Target IP address: 10.15.60.2  
Extended commands [n]:  
Sweep range of sizes [n]:  
Type escape sequence to abort.  
Sending 5, 100-byte ICMP Echos to 10.15.60.2, timeout is 2 seconds:  
?????

Ran the following show commands while it wasn't working. 

asa# sh conn | include x.x.x.121  
asa# sh xlate | include x.x.x.121  
PAT Global x.x.x.121(80) Local 10.15.60.43(80)  
PAT Global x.x.x.121(443) Local 10.15.60.43(443)  
PAT Global x.x.x.121(3389) Local 10.15.60.2(3389)  
PAT Global x.x.x.121(222) Local 10.15.60.42(22)

The ASA is now logging to a syslog server at informational level. Hopefully that will show what is going on.

Best Answer

Looking through the "fixed" bugs (8.2.5 Interim Release Notes):

CSCti62667 Connections stay open w/ 'sysopt connection timewait' & NetFlow
CSCtl06156 NAT Xlate idle timer doesn't reset with Conn.
CSCtr94429 ASA: Local-host and all conns are torn down when client hits conn limit

You've confirmed the system is not "Restricted(R)" -- unlimited interfaces and users. That's the most common reason for the failure mode you've described. (It stops allowing new connections.) Threat-detection could also result in new connections being dropped. Neither of these should cause traffic to stop for existing xlates.

clear xlate needs a follow up clear conn all to completely clear everything. If it's not clearing them, then there's something else wrong. (or they're being recreated instantly.)

Cisco documents the 5510 as supporting up to 50k (130k for a secplus license) concurrent connections. I'd be very surprised if you've hit that wall. (show conn count and look at the "most used". %ASA-5-321001 would be logged if you did.)

Beyond that, if it's not logging any reason(s) for dropping/refusing connections, I'm at a loss where else to look.

Related Topic