How to properly interpet the results of tracert, in particular the second to last hop before a timeout

icmpnetworkingroutingtcpiptrace

I am troubleshooting a wireless internet connection that is continuously dropping. The isp says the radio signal is good so it must by my cisco asa 5505. I don't believe them…

For discussion assume the following:

Remote site public ip is 10.1.1.50 and its default gateway is 10.1.1.1

When i am doing a tracert from a remote location to 10.1.1.50, should the second to last hop of the tracert always be 10.1.1.1?

When the connection is down, the second to last hop isn't 10.1.1.1 however 10.1.1.1 is pingable and I can do a tracert to it.

I am thinking this is a routing issue by the isp.
Is my logic valid? If 10.1.1.1 is reachable shouldn't it be the second to last hop on a tracert before I get timeouts?

Best Answer

In 99% of cases, the penultimate hop of a traceroute will not be the default gateway of the destination host. This is because of the way that traceroute works.

All IP packets have a -somewhat misnamed- time-to-live (TTL) field. This field is decremented by one by every router that forwards a packet. If a router decrements the TTL to 0, it drops the packet and generates an ICMP TTL exceeded error packet and sends it back to the person who sent the original packet. The error packet will have a destination IP of the source of the original packet (in this case, the host that initiated the traceroute). The source IP of the error packet will generally be the IP address of the outgoing interface, i.e. the one facing towards the rest of the network.

Traceroute takes advantage of this fact; it sends packets with sequentially increasing TTLs; this causes each router in the path between source and destination to send ICMP unreachable messages back to the host performing the traceroute. Each error will have a source IP of the final router the probe packet reached before being dropped. This allows traceroute to build a picture of the path between source and destination.

Consider the diagram below (ignoring the possibility of multiple paths):

________     1.1____1.2                2.1____2.2     ________
|Host A|-----|Router 1|--- Internet ---|Router 2|-----|Host B|
--------     ----------                ----------     --------

If host A is performing a traceroute to host B, then the penultimate hop will be router 2, which will receive a probe packet as follows:

SrcIP: A | DstIP: B | TTL: 1

Router 2 will decrement the TTL to 0; which will cause it to generate an ICMP TTL expired:

SrcIP: 2.1 | DstIP: A | TTL <default>

Hence when traceroute receives this error message, the IP address it will see for the penultimate hop will be the internet facing interface of router 2, rather than the one facing host B.