What does it mean when I get small variations in ping results when pinging an empty LAN ?
For example :
- When pinging a resource in the same vlan (layer 2) : no variations, ping < 1ms constantly
- When pinging another vlan ip address (layer 3, routed by the L3 switch) : the ping varies between 1 ms and <1ms
- When pinging another vlan ip address (layer 3, through a firewall) : the ping varies between 1ms and "<1ms"
- When pinging a server in another vlan (layer 3, routed by the L3 switch), the ping varies between <1ms and 3ms
The firewall has no traffic filters.
My questions is : is this behavior normal? I am not worried about the response time in itself, but more by the variation that I cannot really explain.
The switches are not busy, so is the firewall because my network is empty at this time, so it should not be a load issue.
I'm looking forward hearing thoughts on this issue.
Best Answer
I'd say it looks normal. But I think you're doing this a bit wrong.
Ping has one function only: probing, i.e. "Hello, are you there" - "Hello, yes I am here". While it shows you the latency between two hosts, it is not a network performance tool.
Your "empty" network is never silent. When you connect two network devices together - any devices - they will start exchanging all kinds of information immediately; availability, capabilities, link status etc. This is (like ICMP) network management traffic, and handled by the CPU.
The primary function of any network device is traffic forwarding. When the first packet of a discussion comes, a Layer 2 switch will first check the MAC/ARP tables to see if the port leading to the destination is known. If not, it will ARP the network to learn the destination port. Once it is known, the traffic forwarding is delegated to the forwarding engine. CPU will not touch packets in that discussion, but it will still keep communicating with the other network devices. L3 forwarding (i.e routing) is often done entirely by the CPU. This depends on the device, some routing switches are capable of hardrware packet forwarding similar to L2 switching. This is known as "route once / switch many" or "Multilayer switching".
The network devices are configured to maintain the primary functionality of the network. All mgmt traffic it's given secondary priority, and ICMP ping has the lowest of low priorities. This is - as far as I know - true for all devices. Allowing ICMP a higher priority would open door for DOS attacks such as Ping of Death and Ping Flood.
Hence tests 2 and 3 are irrelevant. Devices are responding, which shows traffic is flowing correctly. The devices will not sacrifice CPU cycles for ping, they will respond only when (or if!) they have time. Once the network gets more traffic, you can expect higher variation on network device ping responses. It's still expected behavior, their latency has no relation whatsoever to network performance. Host-to-host response times should be less affected.
Tests 1 and 4 are more sensible; but they aren't a valid comparison between L2 and L3 forwarding performance. In test 1 you're pinging a "resource", in test 4 you're pinging a server - i.e. not the same device. While the network may be playing part, the latency depends on the recipient. What's above applies also to any computer; a NIC is a network device and ICMP is handled by CPU... maybe your server is taking it's own sweet time responding. When you're pinging to a different subnet you're bringing in yet another potential source of latency - the default gateways.
If you want to test the forwarding capability of your network, you need to do it differently.
The Ping Fallacy