Cisco – Does vPC peer-gateway cause unicast flooding

ciscocisco-nexusswitchvpc

This is my network topology where I have SW-1 is configured for HSRP for multiple VLANs, SW-2 & SW-3 connected to SW-1 using vPC. Last few month I am dealing with this unicast flooding where whenever I add a new switch in network or spanning-tree changed I noticed for 5 second or so all my switch ports seeing entire network traffic like traffic get mirror on every single switch in datacenter.

After collecting lots of data I found this flooding only happens in one side of network which is SW-3 so whatever TOR switch connected to SW-3 seeing unicast flooding, after comparing configuration I found only one parameter is missing in SW-3 which is peer-gateway. I have no idea that is the issue but i want to see what you guys suggesting, I thought peer-gateway is only for HSRP switch not for L2 switch.

enter image description here

Here is the config for vPC

SW-1 (ROOT bridge)

vpc domain 1
  peer-switch
  role priority 10
  peer-keepalive destination 172.29.0.7 source 172.29.0.6
  peer-gateway
  auto-recovery
  ip arp synchronize

SW-2

vpc domain 2
  peer-switch
  role priority 10
  peer-keepalive destination 172.29.0.103 source 172.29.0.102
  peer-gateway
  auto-recovery
  ip arp synchronize

SW-3 ( missing peer-gateway here)

vpc domain 3
  peer-switch
  role priority 10
  peer-keepalive destination 172.29.0.161 source 172.29.0.162
  auto-recovery
  ip arp synchronize

Should I need to configure peer-gateway on SW-3 or its totally not related and I should keep digging somewhere else?

I have earlier posted question here which I may delete it if this question get resolved: Strange flooding during installing new switch in network

Best Answer

I don't think that peer-gateway is needed on SW-3 at all, as it serves somewhat of a corner case (see below), only useful if the the given nexus has SVIs (interface vlan XXX) and acts as default gateway for the said VLANs and there are special devices in that VLAN. As I understand your diagram and text, this not the case for SW-3.

While I suggest turning peer-gateway off, I would deepen the research into the spanning-tree configuration of all switches involved.

  • Are spanning-tree port type [network|normal|edge|edge trunk] deployed correctly and pervasively? (network for VPC peer links, normal for inter switch links, edge/edge trunk (with BPDUguard) for end systems/servers)
  • Are the 4x Port-Channels between SW1 and SW2/3 and also the port-channels downward to their ToR switches operating correctly, with no members in individual state?

Taken from the Cisco Nexus 9000 Series NX-OS Interfaces Configuration Guide, Release 7.x

Some network-attached storage (NAS) devices or load balancers might have features that help to optimize the performances of particular applications. These features enable the device to avoid a routing-table lookup when responding to a request that originated from a host that is not locally attached to the same subnet. Such devices might reply to traffic using the MAC address of the sender Cisco Nexus 9000 Series device rather than the common HSRP gateway. This behavior is noncomplaint with some basic Ethernet RFC standards. Packets that reach a vPC device for the nonlocal router MAC address are sent across the peer link and could be dropped by the built in vPC loop avoidance mechanism if the final destination is behind another vPC.

The vPC peer-gateway capability allows a vPC switch to act as the active gateway for packets that are addressed to the router MAC address of the vPC peer. This feature enables local forwarding of packets without the need to cross the vPC peer link. In this scenario, the feature optimizes use of the peer link and avoids potential traffic loss.

Related Topic