Cisco – In a multi-tenant environment what should be done to make your switchports silent on Cisco and Juniper switches

There might be some helpful information for you at the Juniper Knowledge Center.

If RPD is consuming high CPU, then perform the following checks and verify the following parameters:

• Check the interfaces: Check if any interfaces are flapping on the router. This can be verified by looking at the output of the show log messages and show interfaces ge-x/y/z extensive commands. Troubleshoot why they are flapping; if possible you can consider enabling the hold-time for link up and link down.

• Check if there are syslog error messages related to interfaces or any FPC/PIC, by looking at the output of show log messages.

• Check the routes: Verify the total number of routes that are learned by the router by looking at the output of show route summary. Check if it has reached the maximum limit.

• Check the RPD tasks: Identify what is keeping the process busy. This can be checked by first enabling set task accounting on. Important: This itself might increase the load on CPU and its utilization; so do not forget to turn it off when you are done with the required output collection. Then run show task accounting and look for the thread with the high CPU time:

  user@router> show task accounting
  Task                       Started    User Time  System Time  Longest Run
  Scheduler                   146051        1.085        0.090        0.000
  Memory                           1        0.000            0        0.000  <omit>
  BGP.128.0.0.4+179              268       13.975        0.087        0.328
  BGP.0.0.0.0+179      18375163 1w5d 23:16:57.823    48:52.877        0.142
  BGP RT Background              134        8.826        0.023        0.099
  

Find out why a thread, which is related to a particular prefix or a protocol, is taking high CPU.

• You can also verify if routes are oscillating (or route churns) by looking at the output of the shell command: %rtsockmon –t

• Check RPD Memory. Some times High memory utilization might indirectly lead to high CPU.

Agree with both Ron and Mike Pennington here. The entire point of Spanning Tree is to prevent broadcast storms from blowing up your network and you have just recently observed a practical lesson in how that happens.

Knowing the version of your Netgear switch would be helpful. I'm looking at an older version of the ProSafe switch user manual for the GS748TS and the Spanning-Tree settings are quite granular, so you should be able to configure the switch to prevent such things from happening in the future.

On the older NetGear ProSafe switches I have used the default configuration is that STP is enabled in RSTP (rapid spanning tree) with BPDU flooded to all ports on the switch. This is a good configuration for everyday use. In order to help you more, we would need to know how many VLANs are on your network and so forth.

But at the end of the day this is a hard lesson in the need for company policies that govern what can and cannot be connected to a production network. At my employer (a large Fortune 500 manufacturing enterprise) our network policy is that all access ports on switches are configured with BPDUguard enabled such that when any unknown switching device is plugged into it the port becomes error disabled. If you have SNMP monitoring on your switches, then you will get an alert.

There is a small price to be paid in user frustration but this is more than made up for in the fact that you will automatically have visibility and hopefully control over what gets connected and you won't have to deal with the network outages in the future.