We have an L2 path between a tape drive and a NetApp. The path goes through 3 switches (a 6509 and two N5K) but the two devices are on the same VLAN. All 10 gig fiber.
I found out that while most of the ports in the path are set to an MTU of 9216, one of them (the port on the 6509 core going to the first N5K) is set to 1500. The interface on the other side is set to 9216.
We are getting a max of 750 Mpbs between these two systems and I'm thinking this might be the cause.
I'm guessing that the systems are using PMTUD in this situation and are using 1500 byte frames when talking to each other but this is causing less optimal data transfer.
I've been told by Cisco that to change this, I need to enable system jumbomtu 9216 on the 6509 (it's not set right now) and then set the interface MTU to 9216 as well.
Since this is our core switch I'm hesitant to do this without knowing how the system jumbomtu will affect things. Since it has a redundant non-VSS sister, I should make the change on both of them.
Would there be any ill effects of making this change?
I'm reading things in Cisco's docs that I don't fully understand. For example, the setting jumbo MTU config section says:
"For Layer 2 Ethernet ports, you can configure only the global egress LAN port MTU size" <- (which is set using system jumbomtu)
There are several L2 (non routed, non SVI) ports configured for both 1500 and 9216 yet the system jumbomtu command is not set on this switch. Does that mean that the 9216 setting on these L2 ports is getting ignored?
If I set system jumbomtu 9216, will this effectively change all the L2 ports to an egress MTU of 9216?
The Catalyst switches are running IOS 12.2. No CatOS.
Best Answer
I think you may be confusing the VLAN database MTU and the interface MTU. Pay special attention to the note at the end. Enabling jumbo frames in the VLAN database for a VLAN will not suddenly cause all the switch interfaces in that VLAN to be enabled for jumbo frames.
Jumbo/Giant Frame Support on Catalyst Switches Configuration Example
In any case, a switch will not send out jumbo frames to an interface, even on an interface with jumbo frames enabled, unless it receives incoming jumbo frames destined for that interface.
Edit:
Your concern about the
system jumbomtu
command is misplaced. That command sets the non-default MTU size to which you can configure some types of interfaces, but it doesn't actually set the interface MTU size. The warning is about setting the value to the default ethernet MTU value. Doing that will change all the interfaces with non-default MTU sizes back to the default value.The default for the
system jumbomtu
is 9216, which seems to be what you want, so you should not need to change that value. If, however, you set your 10 Gb MTU to 9216, then you change thesystem jumbomtu
value to 1500, you will find that any interfaces where you changed the MTU to 9216 will now have an MTU" of 1500 because thesystem jumbomtu
is now different than what you are allowed to set the MTU.In other words, for some types of interfaces, You can either use the default MTU of 1500, or you can change the MTU to the value set by the
system jumbomtu
command, but not any other value.