Telecom provider not supporting MTU higher than 1515 for leased line

mtu

We have ordered a 100 Mbps leased line and did not specify our requirement to have a minimum of 1530 MTU supported. The telco now sent us a feedback that they do not support 1530, but only 1515.

Did anyone experience something similar? And what can be the rational behind for the Telecom provider not supporting higher MTU. What kind of hardware is deciding this in the case of a leased line service?

Edit: line protocol is Ethernet; also I corrected: it is a 100 Mbps we ordered.
Hope it clarifies. Thanks for your help.

Best Answer

In all likelihood, one or more segments of the leased line are tunnelled through some other technology (e.g. IP, ATM, Frame Relay). Adding up all the different networking layers tends to create this sort of problem. I've experienced this directly, and companies generally discover this "minor detail" from the telco only after signing multi-year contracts that they can't pull out of. Changing every single device in your network to support a smaller-than-standard MTU is a nightmare, and should be avoided if at all possible.

Related Solutions

How is the MTU is 65535 in UDP but ethernet does not allow frame size more than 1500 bytes

UDP datagrams have little to do with the MTU size you can make them as big as you like up to the 64K is maximum mentioned above. You can even send one of them in an entire packet as long as you are using jumbo frames with a size larger the large datagram.

However jumbo frames have to be supported by all the equipment the frame will pass over and this a problem. For practical purposes Ethernet frames are the most common tranport size, the MTU for these is circa 1500 bytes, I will say 1500 going forward, but it isn't always. When you create a UDP datagram larger than the underlying MTU (which as indicated is most often be ethernet) then it will be quietly be broken up into a number of 1500 byte frames. If you tcpdump this traffic you will see a number of packets broken at MTU boundary which will have the more fragments flag set along with a fragment number. The first packet will have a fragment number of 0 and the more fragments set and the last one will have a non-zero fragment number and more fragments not set.

So why care? The implementation detail actually matters. Fragmentation can hurt performance in the network not a big issue anymore but one to be aware of. If a huge datagram size it used then should any fragment be lost the whole datagrams will need to be resent. Equally at high volumes and today these are perfectly achievable volumes then mis-association of frames at reassembly is possible. There can also be problems getting fragmented UDP packets to traverse enterprise firewall configurations where load balancers spread the packets out, if one fragment is on one firewall and the other on a different one then the traffic will get dropped as incomplete.

So don't create UDP datagrams bigger than the MTU size fragmentation unless you have to and if you have to specify that the infrastructure being communicated between is close (same subnet close) at which point jumbo frames would likely be a good option.

Packetloss over Internet for “Linux-Linux” but not for “Windows-Linux” (tl;dr: it’s MTU)

Your packet trace shows:

22:29:22.180852 IP (tos 0x0, ttl 64, id 52989, offset 0, flags [DF], proto TCP (6), length 1900)   
SITE_B_LAN_ADDR.54358 > SITE_A.SSH_PORT: Flags [P.], cksum 0x05c4 (incorrect -> 0xadce), seq 22:1870, ack 22, win 229, options [nop,nop,TS val 4294917498 ecr 71539420], length 1848

Note its a 1900 sized byte length with a dont fragment option set on the packet. Typical MTUs tend to be between 1400-1500 bytes.

Your probably getting packet too big ICMP messages back but your dropping all ICMP traffic inbound at the site A firewall.

To test for this you'd have to do the packet trace on your firewall for icmp and tcp 22.

Make sure you permit ICMP packet too big messages inbound at site A.

Alternatively you could try setting the MTU on your Linux boxes at Site A to something under the size of your network MTU. I am hazarding a guess that on Fedora you have jumbo packets enabled but on Windows you do not.