I spent some time on this topic, but a lot of things is still unclear to me.
Basically I don't understand how the pre-fragmentation is supposed to be configured on modern IOS boxes (does it have to be configured at all?) and what is the relationship between "ip mtu", "crypto ipsec fragment before-encryption" and "crypto ipsec df-bit" commands.
Imagine simplest scenario possible: H1 —- R1 —– R2 —- H2. There is a IPsec over VTI tunnel between R1 and R2. IPsec over VTI automatically accounts for ESP overhead. When the tunnel is built, the MTU is automatically lowered to 1432 bytes.
In this scenario two things can happen.
- Packet comes in to R1. R1 realize that the size of the packet + overhead exceed egress interface MTU, so it will fragment it and THEN encrypt. End host H2 will be responsible for assembling the packet.
- Packet comes in to R1. It won't fit into egress interface, so it must be fragmented. Router first encrypt and then fragment. This is undesired behavior, because R2 will need to assembly the fragments before decryption. This means process switching, increased CPU load and bad things in general.
Now, the cisco DocCD says:
When a packet is nearly the size of the MTU of the outbound link of the encrypting router and it is encapsulated with IPsec headers, it is likely to exceed the MTU of the outbound link. This causes packet fragmentation after encryption. The decrypting router must then reassemble these packets in the process path, which decreases the decrypting router's performance.
The Pre-fragmentation for IPsec VPNs feature increases the decrypting router's performance by enabling it to operate in the high-performance CEF path instead of the process path. An encrypting router can predetermine the encapsulated packet size from information available in transform sets, which are configured as part of the IPsec security association (SA). If it is predetermined that the packet exceeds the MTU of the output interface, the packet is fragmented before encryption. This function avoids process-level reassembly before decryption and helps improve decryption performance and overall IPsec traffic throughput.
Note
The pre-fragmentation feature is turned off by default for tunnel interfaces. To receive pre-fragmentation performance benefits, turn pre-fragmentation on after ensuring that the tunnel interfaces have the same MTU on both ends.
Crypto maps are no longer used to define fragmentation behavior that occurred before and after encryption. Now, IPsec Virtual Tunnel Interface (also referred to as Virtual-Template interface) (VTI) fragmentation behavior is determined by the IP MTU settings that are configured on the VTI.
I've checked what are the default settings in newer IOS and they are as follows:
crypto ipsec df-bit copy
crypto ipsec fragmentation before-encrypt
So how does this reflect above statements? Is the pre-fragmentation turned off by default, or not?
When should we use "ip mtu" on an interface and when not?
Things are even more confusing, if we add the chart from the cisco DocCD to the mix.
http://www.cisco.com/c/en/us/td/docs/ios-xml/ios/sec_conn_dplane/configuration/15-mt/sec-ipsec-data-plane-15-mt-book/sec-pre-frag-vpns.html
Chart describes the relationship between "crypto ipsec df-bit" and pre-fragmentation. How can the pre-fragmentation be turned on and off with the same "crypto ipsec df-bit" setting and actual DF bit in the IP header? (last 4 rows in the chart)
If pre-fragmentation works by default, what is the purpose of "ip mtu" command? Any best practices?
Best Answer
I had a similar issue using a Static VTI IPSec tunnel. Initially the tunnel interface IP MTU was set to 1400 bytes with the "Crypto IPSec DF-bit clear" command set under the global configuration. The expected behaviour was for packets received (being forwarded to the VTI) should have the DF bit cleared and allow the packet to be fragmented before encryption is required. We found that any packets arriving with DF bit set that exceeded the tunnel IP MTU were being dropped and not switched to the tunnel interface.
After a long discussion with Cisco TAC it was explained that that VTI tunnels do not support fragmentation before encryption (as per bug ID CSCsr97396/CSCsh30577), This is apparently undocumented in any publically available Cisco documents including configuration guides... poor show Cisco considering this bug has been known since 2007!!!
The fix is to ensure the tunnel interface IP MTU is large enough (changed IP MTU to 1500) not to cause fragmentation. In the solution we deployed the egress interface is a point to point ATM interface (OC12/STM4 ATM SPA) with a default MTU of 4470 so no post encryption fragmentation will occur.
BTW, we are using ASR1006 router with ESP-10, RP2 with IOS-XE 3.13 if that helps.