MPLS Load Balancing on Huawei S5700 – Best Practices

You certainly want to ECMP LSP's, how many to use depends on traffic levels relative to link bandwidths (critically you want to avoid LSP's being unsignalable due to being larger then a choke point).

For auto-bandwidth intervals it depends on what your aiming for, a few times an hour isn't a bad number, but less frequent may be better if you have fewer flows.

The two Windstream Routers each have an MPLS port:

• 192.168.1.2 = Host MPLS
• 192.168.2.2 = Remote MPLS

The Windstream routers also each have an open internet port to which I have attached a Firewall Router for filtering the internet, making the internet gateways:

• 192.168.1.1 = Host Gateway
• 192.168.2.1 = Remote Gateway

Problem Summary

Your problem is that you've got multiple routers on the same subnet:

• The Internet gateway at 192.168.1.1
• The Windstream MPLS router at 192.168.1.2

This is a faulty design; I completely understand that it "seems like" there is nothing wrong with this, but as you're discovering, this is a hard way to build the network.

Following up on our chat discussion, the exact problem is that SAINTJOSEPH's TCP SYN packets are delivered correctly; however, stateful inspection on your PaloAlto firewall drops SAINTSERVIUS' TCP SYN-ACK response, due to asymmetric routing in your environment. This is illustrated in the two following diagrams.

TCP SYN from SAINTJOSEPH to SAINTSERVIUS:

Your PaloAlto firewall drops the TCP SYN-ACK from SAINTSERVIUS to SAINTJOSEPH:

This tracert makes it very clear that SAINTSERVIUS defaults through 192.168.1.1 (the PaloAlto firewall):

Long Term Solutions

You should only have a single router next-hop for every subnet; however, you currently have two. This results in the drops shown in your wireshark screen capture (which indicates that TCP SYN-ACK packets never reach Windstream's MPLS network).

These are two long-term solutions which we discussed... I'm also including the quick hack we did to validate that the problem is stateful packet drops on the PaloAltos. I am assuming you are using multiple interfaces on the PaloAlto.

• Option A: Keep the PaloAlto firewalls inline with your interoffice connections (more maintenance)
• Option B: Move the PaloAlto firewalls in front of the internet connections (less maintenance, but also less comfortable)
• Option C: Quick Windows static routing hack

Better Design, Option A (MPLS re-addressing required)

This is a another way to lay out the subnet for SAINTSERVIUS (retaining your numbering scheme with /23 subnets, although it's not required...). This option keeps interoffice routing on the PaloAlto firewalls, which feels more familiar to you right now.

This is a long-term solution. You would have to work with Windstream to readdress your infrastructure. This is a lot of work. In my opinion, keeping the firewalls in the middle of your interoffice traffic is less preferable.

Better Design, Option B (MPLS re-addressing required)

This is a another way to lay out the subnet for SAINTSERVIUS (retaining your numbering scheme with /23 subnets, although it's not required...). This option offloads interoffice LAN routing to your PowerConnect switches, and relies on the PaloAlto firewalls to protect against internet threats.

This is a long-term solution. You would have to work with Windstream to readdress your infrastructure. This is a lot of work, but it offers the advantage of not having to deal with firewalls for your interoffice communication.

Suboptimal Workaround, Option C (what you used after our chat)

Open a cmd.exe window and add this route on SAINTSERVIUS as the Administrator:

route ADD 192.168.2.0 MASK 255.255.254.0 192.168.1.2

Closing remarks

I should mention that you are one of the few people who followed through with sufficient details about your question When you started, we didn't have enough details to answer the question. Now, the issues are very clear; thank you for putting the time / effort into documenting the problem well.

Personal note: If you would like an electronic copies of the diagrams as SVG / Inkscape format, please feel free to contact me at my personal email (listed in my user profile).