I'll first say that I didn't design this network from the get go, so the topology came as a surprise even to me.
There are two subnets (one is our companys and one is our clients) which reside in the same physical location, and because of that the networks are separated by VLAN:s. However, in addition to this both us and our client have different firewalls, so we have an additional virtual router (VyOS) which acts as a bridge between our networks. VyOS router has two interfaces (eth0 and eth1).
I'm able to ping both networks just fine, but if I try to go to our clients web server at their subnet the connection fails. What is especially interesting is that when I manually set my gateway to point at 192.168.5.34 (VyOS) instead of our gateway at 192.168.5.1 the connection works, so there must be something failing at the firewall when it has to redirect the traffic back out from the same interface where it came from. Also, if I configure source nat it works as well.
Here is the information about the networks:
our subnet: 192.168.5.0/24
firewall: 192.168.5.1
VyOS eth1: 192.168.5.34
client subnet: 192.168.1.0/24
firewall: 192.168.1.1
VyOS eth0: 192.168.1.34
EDIT: This is the route the traffic takes when I try to connect to the web server over HTTP
192.168.5.172 -> 192.168.5.1 -> 192.168.5.34 -> 192.168.1.28 |
192.168.1.28 -> 192.168.1.1 (seems like the connection fails here, at our Juniper firewall)
Best Answer
Your network is working as expected. Lets draw it:
This is a simplification I'm sure, but has enough information here to show the problem and the various fixes (and their drawbacks)
Assumptions
For those following the OSI stack, this is all IP and is at layer 3.
1. Directly Connected Networks
Your PC has a packet to send to an IP. First thing it does is checks if Destination IP is in the same local IP network. If both the SRC and DST addresses share the same Network (being the bit of the IP that remains after the subnet mask is applied) then the OS sents the packet out an ethernet interface. The OS labels the outbound packet with the source IP of the interface on which it exits.
Your test PC sends packets with a SRC IP of 192.168.5.172 Destination is 192.168.5.250 Your netmask is /24 which is 255.255.255.0, leaving a network of 192.168.5.x Therefore your PC can send directly by putting the packet out the ethernet and the destination will receive it.
2. Default Gateways
A default gateway, default route, gateway, router of last resort, is the thing you send packets to when you don't have a better route to send them.
Here are the default routes in your network:
So if the destination IP is not local (directly connected) then TestPC knows to address the packet via the default gateway.
There is only one default gateway IP address for a device (handwave - yes I'm trying to keep it simple.)
(side comment) If the VyOS box has a default gateway set, it is probably via your office Juniper. That's nice for updates and stuff, but its not relevant to your environment. Its also totally reasonable for VyOS to not have any default gateway configured at all.
3. Specific Routes
Your traceroute from the question indicates that the two office firewalls have specific knowledge of the other LAN.
So each firewall will have an additional route configured.
Your Company Juniper knows to "access 192.168.1.0/24 via 192.168.5.34"
Their firewall knows that "192.168.5.0 255.255.255.0 via 192.168.1.34" or similar. Pictorially that means:
4. Be one with the packet
So lets visualise this from the point of view of a packet.
a. TestPC pings google (I'm using ping because its stateless and easier than explaining a TCP connection like HTTP)
This as a picture:
b. TestPC will now ping Customer's Server at 192.168.1.28
Here's that last as a picture:
How do we fix this? There are multiple different fixes.
1. Interconnect Network
This is the "proper" design, using a small interconnect network between your two firewalls, and doing away with the VyOS device. I've used 172.22.22.x/24 as an interconnect LAN. A /24 is quite large, but keeping this simple.
Positives
Drawbacks
2. Add a static route on the Customer's firewall
It seems to be missing. If you added the blue arrow from Customer Firewall to VyOS then it would work better.
3. Add a SOURCE NAT at the VyOS device
Someone famous once said "If your plan involves adding more layers of NAT then you don't understand the root problem" I don't really think this is a good idea.
If the VyOS device NATs all traffic between the two networks to its OWN IP in that LAN then reply traffic will not attempt to go to the default gateway.
Main downside is that you'll see all traffic coming from the VyOS IP, and you will have no idea who the real sender is. This makes forensics somewhat more difficult.
4. Static route all the things!
This is a terrible answer, but in some situations it might be appropriate.
If every LAN device had a routing table like this, then they'd all know how to talk to the custy LAN.
Downside, this is a management nightmare. It might be workable if you have only a couple devices that never move, but adding them dynamically is messy.
Possible saves
In summary
LAN Maps are awesome solutions for understanding your networking problem. Love them! I've used http://www.gliffy.com/ to create the background image for these illustrations.
KISS If a solution is tedious and complex, its probably the wrong solution.