CentOS, two nics eth0 eth1 with different subnets aren’t reachable outside VLAN/subnet

centosnetworkingroutingstatic-routes

I am having an issue with a CentOS 6.3 box.
We have two NICs (eth0 and eth1) in the server which each have an ip from a different subnet assigned, let's say eth0:192.168.1.2/24 (gateway 192.168.1.1) and eth1:192.168.2.2/24 (gateway 192.168.2.1). The real ips are world routable though.

Each NIC is connected to a different switch but ends up on one router in the end.
On the router these two subnets are in different VLANs, the ports to the NICs are untagged so no vlan id is passed onto the server.

Now on FreeBSD I just assign the ips to each NIC and it just works, both IPs are reachable.
On CentOS I can only reach the IP of which the gateway is active as the default route. I can ping the IP anything within the same VLAN/subnet but outside that it is unreachable.

When I do a traceroute to each of the gateways i see they go over the appropriate NIC, makes sense as the gatewaty is inside the scope. However outside the subnet I can only ping 192.168.1.2 currently.

IPtables is not active at the moment either.

What would be the required action to get this to work?

I googled around for hours, tried different approaches but it won't work.
I have the feeling I am missing something important, hopefully an easy fix 🙂

Any help is appreciated
Thanks !
Scott

ROUTING

# route -n  
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
192.168.2.0     0.0.0.0         255.255.255.0   U     0      0        0 eth1
169.254.0.0     0.0.0.0         255.255.0.0     U     1002   0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     1003   0        0 eth1
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth0

IP INFO

# ip addr list
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether b8:ac:6f:16:70:c7 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.2/24 brd 192.168.1.255 scope global eth0
    inet6 fe80::baac:6fff:fe16:70c7/64 scope link 
       valid_lft forever preferred_lft forever

3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether b8:ac:6f:16:70:c9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.2/24 brd 192.168.2.255 scope global eth1
    inet6 fe80::baac:6fff:fe16:70c9/64 scope link 
       valid_lft forever preferred_lft forever

Best Answer

You need to create a multipath route or active a dynamic routing protocol (ospf, isis).

Mulitpath example:

ip route del default via 192.168.1.1 dev eth0
ip route add default scope global nexthop via 192.168.1.1 dev eth0 weight 1 \
        nexthop via 192.168.2.1 dev eth1 weight 1