Hello people of serverfault. I am currently running into some issues.
Currently I have a total of 7 servers. 6 of them have Ubuntu 12.04. The other one is a clearos. Currently the clearos machine connects to the modem/internet and then my poweredge 3348 managed switch.
3 of those machines have no issues and I can ping the gateway address 10.0.1.1 and ping the ip's of all 3 nic's of each server from the gateway with no issues.
The other 3 servers have issues. Only 1 nic from each server is pingable from the gateway and the other two are unavailable.
Here is an example of my mail server.
miverson@mail:~$ ifconfig
eth0 Link encap:Ethernet HWaddr 00:0b:db:e2:96:5a
inet addr:10.0.1.220 Bcast:192.168.255.255 Mask:255.255.255.0
inet6 addr: fe80::20b:dbff:fee2:965a/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:26013 errors:0 dropped:0 overruns:0 frame:0
TX packets:17550 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3801680 (3.8 MB) TX bytes:1921219 (1.9 MB)
Interrupt:28
eth1 Link encap:Ethernet HWaddr 00:0b:db:e2:96:5b
inet addr:10.0.1.221 Bcast:192.168.255.255 Mask:255.255.255.0
inet6 addr: fe80::20b:dbff:fee2:965b/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:8131 errors:0 dropped:0 overruns:0 frame:0
TX packets:11 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:597342 (597.3 KB) TX bytes:2222 (2.2 KB)
Interrupt:29
eth2 Link encap:Ethernet HWaddr 00:11:25:bf:42:59
inet addr:10.0.1.222 Bcast:192.168.255.255 Mask:255.255.255.0
inet6 addr: fe80::211:25ff:febf:4259/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:6581 errors:0 dropped:0 overruns:0 frame:0
TX packets:20 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:459462 (459.4 KB) TX bytes:3672 (3.6 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:54 errors:0 dropped:0 overruns:0 frame:0
TX packets:54 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:4644 (4.6 KB) TX bytes:4644 (4.6 KB)
My sudo route command from that server
miverson@mail:~$ sudo route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default 10.0.1.1 0.0.0.0 UG 0 0 0 eth0
10.0.1.0 * 255.255.255.0 U 0 0 0 eth0
10.0.1.0 * 255.255.255.0 U 0 0 0 eth1
10.0.1.0 * 255.255.255.0 U 0 0 0 eth2
and my arp command
miverson@mail:~$ arp
Address HWtype HWaddress Flags Mask Iface
10.0.1.210 ether 00:0b:db:e7:56:e8 C eth0
10.0.1.1 ether 00:0d:56:b9:33:be C eth0
10.0.1.144 ether 90:27:e4:f4:9d:60 C eth0
MySQL.therabbitry.org ether 00:0b:db:e7:56:e2 C eth0
When I ping 10.0.1.220 I get a response and it's fine, but if I ping 10.0.1.221 or 10.0.1.222 from the gateway I get 100% packet loss.
Help me out and let me know what my solution is, or if you have any other questions.
EDIT: I just realized that my Bcast address doesn't look correct. Any recommendations? Eth0 Eth1 Eth2 are all set via dhcp and are not static.
Matt
Best Answer
A couple of things:
Your broadcast address is incorrect for all your interfaces. It should be set to 10.0.1.255, assuming your netmask of 255.255.255.0 is correct.
Your default route seems to be set correctly however it will only work with a single interface. Think of it like this: Your default route is the route of last resort. If your server has no idea how to get to where you want your traffic to go, it'll try the default route. Your default route says, try to find the gateway at 10.0.1.1/24 via
eth0
, he should know where to go from there.Your other interfaces
eth1
andeth2
have routes but they are not involved in the default route. This is probably why you pings are successfully returned to 10.0.1.220. Assuming you are not doing any VLANing, as long as all of your interfaces have the same correct subnet mask they should be able to communicate "link-local", that is without having to involve a router since all the interfaces are presumably in the same subnet.This is your clue. Either your router is misconfigured or your switch is doing something to disrupt the continuity of Layer-2. Additionally check out this fantastic answer on how subnetting works.