Linux – My Ubuntu 10.04 server kills all WAN bandwidth when it’s attached to the LAN. Where do you begin troubleshooting

bandwidthlinuxtroubleshooting

First I should say that my Linux knowledge is minimal; just enough to set up some servers (Apache, Tomcat, Couch, etc). I built a MiniITX server to host some simple sites, act as an SSH tunnel while I'm away, and act as a torrent server. It was not properly secured for a long time (iptables was empty, all ports open, no firewall) though my router did not have much port forwarding set up beyond HTTP, FTP, and SSH.

A week or two ago my bandwidth at home dropped from around 27Mbps to 2Mbps and my upload went from 7Mbps to 0.06Mbps. When I unplug the server from the LAN, by bandwidth shoots back up.

I threw up a restrictive iptables, removed most of the port forwarding, and checked my router logs to see if there were any open connections from the server (malware?) but there were none.

What would you do? What are the first things you'd check?

I can of course reinstall everything from scratch, but I'd like to find the root cause.

EDIT

Connected to LAN:

sudo route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.0.0     *               255.255.255.0   U     0      0        0 eth0
default         192.168.0.1     0.0.0.0         UG    100    0        0 eth0

iptraf > IP traffic monitor > eth0
alt text

which shows me x103 (the problem server) and x130 (the Mac from which I'm SSHing in) with Packets and Bytes just flying up at a constant rate, never stopping. I'm guessing that this is an infinite feedback loop, where any iptraf update needs to be sent over the wire, resulting in another update being logged, etc. Anyway, it's showing a TCP flow rate of 26 kbits/s which simply cannot account for the multi-Mbps drop in both upload and download.

iptraf > Detailed interface statistics > eth0

alt text

iptraf > Statistical breakdowns > By TCP/UDP port > eth0

alt text

Best Answer

TCPDUMP can show you the traffic on the wire. This can get messy, so you may want to redirect it to a file and browse it later:
tcpdump -s 0 -Ai eth0

-s 0 sets the length of the packet capture, 0 means as much as possible so feel free to adjust as needed.
-A prints the traffic in ASCII so you can read some of it. i eth0 sets the interface to watch. You should make sure this is your WAN interface.

With any luck you'll see what's generating your bandwidth problems.

Related Topic