Linux – Server Won’t Autonegotiate to 1Gb Interface

1. I have yet to see a problem with auto-negotiation of network speeds that isn't caused by either (a) a mismatch of manual on one end of the link and auto on the other or (b) a failing component of the link (cable, port, etc).

2. This depends on the admin, but my experience has shown me that if you manually specify the link speeds and duplex settings, than you are bound to run into speed mismatches. Why? Because it is nearly impossible to document the various connections between switches and servers and then follow that documentation when making changes. Most failures I have seen are because of 1(a) and you only get in to that situation when you start manually setting speed/duplex settings.

As mention in the Cisco documentation:

If you disable autonegotiation, it hides link drops and other physical layer problems. Only disable autonegotiation to end-devices, such as older Gigabit NICs that do not support Gigabit autonegotiation. Do not disable autonegotiation between switches unless absolutely required, as physical layer problems can go undetected and result in spanning tree loops.

Unless you are prepared to setup a change management system for network changes that requires the verification of speed/duplex (and don't forget flow control) or are willing to deal with occasional mismatches that come from manually specifying these settings on all network devices, then stick with the default configuration of auto/auto.

In the future, consider monitoring the errors on the switch ports with MRTG so you can spot these issues before you have a problem.

Edit: I do see a lot of people referencing negotiation failures on old equipment. Yes this was an issue a long time ago when the standards were being created and not all devices followed them. Are your NICs and switches less than 10 years old? If so, then this won't be an issue.

As @Luis pointed out, 'b' = bit, and 'B' = byte. iperf helps us out by reporting times in Mbits/sec:

[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.01 GBytes   865 Mbits/sec

Does ifconfig eth0 show any errors on either host? From your reports, it seems as if either the CPU is overloaded on one of the two iperf machines, or the switch (please tell me it's not multiple switches) between them is overloaded, or you have a 100Mbit constriction somewhere between your hosts. I've found people cascaded off of a small 100Mbit switch one too many times, so it's good to know where your cable runs start and finish. Are you using a smart switch? Does it show that your ports are active at gigabit speeds?

Once you figure out your problem, you might want to enable jumbo frames on your hosts in order to get even better throughput when you're making bulk data transfers, provided you have a reasonably-modern switch.