Ubuntu – Hyper-V Ubuntu Networking Problems Copying Large Amounts of Data

hyper-vUbuntu

I am trying to copy a large amount (about 50 GB) of data over my network from a Hyper-V-hosted virtual machine running Ubuntu 11.04 (Natty Narwhal) to another (non-virtual) Ubuntu host that I plan to use for testing upgrades to one of our web applications. The problem I am having is with the virtual machine, which I shall refer to in what follows as "source.host". This machine is running 64-bit Ubuntu Server with the 2.6.38-8-server kernel and the Microsoft Linux Integration Components for Hyper-V kernel modules (hv_utils, hv_timesource, hv_netvsc, hv_blkvsc, hv_storvsc, and hv_vmbus) loaded. It uses a Hyper-V "synthetic network adapter" for its networking interface.

To do the copy, I log on to the machine with the data and run the following commands (Call the remote machine "destination.host".):

$ cd /path/to/data
$ tar -cvf - datafolder/ | ssh user@destination.host "cat > ~/data.tar"

This runs for a while and then suddenly stops after transferring somewhere from 2-6 GB. The terminal on the source.host machine displays a Write failed: broken pipe error. The odd part is this: after this occurs, the "source.host" machine is no longer able to talk to the rest of the network. I cannot ping any other hosts on the network from the "source.host" machine, and I cannot ping the "source.host" machine from any other host on the network. I am equally unable to access the any of the web services hosted on "source.host".

Running ifconfig on "source.host" shows the network adapter to be up and running as usual with the correct IP address and everything. I tried restarting the networking service with

$ /etc/init.d/networking restart

but the problem does not go away. Restarting the machine makes it capable of talking to the network again — it can ping and be pinged by other hosts, and the web services are also accessible and usable as normal — but attempting the copy operation again results in the same failure, requiring another restart.

As an experiment, I tried replacing the tarssh pipeline above with a straight scp:

$ scp -r datafolder/ user@destination.host:~

but to no avail

Thinking that the issue might have to do with the kernel packet-send buffers filling up, I tried increasing the buffer size to 12 MB (up from the 128 KB default) with

# echo 12582911 > /proc/sys/net/core/wmem_max

but this also had no effect.

I'm guessing at this point that it might be a problem with the Microsoft synthetic network driver, but I don't really know. Does anyone have any suggestions?

Thank you very much in advance!

Best Answer

Have you tried disabling broadcom teaming if enabled on the hyper-v network connection?

We have had to set ours to Active/Passive as the broadcom drivers + linux vm were causing mac address confusion and eventually causing the Linux guest to lose all network connectivity until rebooted/live migrated or virtual networks re-connected.