Network speed between a VM and another machine which is not residing on the same host, is 11MB/s at most

networkingvmware-esxi

Problem
Network speed between a VM and another machine which is not residing on the same host, is 11MB/s at most.

Topology

topology

Facts

  • ESXi5 version is 5.0.0.504890
  • VM has the latest Vmware Tools installed
  • VM is using E1000 network driver
  • Physical box has Win Srv 2008 R2 as the OS
  • CrystalDiskMark says the drive on physical box can read/write 100MB/s
  • vCenter is another vm on esx
  • both vm and physical box are showing 1Gbps link speed
  • Configuration > Networking shows vmnic0 as 1000 Full
  • NTttcp is a client/server tool from Microsoft for measuring pure network throughput

Here's what I've done so far:

Test1:

  • VM is running Filezilla FTP Server (default settings, one user account made)
  • Physical box is running Filezilla FTP Client (default settings)
  • Physical box is uploading a big file to FTP server
  • Transfer speed (as observed by Windows Task Manager on both machines): ~11MB/s (bad)
  • Physical box is downloading that file from FTP server
  • Transfer speed (as observed by Windows Task Manager on both machines): still ~11MB/s (bad)

Could it be disk performance issue?

Test2:

  • Physical box is running ntttcpr.exe -a 6 -m 6,0,VM_IP_ADDRESS
  • VM is running ntttcps.exe -a 6 -m 6,0,PHY_BOX_IP_ADDRESS
  • Transfer speed (as observed by Windows Task Manager on both machines): ~11MB/s (bad)

Could it be switch performance issue?

Test3:

  • physical box is running vSphere Client
  • I open Summary > Storage > datastore > Browse Datastore… from physical box and upload a file to datastore
  • Transfer speed (as observed by Windows Task Manager on physical box): ~26-36MB/s (good)

Could it be a vm specific issue?

Test4:

  • Installed ntttcp to another vm on the same esx server
  • Measured network performance between vms on the same esx server with NTttcp
  • Transfer speed (as observed by Windows Task Manager on physical box): ~90-120MB/s (excellent 🙂

Test5:
I have another esx server on the same site, connecting to the same datastore and same switch. Those two ESX servers have both 2 NICs. One NIC goes to switch while the other goes directly to the other ESX server.

  • vMotioned one of the testing vms off to the other ESX host
  • Measured network performance between vms on different esx servers with NTttcp
  • Transfer speed (as observed by Windows Task Manager on physical box): ~11MB/s (bad)

While I'm aware of these:

they did not help (or I must have been missed something)

Best Answer

11Mb/s is too close to 100Mbps to be only a coincidence. It's clear that you have a problem with one of the network ports, either on the switch or on one of your servers's NIC, not being set to 1Gbps/full duplex. There's no doubt about that. The question is which one.

Make sure all your NICs are set to 1Gbps/full duplex, and that every single port of all network devices between all of your servers and storage devices (switches and routers) are also set to 1Gbps/full duplex.

Related Topic