We have ProLiant DL360 Gen8 and Gen9 servers running VMWare ESXi 6.0 with virtual machines under various versions of Windows that are routed via pfSense 2.3.4-RELEASE (64-bit) with Open-VM-Tools package 10.1.0,1.
The virtual machines that work via pfSense are demonstrating very low upload speed, for example: ping 2ms, download 134 Mbps, upload 0.25 Mbps (by the way, 0.25 Mbps is acceptable speed for Remote Desktop connections, but, in practice RDP barely works, the client frequently stalls for a few seconds, or refresh occurs in squares, taking 5-10 seconds to refresh the screen, is unstable, or sometimes even reconnects — making the work via RDP practically impossible).
Tweaks on the affected Windows machines like “netsh interface tcp set global autotuninglevel=highlyrestricted” didn’t change anything.
The virtual machines that have direct connection, bypassing pfSense, don't have these issues – they have about the same upload and download speed.
All virtual machines (pfSense, Windows, etc – all) are using VMXNET3 adapter.
The following options are all unchecked in the pfSense:
[ ] Disable hardware checksum offload
[ ] Disable hardware TCP segmentation offload
[ ] Disable hardware large receive offload
There is no traffic-shape on pfSense. What can be the reason?
If I CHECK the option "Disable hardware large receive offload", it becomes fast again, but I don't want to disable it, I want pfSense to use hardware large receive offload with VMWare VMXNET3.
Update: I have upgraded VMWare to latest 6.5 with all patches and pfSense to 3.4.5 BETA, have updated the firmware to latest versions, and it didn't help.
Best Answer
I want to absolutely confirm the same scenario. Running pfSense on VMware where the upload bandwidth would be painfully slow while download was just fine. For us, it was ONLY if the pfSense VM and the guest VMs were on the same host. When the pfSense VM and host VM were on a different host the problem went away. When disabling the offloads on the pfsense VMs (check the boxes ON) it instantly fixed the problems. I am not sure if it's only the VMXNET 3 NICs but that is how the pfSense VMs are also configured. I hope this helps others as this is not documented anywhere. I will try to get pfSense to update the VMware configuration page on their site.