Under what circumstances could adding a CPU to an “underutilized” VMware vSphere 4.1 VM increase performance

performancevirtualizationvmware-esxvmware-vsphere

Scene: 16-core VMware ESX 4.1 host. Presume that the host is not over loaded in any single resource at all. Storage latencies are very low, total memory utilization is below the physical ram amount, and not a single VM exhibits any %RDY time or significant pagefile activity.

A single-processor Windows 2008 R2 VM, fully up to date with recent VMware Tools, running PeopleSoft application tier (forgive my vagueness), WebLogic, Jrockit, etc. esxtop and vcenter show physical CPU use below 70%, physical memory below 80%, the guest machine is not exhibiting signs of significant disk pagefile activity.

Users report that the machine is not processing as many events as it "should", and when they try logging in, there is significant mouse and keyboard event lag through RDP. Everything about it just feels "slow."

A second CPU is added to the VM. %RDY time does not go up appreciably, so this is judged as a sane action. Perceived and measured application performance improves dramatically.

Everyone scratches their heads, shrugs, and are happy that its fixed. Admins who have read the VMware performance troubleshooting guide are perplexed.

What did we miss?

Best Answer

CPU Ready shows how busy the host CPUs are rather than the guest. High %RDY would probably correspond to high CPU utilization on the VM but not necessarily and if you have a lot of VMs it might not be the one in question with the issue.

From the VMWARE communities:

For every cycle of time the server has a determined number of available CPUS (including cores) to provide out to VMS. Lets say you have 2 quad core cpus. THen you have 8 available CPUS. So if you have 8 VMs running and each of them only has one vcpu then all of them get to run during that cycle. Problem occurs when you have VMs demanding more CPUs than you have available. For example if you have 6 VMS with 1 vcpus. each and 1 VM with 4 vcpus. Obviously they can't run all at the same time. VMware has to try to accomodate all the vCPUS of the 1 VM at the same time.. which means that for that one cycle either 2 VMS are on standy or the multi vCPU VM is on standby.

Certain applications that are multi-threaded will use multiple vCPUs, which will help with performance.

In your case, were you actually doing perfmon stats/logging over time to get a baseline and then doing one after the vCPU was added? That would probably give you a better indication then using the built in VMWare tools of where the differences really lie, especially at the application/process level.

Related Topic