Microsoft Hyper-V Server CPU Scheduling and NUMA Memory Architectures

hyper-vhyper-v-server-2008-r2performancevirtualization

I suppose this question could apply to any virtualization technology, but I am trying to understand some of the details specific to Microsoft Hyper-V Server and how virtual CPU resources are scheduled across physical CPU resources.

It appears that Hyper-V Server schedules virtual CPUs from every VM in a round-robin fashion across all available physical CPU resources using time slicing. By default all VMs are scheduled equally, but a VM's CPU priorities and thresholds can be adjusted to effectively change how frequently they are scheduled.

There are some missing details here that I cannot seem to find anywhere online. This is more for just personal curiosity, but it might affect how I balance VMs across physical hosts in the future.

(1) Does the CPU load on a VM change its effective priority as compared to idle VMs? Suppose I have two VMs, each with one virtual CPU, contending for a single physical core. If VM 1 has a high CPU load and VM 2 is idle, of the N slices available in a given time period for the single resource, do both VMs receive ~N/2 slices? It seems like it would be better that VM 1 receives closer to N slices.

(2) Does Hyper-V automatically maintain any kind of physical CPU affinity? Suppose I have a single VM, two physical processors and non-uniform memory access (where each physical processor has its own local memory which is faster to access). Does Hyper-V schedule the VM's virtual CPU(s) across both physical processors? It seems like if the VM's allocated memory fit entirely in one memory location, then it would be optimal to schedule the VM on that physical CPU instead of across both CPUs.

Best Answer

(1) Hyper-V will give each of your hypothethical virtual processors an equal chance to run, since their weights are the same. However, when the VP from VM 2 ceeds its time by going idle, that spare time will be allocated to the VP from VM 1. So, in practice, VM 1 will use most of the CPU capacity.

(2) Yes, Hyper-V maintains an "ideal physical processor" associated with each virtual processor. A VP will tend to be scheduled on its ideal physical processor. Furthermore, the ideal physical processor for a VM's virtual processors will, if at all possible, be in the same physical NUMA node as the memory assigned to the VM. For a VM with multiple virtual processors, the VPs will tend to have separate ideal physical processors, but within the same physical NUMA node.

Note that VMs larger than a physical NUMA node will span more than one node, leading to non-deterministic scheduling and performance.