Linux – Why is the KVM, RHEL6.2 server swapping

kvm-virtualizationlinuxmemoryswap

I have a RHEL6.2 server that I'm using to run KVM virtual machines.

The server itself has 16 GB of RAM. I want to see the biggest VM I can run on it and not let the qemu-kvm process swap. The VM ram is ~15GB. (Yes, I realize that this is pushing the limit, but read to the end before answering with something like "15GB is too much".)

[root@xxx libvirt]# virsh dumpxml VM2 | grep -i memory
  <memory>15000000</memory>
  <currentMemory>15000000</currentMemory>

[root@xxx libvirt]# ps -ef | grep kvm
root       872     1 16 10:55 ?        00:03:00 /usr/libexec/qemu-kvm [...] -m 14649 -name VM2 [...]


[root@xxx libvirt]# free -k
             total       used       free     shared    buffers     cached
Mem:      16332640   16194440     138200          0       1544      15700
-/+ buffers/cache:   16177196     155444
Swap:     35651568    7583432   28068136

But the RSS of the KVM process is only 880 MB (column 6 below). I expect it to be more like 12-14 GB.

[root@xxx libvirt]# ps -eF | grep kvm
root       872     1 14 4534221 882916 7 10:55 ?       00:03:11 /usr/libexec/qemu-kvm 

And, if I add up the RSS of all processes, its only ~ 1gig.

[root@xxx libvirt]# ps -eF | awk '{print $6}' | grep '[0-9]' | tr '\n' '+' | sed 's/+$/\n/' | bc
1004064

Here are the processes that are the biggest memory users (RSS, column 6).

root      5188 22329  0 27572  1192   4 11:19 pts/0    00:00:00 ps -eF
root     31461     1  0 10746  1236   7 Jul25 ?        00:06:22 [...]
root      6339  6275  0 272676 3288   4 Jul27 ?        00:13:38 [...]
root      2059     1  1 443909 13352  7 Jul17 ?        05:29:11 libvirtd --daemon
root       872     1 13 4534221 928300 2 10:55 ?       00:03:24 /usr/libexec/qemu-kvm [...]

I expect that about 300-500 MB of memory should still be available for the VM.

UPDATE

After rebooting the machine, I cannot reproduce this anymore. The system now works as I expect it to. Here are these expected numbers.

Oh, I should mention. There is a memory intensive program running in the VM. It allocates 80% of the total memory and continually writes random numbers to it.

RSS of the VM:

[root@hb05b15 ~]# ps -eF | grep kvm
root      7330     1 97 4520362 12483728 2 11:59 ?     00:39:55 /usr/libexec/qemu-kvm [...]

Mem and swap numbers:

[root@hb05b15 ~]# free -k
             total       used       free     shared    buffers     cached
Mem:      16332640   13277468    3055172          0      21064     215196
-/+ buffers/cache:   13041208    3291432
Swap:     35651568          0   35651568

Sum up RSS for all processes:

[root@hb05b15 ~]# ps -eF | awk '{print $6}' | grep '[0-9]' | tr '\n' '+' | sed 's/+$/\n/' | bc
12607180

Best Answer

You don't have any RAM left for the operating system, filesystem cache or anything else.

Don't push your VM's RAM so high without leaving adequate headroom.

Now, of course, you could disable swap and see what happens...