Ubuntu – MongoDB CPU usage in top is confusing

mongodbperformanceUbuntu

I have an Ubuntu 12.04 VM running in VirtualBox that has a whole MongoDB cluster on it (4 data nodes, one config node, and a mongos router). When I look at "top", I see that each of those processes takes 5-15% CPU time each. But when I look at the total CPU % used at the top, it says 98% idle. The VM has 8 CPUs cores assigned, but even so, how does this output add up?

top - 20:53:31 up 9 min,  2 users,  load average: 0.46, 0.76, 0.49
Tasks: 190 total,   1 running, 189 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.1%us,  1.9%sy,  0.0%ni, 98.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   8177528k total,  3517632k used,  4659896k free,    84696k buffers
Swap:  4191228k total,        0k used,  4191228k free,  1578544k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 1709 root      20   0 7692m  47m  19m S   16  0.6   0:45.53 mongod
 4215 ben       20   0 83964 3340 1840 S    9  0.0   0:32.92 mongos
 1704 root      20   0 6672m  47m  19m S    8  0.6   0:46.89 mongod
 1693 root      20   0 6717m  46m  19m S    6  0.6   0:45.33 mongod
 1695 root      20   0 6717m  46m  19m S    6  0.6   0:43.44 mongod
 1696 root      20   0  230m  46m  19m S    5  0.6   0:43.47 mongod

16 + 9 + 8 + 6 + 6 + 5 = 50% of one CPU core. 50% / 8 cores = 6% of the total CPU time. And that's not counting a number of additional processes taking 1-2% each. How can it be saying 98% idle?

Best Answer

I think that it's connected with the representation of top. E.g., values in %CPU column are based on elapsed time (as written in top's man page). See also this question.