We're facing intermittent periods of 100% CPU utilization.
Server Configuration:
HP DL580 G7 (4 Processor with 8 cores each; 128GB memory.)
Operating System: Solaris 10_x86 update 9
Application: Oracle 10 R2; ASM for Disk Management. DB size 5TB; SGA 78GB
Storage Subsystem: HP MSA2312sa Dual Controller SAS direct attached storage
On a normal day (CPU utilization 20%) the vmstat output is provided below
kthr memory page disk faults cpu
r b w swap free re mf pi po fr de sr s0 s1 s2 s3 in sy cs us sy id
0 27 26 128133040 6469184 362 4937 829 3 22 0 117 -0 4 0 97 85888 383138 19238 19 2 79
0 20 31 129089972 4009408 294 4341 28 0 0 0 0 0 2 0 96 144240 363898 27797 12 5 82
1 17 31 128869152 3731692 243 4437 0 0 0 0 0 0 6 0 88 142738 385237 26503 10 5 84
1 21 31 128803936 3665112 283 5545 111 0 0 0 0 0 3 0 102 157962 347356 26940 12 5 82
2 20 31 128556548 3515596 274 10806 0 0 0 0 0 0 6 0 99 253881 391554 34754 13 7 80
Processes Summary:
Run Queue Processes- 0~2
Blocked Processes- 17~27
Swapped Processes- 31
CPU Utilization Summary:
User- 10%~20%
System- 2%~7%
Idle- 79%~85%
What can be the cause of such irractic CPU behaviioiur?
Why is the Blocked Processes (b) and Swapped Out Processes (w) much higher the Running Processes (r)?
Are we looking at CPU bottleneck or Memory Bottleneck or IO Bottleneck?
We do run Oracle RMAN backup, but the backup completes at 4AM everyday.
Whereas the CPU utilization shoots up to 100% during the normal business hours (10AM to 6PM), no background backups run during this period.
As for the large queries, we do run fairly long and complex queries. These queries run everyday and the CPU utilization barely crosses 40% but from the past one week we're experiencing short bursts of 100% CPU utilization.
Best Answer
Do you VM's have the same number of processor's as the host system? if so this is a bad thing, and it can prevent the scheduler from working properly. IE if you have an 8 core system then no system on that box should have 8 cores assigned to it. You can have 20 VM's with 4 Cores assigned and that is not a problem, but 1 box with 8 cores assigned can cause problems under load.