Centos – How should I tune apache when I see high CPU, but low memory usage

apache-2.2centosmemoryperformancePHP

My CentOS web nodes (LAMP stack) are experiencing high CPU usage but low memory usage. Nearly all of the processes are apache. I know I could just add additional web nodes until my cpu usage becomes reasonable, but I imagine there are some quick tuning tips that could go a long way here.

CPU bounces between 90-100%. Mysql bounces between 5% and 45%. Apache makes up the rest.

Top:

top - 21:15:49 up 46 days, 20:39,  4 users,  load average: 34.17, 44.62, 51.03
Tasks: 133 total,  32 running, 101 sleeping,   0 stopped,   0 zombie
Cpu(s): 90.4%us,  9.6%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.1%st
Mem:   8388608k total,  6116448k used,  2272160k free,    84160k buffers
Swap: 16777208k total,       68k used, 16777140k free,  3488044k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
15775 mysql     15   0  827m 153m 3712 S 43.7  1.9   0:55.04 mysqld
15911 apache    16   0  275m  10m 3872 S 24.1  0.1   0:03.15 httpd
15816 apache    16   0  288m  23m 4116 R 23.5  0.3   0:11.88 httpd
15807 apache    15   0  281m  17m 5028 S 21.5  0.2   0:11.71 httpd
15897 apache    16   0  277m  13m 4104 S 21.2  0.2   0:02.68 httpd
15834 apache    16   0  276m  11m 3916 S 20.6  0.1   0:05.22 httpd
15842 apache    16   0  275m  11m 4116 S 19.3  0.1   0:07.86 httpd
15870 apache    16   0  294m  30m 5044 R 17.4  0.4   0:06.36 httpd
15782 apache    15   0  278m  13m 4124 R 12.2  0.2   0:08.54 httpd
15819 apache    15   0  280m  17m 5016 S 11.6  0.2   0:07.01 httpd
15683 apache    16   0  288m  23m 4128 R 10.9  0.3   0:09.90 httpd
15876 apache    16   0  284m  19m 3880 R 10.6  0.2   0:04.35 httpd
15878 apache    15   0  276m  11m 4104 S 10.6  0.1   0:06.42 httpd
15913 apache    16   0  275m  10m 3920 R  9.6  0.1   0:07.98 httpd
15898 apache    16   0  280m  16m 4104 S  9.3  0.2   0:02.85 httpd
15817 apache    16   0  277m  13m 4108 S  9.0  0.2   0:05.55 httpd
15843 apache    16   0  280m  15m 4104 R  8.7  0.2   0:10.80 httpd
15812 apache    16   0  280m  16m 5012 R  8.0  0.2   0:05.18 httpd
15844 apache    16   0  281m  16m 4116 R  7.4  0.2   0:08.63 httpd
15833 apache    16   0  281m  18m 5036 R  7.1  0.2   0:04.76 httpd
15795 apache    16   0  280m  15m 3920 R  6.8  0.2   0:08.65 httpd
15704 apache    15   0  280m  16m 4096 S  6.4  0.2   0:09.06 httpd
15849 apache    16   0  280m  15m 4124 R  6.4  0.2   0:05.59 httpd
15806 apache    16   0  276m  11m 3876 S  6.1  0.1   0:10.33 httpd
15902 apache    16   0  280m  15m 3912 R  6.1  0.2   0:03.40 httpd
15688 apache    16   0  279m  14m 4116 R  5.8  0.2   0:06.07 httpd
15904 apache    16   0  280m  15m 3860 R  5.8  0.2   0:02.74 httpd
15703 apache    16   0  281m  17m 5048 R  5.1  0.2   0:03.86 httpd
15705 apache    16   0  281m  17m 5044 R  4.5  0.2   0:13.54 httpd
15821 apache    15   0  276m  11m 4072 S  4.5  0.1   0:04.24 httpd
15830 apache    16   0  278m  13m 4112 R  4.2  0.2   0:06.37 httpd
15850 apache    16   0  277m  12m 3872 R  3.9  0.2   0:04.64 httpd
15912 apache    15   0  276m  11m 3920 S  3.9  0.1   0:05.58 httpd
15804 apache    15   0  280m  15m 3932 R  3.5  0.2   0:09.13 httpd
15805 apache    16   0  281m  15m 3932 R  3.5  0.2   0:09.00 httpd
15899 apache    16   0  277m  12m 4104 R  2.6  0.2   0:03.26 httpd
15701 apache    16   0  277m  12m 3920 R  2.3  0.2   0:08.35 httpd
15781 apache    15   0  274m 9.8m 3924 S  1.9  0.1   0:10.48 httpd
15848 apache    15   0  276m  11m 3876 S  1.9  0.1   0:06.63 httpd
15901 apache    15   0  275m  10m 3916 S  1.9  0.1   0:02.18 httpd
15879 apache    16   0  280m  16m 4104 R  1.3  0.2   0:02.03 httpd
15820 apache    16   0  281m  17m 5260 R  1.0  0.2   0:05.24 httpd

relevant httpd.conf:

Timeout 120
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5

<IfModule prefork.c>
StartServers       8
MinSpareServers    5
MaxSpareServers   20
ServerLimit      256
MaxClients       256
MaxRequestsPerChild  4000
</IfModule>

<IfModule worker.c>
StartServers         2
MaxClients         150
MinSpareThreads     25
MaxSpareThreads     75
ThreadsPerChild     25
MaxRequestsPerChild  0
</IfModule>

Best Answer

It is probably, that some PHP scripts running in mod_php consume too much CPU time. Overall recomendations are general and each of them will have some benefit:

  • Install APC cache and check that is really running (APC has bundled apc.php file which shows APC status and cache hits and misses)
  • Install nginx and install it as a reverse proxy in front of apache - it will reduce overhead of serving static files and leave much more CPU resources to server dynamic content
  • Add CustomLog to apache with %t %T directives and examine the log - you will see wich requests are running the most time, thus consuming most CPU
  • optimize/rewrite those scripts
Related Topic