Ubuntu – Server Load very high despite CPU usage being low

high-loadtopUbuntuubuntu-12.04uptime

When I run the top command I get this

top - 23:20:50 up  1:25,  1 user,  load average: 11.02, 11.20, 10.41
Tasks: 262 total,   3 running, 258 sleeping,   1 stopped,   0 zombie
Cpu(s): 75.6%us,  6.1%sy,  0.0%ni,  3.1%id, 14.3%wa,  0.0%hi,  0.8%si,  0.0%st
Mem:   2028800k total,  1669384k used,   359416k free,   153300k buffers
Swap:   523260k total,     2636k used,   520624k free,   749404k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
10221 www-data  20   0  416m  24m 5376 S   46  1.2   0:27.88 apache2
11290 www-data  20   0  420m  28m 3964 S   28  1.4   0:09.30 apache2
11844 www-data  20   0  424m  31m 5336 S   21  1.6   0:04.00 apache2
11670 www-data  20   0  410m  18m 3688 S   18  1.0   0:04.10 apache2
11147 www-data  20   0  417m  25m 5360 R   15  1.3   0:09.71 apache2
10615 www-data  20   0  418m  26m 5460 S    6  1.3   0:18.89 apache2
 3014 mysql     20   0 1316m 128m 8188 S    6  6.5   4:24.84 mysqld
10852 www-data  20   0  419m  26m 5376 S    6  1.4   0:16.05 apache2
11278 www-data  20   0  420m  28m 3984 S    3  1.5   0:10.39 apache2
 1589 root      20   0     0    0    0 D    1  0.0   1:16.40 jbd2/sda1-8
12024 www-data  20   0 81044 4732 3180 S    1  0.2   0:00.04 sendmail
 5281 root      20   0 97.9m 4696 1800 D    1  0.2   0:56.55 sendmail-mta
11927 root      20   0 17464 1452  932 R    1  0.1   0:00.32 top
12009 root      20   0 99.6m 5232 2720 D    1  0.3   0:00.06 sendmail-mta
 2929 syslog    20   0  243m 3104 1140 S    1  0.2   0:25.32 rsyslogd
 3029 bind      20   0  238m  21m 3032 S    1  1.1   0:27.77 named
 6627 root      20   0  101m 6872 2852 D    1  0.3   0:07.54 sendmail-mta
10525 root      20   0  100m 5308 1536 D    1  0.3   0:02.33 sendmail-mta
14241 root      20   0  100m 6136 2868 S    1  0.3   0:31.78 sendmail-mta
18543 root      20   0  100m 6300 2868 R    1  0.3   0:27.42 sendmail-mta
22589 root      20   0  100m 6472 2884 S    1  0.3   0:22.43 sendmail-mta
31196 root      20   0  100m 6604 2852 D    1  0.3   0:16.98 sendmail-mta
    1 root      20   0 24332 2012 1356 S    0  0.1   0:05.23 init
 1391 root      20   0     0    0    0 S    0  0.0   0:02.97 kworker/0:0
 2549 root      20   0  101m 6728 2852 D    0  0.3   0:12.15 sendmail-mta
 3395 smmsp     20   0 83048 5076 1460 S    0  0.3   0:24.24 sendmail-msp
 3661 ntp       20   0 37772 2252 1620 S    0  0.1   0:00.39 ntpd
 5382 smmsp     20   0 83048 6924 3324 S    0  0.3   0:20.41 sendmail-msp
 5483 root      20   0 97.9m 4696 1800 D    0  0.2   0:56.38 sendmail-mta
 7502 root      20   0     0    0    0 S    0  0.0   0:00.80 kworker/1:0
12025 root      20   0 99700 3956 1660 D    0  0.2   0:00.01 sendmail-mta
    2 root      20   0     0    0    0 S    0  0.0   0:00.00 kthreadd
    3 root      20   0     0    0    0 S    0  0.0   0:00.10 ksoftirqd/0
    6 root      RT   0     0    0    0 S    0  0.0   0:00.00 migration/0
    7 root      RT   0     0    0    0 S    0  0.0   0:00.00 migration/1
    9 root      20   0     0    0    0 S    0  0.0   0:00.58 ksoftirqd/1
   11 root       0 -20     0    0    0 S    0  0.0   0:00.00 cpuset

Yet when I run uptime I get

22:53:23 up 57 min,  1 user,  load average: 8.38, 9.22, 8.88

And as a result, my vBulletin forums are locking out all users.

Something obvious seems to be wrong, how can I identify and solve the issue?

Many thanks.

Best Answer

So you actually are using a lot of CPU. Either get a better server or make your forums become less popular. You also seem to be sending quite a bit of mail... is your forum hacked and is somebody using it as a spam source? Check your mail logs...