I have a wordpress website wich I did a heavy load test on. As you can see below the CPU usage is of course high but at the same time the RAM usage never goes higher than 0,4%. The load test doesn't produce any errors but the loading time of the website is very long. I'm worried that the available capacity of the RAM is not really used. Or is that low RAM usage normal for 10000 clients/second?
Server specs:
- CPU: AMD Ryzen 7 PRO 3700 8-Core Processor (16 core(s))
- RAM: 64 GB
- SSD: 960 GB NVMe
- Bandwidth: 1 Gbps
- OS: Ubuntu 20.04.4 LTS
No caching activated
I get those results with the top
command
top - 14:12:22 up 4:11, 2 users, load average: 29.49, 16.27, 7.29
Tasks: 467 total, 143 running, 301 sleeping, 23 stopped, 0 zombie
%Cpu(s): 27.6 us, 45.5 sy, 0.0 ni, 24.8 id, 0.0 wa, 0.0 hi, 2.1 si, 0.0 st
MiB Mem : 64306.9 total, 50683.7 free, 7559.5 used, 6063.7 buff/cache
MiB Swap: 7630.0 total, 7630.0 free, 0.0 used. 55881.2 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
37346 nginx 20 0 270356 266728 6364 R 78.1 0.4 2:19.89 nginx
57982 fibonac+ 20 0 355720 153908 105132 S 14.0 0.2 0:03.36 php-fpm
61772 fibonac+ 20 0 353552 150692 105776 S 14.0 0.2 0:01.26 php-fpm
1524 mysql 20 0 9117616 192712 20468 S 13.6 0.3 1:17.50 mysqld
57048 fibonac+ 20 0 355788 153888 105072 S 13.6 0.2 0:04.08 php-fpm
59186 fibonac+ 20 0 353712 151920 105132 S 13.6 0.2 0:02.86 php-fpm
61435 fibonac+ 20 0 353680 150108 105120 R 13.6 0.2 0:01.66 php-fpm
57693 fibonac+ 20 0 355760 153964 105132 S 13.3 0.2 0:04.53 php-fpm
59195 fibonac+ 20 0 353684 151912 105124 S 13.3 0.2 0:02.86 php-fpm
51154 fibonac+ 20 0 357824 156160 105136 R 12.3 0.2 0:08.10 php-fpm
51197 fibonac+ 20 0 357780 156064 105088 R 12.0 0.2 0:07.05 php-fpm
58034 fibonac+ 20 0 355728 153616 105132 R 11.0 0.2 0:03.17 php-fpm
51261 fibonac+ 20 0 357780 156064 105136 R 10.0 0.2 0:06.74 php-fpm
52563 fibonac+ 20 0 358116 162056 110892 R 10.0 0.2 0:06.62 php-fpm
57410 fibonac+ 20 0 355728 153948 105132 R 10.0 0.2 0:03.86 php-fpm
59521 fibonac+ 20 0 351636 149816 105136 R 10.0 0.2 0:01.98 php-fpm
60246 fibonac+ 20 0 353752 151856 105140 R 10.0 0.2 0:02.70 php-fpm
51147 fibonac+ 20 0 357812 156156 105136 R 9.6 0.2 0:07.93 php-fpm
51210 fibonac+ 20 0 357780 156136 105136 R 9.6 0.2 0:08.08 php-fpm
51225 fibonac+ 20 0 357776 156112 105128 R 9.6 0.2 0:07.47 php-fpm
51241 fibonac+ 20 0 357780 156112 105136 R 9.6 0.2 0:06.95 php-fpm
51251 fibonac+ 20 0 357784 156072 105136 R 9.3 0.2 0:06.92 php-fpm
51255 fibonac+ 20 0 357784 156132 105140 R 9.3 0.2 0:07.43 php-fpm
57054 fibonac+ 20 0 355784 153952 105124 R 9.3 0.2 0:03.91 php-fpm
57056 fibonac+ 20 0 355768 153960 105132 R 9.3 0.2 0:03.85 php-fpm
59192 fibonac+ 20 0 353708 151508 105120 R 9.3 0.2 0:02.73 php-fpm
61746 fibonac+ 20 0 343316 140124 105140 R 9.3 0.2 0:00.74 php-fpm
51233 fibonac+ 20 0 357776 156112 105140 R 9.0 0.2 0:06.65 php-fpm
51243 fibonac+ 20 0 357784 156140 105120 R 9.0 0.2 0:07.97 php-fpm
55161 fibonac+ 20 0 355728 153988 105132 R 9.0 0.2 0:04.28 php-fpm
57424 fibonac+ 20 0 355736 153600 105132 R 9.0 0.2 0:03.30 php-fpm
59185 fibonac+ 20 0 353684 151496 104832 R 9.0 0.2 0:02.73 php-fpm
51177 fibonac+ 20 0 357812 156132 105136 R 8.6 0.2 0:07.17 php-fpm
57401 fibonac+ 20 0 355728 153620 105132 R 8.6 0.2 0:03.25 php-fpm
51239 fibonac+ 20 0 357836 156128 105136 R 8.3 0.2 0:06.93 php-fpm
54835 fibonac+ 20 0 357776 155544 104892 R 8.3 0.2 0:04.93 php-fpm
61764 fibonac+ 20 0 345492 143512 105588 R 8.3 0.2 0:01.07 php-fpm
50951 fibonac+ 20 0 357860 156164 105140 R 8.0 0.2 0:08.77 php-fpm
51024 fibonac+ 20 0 357816 156128 105136 R 8.0 0.2 0:07.97 php-fpm
51160 fibonac+ 20 0 357808 156156 105136 R 8.0 0.2 0:07.45 php-fpm
54833 fibonac+ 20 0 357832 155788 104816 R 8.0 0.2 0:06.09 php-fpm
57047 fibonac+ 20 0 355788 153864 105068 R 8.0 0.2 0:03.91 php-fpm
59193 fibonac+ 20 0 355760 153208 105080 R 8.0 0.2 0:03.08 php-fpm
63679 fibonac+ 20 0 260456 107648 80836 R 8.0 0.2 0:00.24 php-fpm
63682 fibonac+ 20 0 260456 107672 80860 R 8.0 0.2 0:00.24 php-fpm
51028 fibonac+ 20 0 357856 156068 105136 R 7.6 0.2 0:06.58 php-fpm
53620 fibonac+ 20 0 357776 156048 105132 R 7.6 0.2 0:05.68 php-fpm
54832 fibonac+ 20 0 357776 155968 105052 R 7.6 0.2 0:05.61 php-fpm
55152 fibonac+ 20 0 357780 156040 105132 R 7.6 0.2 0:05.36 php-fpm
57981 fibonac+ 20 0 355732 153596 105132 R 7.6 0.2 0:03.30 php-fpm
58035 fibonac+ 20 0 355736 153604 105132 R 7.6 0.2 0:03.29 php-fpm
59184 fibonac+ 20 0 353736 151484 105100 R 7.6 0.2 0:02.47 php-fpm
59189 fibonac+ 20 0 356072 159176 110760 R 7.6 0.2 0:05.38 php-fpm
63683 fibonac+ 20 0 260456 106668 80072 R 7.6 0.2 0:00.23 php-fpm
51208 fibonac+ 20 0 357780 156188 105140 R 7.3 0.2 0:07.25 php-fpm
51237 fibonac+ 20 0 357776 156184 105136 R 7.3 0.2 0:07.43 php-fpm
Best Answer
What order of magnitude of requests do you actually expect to get? Stack Exchange, the entire network, peaks at maybe 5,000 requests per second for ~1.3 billion page views per month. A direct comparison is difficult, but I assume what you are doing is smaller than that.
You are correct that much of your memory is not used at this point in time. 50683 MB free is a lot in both absolute number, and as 78% of 64306 MB total. Whether to treat this is as wasteful, generous allocation for growth, or one size fits most 64 GB servers, is your judgement call, as a part of capacity planning.
Sixty (?) web server processes plus some other odds and ends is no big deal for 64 GB of memory. Notice RES of about 150 MB each. While in theory that could sum to 5,000 MB, memory accounting is always more complicated than the simple assumptions people make. Linux is lazy and tends to allocate physical memory pages for unique data. Dozens of copies of the same task "compresses" well. Especially for computational benchmark workloads, which likely can drive up CPU with a small working set.
With plenty of free memory, but poor response time to an application, there definitely is other limiting factors to performance. Finding it requires a methodical approach to examining all aspects of the system. Anything from php tuning parameters, to other resources (network?), to application concurrency issues.
As a practical matter, on Linux install debug symbols and run
perf top
Knowing which functions are on CPU can help analyze what is going on deep in user or kernel code. Ideally you also have something like an APM tool that can profile code.