Does apache 2.4 have a hard limit on connections somewhere

apache-2.4php5

Im using Apache 2.4 with prefork.

My MPM directives are set as follows:

<IfModule mpm_prefork_module>
MinSpareServers 200
MaxSpareServers 400
StartServers 200
ServerLimit 1300
MaxClients 1300
MaxRequestWorkers   1300
MaxRequestsPerChild 10000
</IfModule>

However my server-status page shows:

Parent Server Config. Generation: 1
Parent Server MPM Generation: 0
Server uptime: 34 minutes 54 seconds
Total accesses: 551221 - Total Traffic: 11.8 GB
CPU Usage: u8151.55 s2320.18 cu.02 cs.11 - 500% CPU load
263 requests/sec - 5.8 MB/second - 22.5 kB/request
1024 requests currently being processed, 0 idle workers

WKWKKWWWWKWKKKWKKCRWWKWWKKWKKWKWKWKWWWWWKKKWKWWKKWKKWWWWWKWWKWWK
WWWWKWKWWWWWKWKKWKWKWCKWKKKKKWKKWWKKCKKKKWWWKKWKKWWWKKWCKKWKKKKW
WWKKWKKWWWKWKWWKKKWKKWWWKKWWKKKWWKKKKWKKWKCKKWKKWKKKKWWWKWKKWKWW
WWKWWKWWKKWKWKKWKRWKWKKWWWWWWKCKKKKKWWWKWWWKKKKWWKWWKWWKKKKWWKKK
WKKKWWWWWWWKWKWKWWKKWKKKKKKWKWWKWWWKKKKWKKKWWKKWWKKWKWKWWWWKKWKW
KWKWWWWKCWKKKCWKKWWWWWKWKWWKWKWRWKKWWKWKWKKWWKKWKWKWKKKKWWKKKWKW
WKKKKKWKKKKWWKWKWKWWWKKKWWKWCCKWKWWKKWKKWWCKKWWWWKKWKWWWWWWWWKKK
WWKKKWKWKKKKKKKKWKKKKKKKWWWCKWWWKKWWWWKWCWWKKKKWKWWKKKWKWKWWWWWW
WKWKKKWKWKKWWWWKWWWWWKKCWKWWKWKWWWWWWKWWWWKKWWWKWWWWKWKWKKKWKKWK
WWKKKWKWWWKKWWWKWKWKWKKKKWKWWWWWWWKKWWWWKKKCKKKWKWKWWWWKWKWWWWWW
WWKKKWKWWWKWWKWWKKWKWKWKWKKWKKWKKKCKKWWWKWCWKKWWKKWWKKWKKWWKKWWK
WWKKWKWKKWKWKWKWWKWKKWWKKWWKKKWKWWKKWWWKKWWKKCKKKWWKKWKWWWCWKKWK
KKWKWKWWWWWKKKKKKKWKWKWKWWWWKKKWWKWKWKKKWKWWWKWCWWKWKWWKKKWWWCWK
WCWCKWWKKWKWWKWKKWWWKKWWKRKWWKKWWWKWKWWKWKKWWWKKWWKWKWKWWWKWKWKW
WKWKWWKWKKKWWWKWWWKCWWWWKWWWKKKKKKWKWKKKKKKKKWKKWWWWWWWWWKWWWKKW
KWKKKWWKKWWKWKWWKKWWKKKKKWKWWWKKWKWWKWWWKKKWKWCKKKKWWKWWKWCKKCKK
SSSSSSSSSSSSSSSSSSSSSSSSS.......................................
................................................................
................................................................
................................................................
....................

Notice the line that says 1024 requests currently being processeed with 0 idle workers. It just pegs at 1024 requests.

Now at the top of my /etc/init.d/httpd I have ulimit -HSn 32768 so I don't think it's an open files limit issue.

I can't really think of why connections are getting cut at 1024 even though my MaxClients is set higher and server-status seems to want more connections but stops at 1024.

Best Answer

According to the Apache 2.4 ServerLimit docs:

There is a hard limit of ServerLimit 20000 compiled into the server (for the prefork MPM 200000).

So, you should not be hitting the hard limit.

I've had trouble adjusting ServerLimit in earlier versions; try apache2ctl stop and then apache2ctl start. (Using graceful and even restart failed to pick up the change.)

On a side note: You may want to reduce your KeepAliveTimeout and/or MaxKeepAliveRequests, or try KeepAlive Off since you have so many workers in the 'K' state. Also, those 'S'es indicate that the server is having trouble starting up workers, likely due to the 500% CPU load (which I assume is the max available.) If tweaking the KeepAlive settings doesn't help, you may need to look into upgrading the server or load balancing.

Related Solutions

Apache Scoreboard Full – Meaning of “AH00485: Scoreboard is Full, Not at MaxRequestWorkers”

We had the same problem on Apache 2.4.6. After monitoring the server and adjusting the setting for several hours it appears to us that Apache may have a bug. What appears to happen is that the server processes occasionally goes into the G state (Gracefully finishing) and restarts to accept new requests, that's normal. What is not normal is that for some reason this can take up to a few minutes to restart. If you only have a few server process running and they all go into the G state at the same time then your scoreboard fills up and you won't be able to server any more requests.

What we did was increase the number of servers so there is a less of a chance that they will all go into the G state at the same time. Also make sure you allocate at least 25 threads (MaxRequestWorkers) for each server process because that appears to be the default (i.e. if 5 Servers x 25 ThreadsPerChild = 125 MaxRequestWorkers). You can change ThreadsPerChild if you like, we left it at default. If you don't allocate enough threads the additional servers will not start. We left MinSpareThreads at the default value which is 25 and the default for MaxSpareThreads which is 75. If you do modify these settings, the value for MaxSpareThreads must be greater than or equal to the sum of MinSpareThreads and ThreadsPerChild. Also MaxRequestWorkers must be equal to or less than the ServerLimit.

Here is what worked for us but it might not be the best configuration for you.

StartServers 3
MinSpareServers 5
MaxSpareServers 10
ServerLimit 250
MaxRequestWorkers 250
MaxConnectionsPerChild 1000
KeepAlive Off

Edit: This is a confirmed bug in httpd's mpm_event module which might not be fixable through configuration.
The linked bugtracker entry has a presumed patch and more discussion about how to fix this until a new version of the event module is officially released.

Ubuntu – Optimize Apache for high concurrent requests

Stop using prefork module and start using event mpm in apache. Prefork is the old method and moving forward you want to install event mpm.

Best Answer

Related Solutions

Apache Scoreboard Full – Meaning of “AH00485: Scoreboard is Full, Not at MaxRequestWorkers”

Ubuntu – Optimize Apache for high concurrent requests

Related Topic