Apache mpm_worker + php : processes not terminating

apache-2.2mpm-workerprocess

I'm seeing a weird issue on our web servers where the Apache processes seem to be remaining in limbo with 1 or 2 threads. This is disconcerting for me since I was just thinking I wanted to impose a stricter ServerLimit so that I can more safely up my php memory_limit.

Would anyone know of a nice way to troubleshoot this? Figure out what the hell those processes are doing? What state they are in?

I'm not entirely sure, but I would assume that this is caused by the fact that we never restart the Apache service, and do execute 2-3 reloads a day for various reasons.

As an example, one of the Web Servers is sitting around bored while the other is fairly taxed : 

  PID        PPID       RES        THREADS   
  *15608     1          29 MB      1         
  8880       1          2 MB       3         
  14989      1          0 MB       1         
  25029      1          0 MB       1         
  2115       15608      3 MB       2         
  13058      15608      361 MB     2         
  14556      15608      157 MB     27        
  19962      15608      23 MB      1         
  21219      15608      387 MB     3         
  21600      15608      3 MB       3         
  21828      15608      3 MB       2         
  30152      15608      3 MB       2

In this example, there's 3 processes which I'm assuming were once the control process (pre reload), which are still running for whatever reason. And of the processes attached to the new control process, a bunch are still up with 2-3 threads each.

From 11 hours later, a lot of those same PIDs are still around with 1 thread. You'd think if MaxRequests had been reached the darned thing would have quit. 

memory usage report for apache2
  pid count                     : 11                  
  thread count                  : 46                  
  shared mem usage              : 670 MB              
  total resident size           : 691 MB              
  total cpu usage               : 79 %                
  total memory usage            : 7 %                 
  average mem / pid             : 62 MB               

process breakdown
  PID        PPID       RES        THREADS   
  *15608     1          29 MB      1         
  8880       1          2 MB       3         
  14989      1          0 MB       1         
  25029      1          0 MB       1         
  2115       15608      3 MB       2         
  19962      15608      23 MB      1         
  21219      15608      387 MB     3         
  21600      15608      3 MB       3         
  21828      15608      3 MB       2         
  29239      15608      238 MB     27        
  30152      15608      3 MB       2         

process tree
     |-apache2-+-3*[apache2---{apache2}]
     |         |-apache2
     |         |-2*[apache2---2*[{apache2}]]
     |         `-apache2---26*[{apache2}]
     |-apache2---2*[{apache2}]
     |-2*[apache2]

Best Answer

First thing to check is whether these have spawned any additional processes (have you got pstree available?). Are the php exec functions enabled?

Other things to look at at whether you've got keepalives enabled and whether maxRequests is set to a sensible value. Are the threads idle or processing stuff?

Is the PHP talking to a database backend? Once the thread of execution goes into the DB client lib, PHP constraints regarding memory usage and time limits are no longer applied until the thread goes back to PHP. Are you seeing slow queries on your database?

Other things worth checking are whether the MTA is configured correctly - e.g. if you are using sendmail with a smart relay and the address you've configured is wrong, then php's mail() function can block for long periods.

Rob Olmos's reply regarding Rasmus Lerdorf's comment was made 6 years ago while there may still be some truth in the statement, it's not likely to apply to the mainstream extensions.

You didn't say what OS you're using - IMHO the benefits of threading on Linux compared to pre-forking are barely detectable.

Certainly if you've exhausted the other avenues and you don't have a compelling reason to stick with the threaded apache, then its worth trying the pre-fork - but do have a look elsewhere first.

Related Topic