I've noticed recently that a VPS that I administrate has some strange Apache behaviour. Essentially, any HTTP request hangs for about a second before it is actually served.
Output from ab
shows something like this:
$ ab -n 10 -c 1 http://server-public-ip/path-to-static-file
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
<snipped>
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 3.5 0 11
Processing: 2 1373 1784.6 1000 4854
Waiting: 2 1372 1784.6 1000 4854
Total: 2 1374 1784.4 1011 4854
<snipped>
My initial thought was that my mod_perl website was running slowly, but this is running on a statically-served file! My virtual host config looks like:
<VirtualHost server-ip:80>
<Location />
DefaultType text/html
Order deny,allow
Allow from all
SetHandler perl-script
PerlHandler HTML::Mason::ApacheHandler
PerlSetVar MasonDataDir "/var/tmp/mason-cache"
PerlSetVar MasonCompRoot "<path-to-comproot>"
ReWriteEngine on
ReWriteRule ^(.*)/$ $1/index
ErrorDocument 404 /not_found
</Location>
<Location /static>
Order deny,allow
Allow from all
SetHandler default-handler
</LocationMatch>
</VirtualHost>
There is plenty of memory free (at least 32MB entirely free, i.e. not used for cache), lots of processor resources (load average is 0.00 on 1/5/15-minute windows), lots of disk I/O available (median disk transfer is less than 5 KB/s) . . . I've no idea what the problem might be. After the transfer begins it runs at full speed (more than 2MB/s, which is about the limit of my local connection.)
I'm using Debian stable, running a 2.6.32 kernel and Apache 2.2.16 (using prefork). There is little to no traffic going to the server, and I've tried tweaking the prefork settings in both directions (lots of spare workers to no spare workers).
Any thoughts?
Best Answer
Keep in mind that even if you have separate handlers for different paths, the prefork children get reused based on your
MaxRequestsPerChild
setting. This means that a child may end up servicing a request to the perl handler before it services one to the static content (especially if you've got a load balancer with a health check against the perl handler). If there's a quirk in the Perl that is gumming up the works, it can gum up the static content on subsequent requests.For this and other reasons, at a prior workplace we had a practice of isolating the static content to its own httpd process and using the load balancer to split traffic between the two based on URI context. The second webserver never even loads mod_perl, isolating any problems introduced by code to the other process.