Linux – Apache has a long lag before responding

apache-2.2centos6latencylinux

I have an Apache server on CentOS that occasionally takes 1-5 seconds to serve a file, including a text file that is 2 characters. I have a JMeter test that gets a few different types of files, and the delay often happens on whatever file it grabs first, but not always. This is true whether there are several request in parallel or sequentially. The pages it actually needs to serve are PHP/Oracle, but I'm assuming the problem isn't in the code because PHP isn't parsing the text file (and I did confirm that).

  • HostnameLookups is off. I grepped /etc/httpd to confirm I rooted them all out.
  • People inside the network and outside the network experience the lag.
  • If I run wget from within the server itself is also subject to the lag, which leads me to think it's not a DNS issue.
  • When I see it with Apache Bench, it shows a lag both during connect and processing.
  • I've never seen the lag on a ping.
  • We temporarily added an exception to the firewall, but restored it when it made no difference.
  • I set MaxClients according the the formula given [here] (https://servercheck.in/blog/3-small-tweaks-make-apache-fly)
  • The server supports SSL, but the tests I'm running generally test over HTTP. Every once in a while I run tests over HTTPS, and its speed is generally representative of its less secure counterpart.
  • I am using AllowOverride to parse .htaccess files, but when I disabled it and tested, I still saw the same lag.
  • There are very few no allow from all/deny from all settings lurking in httpd.conf
  • I'm still looking into Apache MPM and valgrind.

Some relevant Apache settings:

StartServers            8
MinSpareServers         5
MaxSpareServers        20
ServerLimit           256
MaxClients            136
MaxRequestsPerChild  4000

Here's a screenshot of the five-second wait time getting the text file. wget reports the transaction took 0 seconds, but you can see the start and end times are 5 seconds apart.
JMeter results
ab results didn't seem helpful, but I may be failing to interpret them properly.

Some of the resources I've consulted:

Any help would be much appreciated. Thank you.

Best Answer

A colleague pointed me at this page which recommended adding the following to /etc/resolv.conf:

options single-request-reopen

I'm not sure if that's what fixed it, or if it was some combination of several things I tried, but other people accessing the resource indicated a significant speed up after I added this. I hope it helps someone else.

Related Topic