Php – Apache Server with memcache, varnish and php slow request times

apache-2.2memcachedPHPvarnish

My issue is that these servers are taking rather long for request about 2 seconds on average just to serve files. When we had just one server doing everything it was noticeably faster even with the same web app (Drupal 6 and Drupal 7). I want to get this number down to a reasonable level and so I need some help getting to the bottom of why the request times are so slow.

This can cause the webapp to hang on post or put and generally leads to a bad user experience on my sites.

PS: I am more of a server newbie so this has confounded me for quite some time.

The domains:
collabornation.net
nptrainingworks.com
(they run off the same two webservers using vhost configs)

The Gear:
Two Rackspace 4 Gig servers running CentOS 6.2 Final
They have a mounted file system (gluster) that is used to keep files the same on both machines. They are behind a rackspace load balancer running round robin. Mysql is run using php-pdo and php-mysql as such mysql is run on another instance running memcache on that machine with phpMyAdmin located there as well.

Apache version number 2.2.15-15.el6.centos.1 (httpd.x86_64)
Varnish version number 3.0.2-1.el5 (varnish.x86_64)
PHP version number 5.3.14-1.el6.remi (php.x86_64)

Configs Linked Below

Apache Conf

Vhost Conf

Varnish Backends

Varnish Defaults

Varnish Acl

PHP INI

Again need some help, much appreciated!

Best Answer

Start looking into the database and connections to and from the database instance (hopefully a separate server) from the web server. You may be experiencing high wait times on your database, in which case you'll need to performance tune the database.