If i may add a personal opinion..
Use Nginx, its faster, simpler, and much easier to do what you want.. ( and uses much less memory )
Its a personal preference of mine.. :D
https://calomel.org/nginx.html is a nice detailed example
In a word, history. This is a very abbreviated summary of 15+ years of history (you can find out more on the Internet, if you are really interested):
First, while PHP has all the features of a programming language, this wasn't always so. It started out as a "hypertext preprocessor" whose purpose was to be embedded in HTML pages and parsed by a CGI program or the web server itself. It was meant to be run by an existing web server, not to be a web server. In its early iterations it was a very simple language (and some would say it still is).
PHP dates back to the late 1990s, and the only way for a web server to run dynamic content back then was through CGI. CGI's big problem, though, was that it was rather slow, because it forked a new process for every request. Not a big deal in the early days, but when the dot-com boom hit, it became something of an issue. PHP became popular enough that it was embedded into Apache as a loadable module, which gave better performance than CGI and offered some other benefits.
For the longest time, PHP was only runnable in one of those two ways, but since it had become the language with the vast majority of market share, nobody really cared much. Though eventually some people wanted to use servers other than Apache, and for a while CGI was the only way to do that, until a FastCGI server API known as php-fpm (FastCGI Process Manager) was eventually contributed into PHP. Unlike CGI, FastCGI keeps a pool of processes always running and ready to serve incoming requests.
In the 2000s came other languages, which do things differently. For instance, in Ruby application libraries are shipped in what are called gems, and a program can easily be created simply by gluing together some existing gems with business logic. Rack is a Ruby gem which provides a Ruby API for building web servers, and servers such as mongrel, unicorn, thin, etc., and frameworks such as Rails and Sinatra, are built on top of it.
Other languages such as Python and Java have similar web servers and web server frameworks. As you can see this is a completely different approach from the one taken by PHP, which generally doesn't use HTTP at all to serve requests.
However, recent versions of PHP now have a "built in web server", but it is relatively new and can only handle one connection at a time, making it unsuitable for production use. It's explicitly designed for developer use.
Ultimately it comes down to this: PHP was designed to work in the context of an existing HTML document, while other languages such as Python, Ruby, Java, etc., are general-purpose. The only other web language I know of which works like PHP in this respect is Microsoft's "classic" ASP, which has a similar design.
Best Answer
Generally speaking I'm a fan of divided architectures: A DB server farm, a "public www" server farm (commercial web site), and an app server farm - possibly with other layers of middleware as needed.
The logic behind this is old -- basically that the public www server doesn't need to have access to any sensitive data, so some curious hacker poking around at www.mycompany.com might compromise our marketing site, but they didn't get anything else.
Here are a few other general ideas...
Re: backroom network load, if we're talking about traffic load from contacting your FastCGI hosts I would say sticking a web server on those machines and letting them talk to the outside world directly may be a good idea.
One thing to bear in mind is maintaining isolation/protection of your database in the event of a security compromise, which could be an argument for pushing your FastCGI stuff onto the web servers and writing something more lightweight to talk to your database...
(If we're talking about something else let me know and I'll take a swing at it :-)
Re: the static files issue, there are a number of ways to solve this one.
Put a copy of static data on every server that needs to send it.
Let's call this the "Disk is cheap" method -- It keeps you from dumping a huge amount of load on a single box hosting your static content, and eliminates that single point of failure. The downside is that you have to synchronize that content somehow (deployment scripts, cron'd rsync, cvs/git/svn, etc.)
Install a caching infrastructure on your front-end servers
This is a similar solution to "Disk is cheap" above, only you have one back-end server with the static content, and when the front-end servers need it they cache a copy for $LIFETIME, thus eliminating the need for synchronization scripts (the caching infrastructure does it for you).
Put your static content on a content delivery network
This really only works if you're not doing SSL stuff -- A commercial CDN solves the load problem and gives users geographically distributed points from which they can grab your content. The downside is that most browsers will pitch a fit if you do this with a site that has SSL unless your CDN is also secured (even then paranoid browsers should rightly complain).