Difference Between proxy_cache and fastcgi_cache in Nginx

If your box can handle the entire cache on its own, memcache will only slow you down. APC is shared memory. Used right it will blow away memcached. Nginx fast cgi cache will make all dynamic php pages scream. Even if you set the cache to just 10 seconds, this makes it such that that max hit rate to any given php page will be once every 10 seconds. Makes it impossible to crash a page with load. I run a bunch of websites on a single small box that gets millions of people per month. All you need is nginx caching and APC at this point.

Memcache comes into the picture only when you have to scale your cache beyond a single box.

You seem to have a few misconceptions which I feel needs to be addressed.

First of all, mod_php is only marginally faster, all my tests have shown that the difference is so minuscule that it's not worth factoring in. I also doubt that the security aspect is relevant to you as you seem to be looking at a dedicated server and mod_php really only has an advantage in a shared environment - in fact, in a dedicated environment php-fpm will have the advantage as PHP and your web server now runs as different processes, and that's not even factoring in the awesome logging options in php-fpm such as slow log.

If the world was black and white I'd say go with a pure nginx setup and compile php with php-fpm. More realistically if you already have Apache working then make nginx a reverse proxy to apache and you might save a few hours of setup time and the difference in performance will be tiny.

But lets assume the world is black and white for a second, because this makes for far more awesome setups. You do nginx + php-fpm for your web server. To solve the uploads you use the upload module and upload progress module for nginx. This means that your web server accepts the upload and passes the file path onto PHP when it's done, so that the file doesn't need to be streamed between nginx and PHP via fastcgi protocol, sweet. (I have this in a live setup and it's working great, btw!)

For user downloading you use nginxs x-send-file-like feature called x-accel-redirect, essentially you do your authentication in PHP and set a header which nginx picks up on and starts transfer that file. PHP ends execution and your web server is handling the transfer, sweet! (Again, I have this in a live setup and it's working great)

For distributing files across servers or other long running operations we realize that PHP isn't really best suited for this, so we install gearman, which is a job server that can distribute jobs between workers on different servers, these workers can be written in any language. Therefore you can create a distribute worker and spawn 5 of them using a total of 200 KB of memory instead of the 100 MB PHP would use. Sweet. (I also have this running live, so it's all actually possible)

In case you haven't picked up on it yet, I think many of your problems aren't related to your web server at all, you just think that way because Apache forces it to be related to your web server due to it's structure, often there are far better tools for the job than PHP and PHP is a language which knows this and provides excellent options to off-loading work without ever leaving PHP.

I'd highly recommend nginx, but I also think you should look at other options for your other problems, if you have a scaling or performance problem then feel free to write me. I don't know if you can send messages through here but otherwise write me at martin@bbtn.us as I don't stalk server fault for anything not tagged with nginx. :)