I am exploring the potential scenario for Varnish/Nginx/Apache implementations and have a few questions about this setup.
I am looking to use all three in conjunction, not to replace Apache by Nginx.
At the moment I am not sure whether if Varnish should be put in front of Nginx, which will serve static content and pass dynamic content to Apache or put it between Nginx and Apache?
Should I enable Nginx cache or even enable it and not use Varnish at all?
I have a number scripts which process stats based on Apache logs, what can I do to let my scripts process logs in whole, not just Apache's logs which will hold only dynamic request stats?
How do I make Varnish pass access stats to Apache/bginx since access to cached files will not be logged by Apache/Nginx?
Best Answer
It really depends on the content of the site in question and the caching scheme you're using.
I looked at this scenario previously for a very high traffic site (1M+ uniques daily) and we ended up using Nginx and Apache, without Varnish. This was due to existing caching methods and the amount of dynamic content on page so we would have only been able to have Varnish cache images and static files like css and js. During testing it became an Nginx or Varnish question, since like you we wanted to keep Apache in the configuration. The benchmarking we did showed Nginx performed faster than Varnish at high volume so thats the way we went.
One thing that we could've done but didn't would be to load dynamic page sections in a separate request and then insert the content in browser, that would have allowed us to use Varnish to cache more objects, while passing the dynamic elements to Apache and serving static content via Nginx on cache misses.
As far as the logging issue, you will most likely need to write scripts that will parse/merge your logs together and then you can run your stats scripts against the merge logs. I believe there are some good log merging tools out there, but can' think of one at the moment.