Apache2 Referer Log – filtering the own server referers

apache-2.2

I have a custom log file that displays the referer, requested server/url, and the user agent:

LogFormat "\"%{Referer}i\" -> \"%v%U\" \"%{User-Agent}i\"" referer2

This is working great, almost too great. It dutifully reports any requests for images, css files, javascript, etc. etc. Now I have apache2 serving several domains so I don't believe writing a filter against the domain names is very wise, as they tend to come and go.

"-" -> "http://www.example.com" "Firefox"
"http://www.example.com" -> "http://www.example.com/foo.css" "Firefox"
"http://www.example.com" -> "http://www.example.com/foo.jpg" "Firefox"

What would be the best way to filter out referer requests from example.com which may also be known as other.example.com, or foobar.com, or what-is-he-thinking.com, but all originate from the same IP? Is this possible without having to resolve the IP address of each request?

Best Answer

You may want to look a little more at the docs for the CustomLog directive: http://httpd.apache.org/docs/current/mod/mod_log_config.html#customlog

I assume you've set a CustomLog for referer2. You can also append "env=" as a test to see if something will even trigger logging. Combined with a SetEnvIf directive, you can control whether or not anything gets logged to your CustomLog.

Here's an example from Apache's doc that only logs to the "gif-requests.log" file if the uri ends in .gif:

SetEnvIf Request_URI \.gif$ gif-image
CustomLog gif-requests.log common env=gif-image
CustomLog nongif-requests.log common env=!gif-image
Related Topic