Security – Apache access log full of unauthorized and suspicious requests, how to take action

The filters only need to be added to one config. Try removing what you added to apache2.conf and keep everything in /sites-available/default.

For example, if you want to filter out search bots from your logs, the end of your config file will look like:

    ErrorLog ${APACHE_LOG_DIR}/error.log

    # Possible values include: debug, info, notice, warn, error, crit,
    # alert, emerg.
    LogLevel warn

    # Filter all search bots from the access log.
    SetEnvIfNoCase User-Agent googlebot dontlog
    SetEnvIfNoCase User-Agent bingbot dontlog
    SetEnvIfNoCase User-Agent baiduspider dontlog

    # Custom Log location + options
    CustomLog ${APACHE_LOG_DIR}/access.log combined env=!dontlog
</VirtualHost>

Also, when you do apachectl graceful, it can take a while. If you're just testing the changes, delete the log files, do apachectl graceful, and when you see the log files get re-created, you'll know they are populated using the new filter options you specified.

I figured this out - apparently when you resize an instance on GCP it issues you a new IP address. I had to update Cloud DNS with the new IP addresses and everything worked fine. I think the resize process should have expressly called this out because I wasted a number of hours trying to figure out what was wrong. :-(