AWStats log format for tomcat access logs which has X-Forwarded-For

What you probably want to do here is to analyze all these logfiles once, then keep analyzing only the current logfiles from then on.

The simplest thing to do is unzip all those files into a single file, then have awstats run over it once, then point awstats at your access.log file from then on.

awstats normally has a script called logresolvemerge.pl, which can read the compressed files, and will merge them appropriately for awstats to do analsyis.

To merge all your existing ones, run

perl /usr/share/awstats/tools/logresolvemerge.pl /var/log/nginx/access.log* > /tmp/nginx.tmplog

This will probably take a while.

You can then have awstats run once over this file (set LogFile appropriately).

From then on, you should have awstats run over the most recent logfile - which is what your current configuration is doing.

Depending on how often you are running awstats vs rotating nginx logfiles, you may want to have it read both the current logfile and the previous one. (eg, if you rotate nginx logfiles every day at 12, but have awstats run every day at 1, then whenever awstats runs the logfile will only contain what's been written since the last rotation). You can use logresolvemerge.pl inside your LogFile command like this:

LogFile="/usr/share/awstats/tools/logresolvemerge.pl /var/log/nginx/access.log /var/log/nginx/access.log.1.gz |"

This tells awstats to run the logresolvemerge.pl command with the two logfiles as parameters, and awstats will read in the output of that script (that's what the pipe | does)

One possible issue is here:

log_format  main  '$remote_addr - $remote_user [$time_local]...

Corresponding configuration in AWStats:

LogFormat = "%host %logname %time1

And your log file contains:

1.1.1.1 - - [12/Mar/2017:07:23:53 +1300]

%logname matches to only a single string, that is, the username provided in HTTP authentication. Now, your log file contains two dashes, first one from your configuration, and the second one means an empty username.

So, AWStats tries to interpret the second dash as a timestamp, and that causes it to consider the record as failed.

So, you either need to add the dash to AWStats log format string, or remove the dash from nginx log format.

As a side note, you don't need to quote your last parameters ($request_time, $upstream_cache_status, $sent_http_content_encoding) in nginx log, since they cannot contain spaces.

You can also use %extraX in AWStats configuration if you want to use that information in building reports based on those facts.