Webalizer causing high CPU load

What is Webalizer: (for those wondering)

Webalizer is a fast, free web server log file analysis program, written in C, which produces highly detailed, easily configurable usage reports in HTML format. It was written to solve several problems with currently available analysis packages.

Webalizer Features:

• It is written in C to be extremely fast and highly portable. On a 200 MHz Pentium machine, over 10,000 records can be processed in one second, with a 40 Megabyte file taking roughly 15 seconds (over 150,000 records).
• Supports standard Common log file Format server logs. In addition, several variations of the combined log file Format are supported, allowing statistics to be generated for referring sites and browser types as well. Now also has native support for wu-ftpd xferlog FTP and squid log formats as well.
• Generated reports can be configured from the command line, or by use of one or more configuration files. Detailed information on configuration options can be found in the README file, supplied with all distributions.
• Supports multiple languages. Currently, Catalan, Chinese (traditional and simplified), Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, Galician, German, Greek, Hungarian, Icelandic, Indonesian, Italian, Japanese, Korean, Latvian, Malay, Norwegian, Polish, Portuguese (Portugal and Brazil), Romanian, Russian, Serbian, Slovak, Slovene, Spanish, Swedish, Turkish and Ukrainian are available.
• Unlimited log file sizes and partial logs are supported, allowing logs to be rotated as often as needed, and eliminating the need to keep huge monthly files on the system.
• Distributed under the GNU General Public License, complete source code is available as well as binary distributions for some of the more popular platforms

Webalizer Installation:

There are a few ways to install Webalizer - such as apt-get, yum, and of course from source.

installing via apt-get or yum is as simple as just running the command

yum -y install webalizer or apt-get install webalizer -y

Installing through sourcecode

1. First we need to download the webalizer sourcode first:

   wget ftp://ftp.mrunix.net/pub/webalizer/webalizer-2.23-03-src.tgz

2. Now lets uncompress it

   tar zxvf webalizer*

3. Makesure you have gcc compiler and Gd library installed, you can installed it using yum

   yum install gcc gcc-c++ gd-devel

4. Now go into the webalizer* directory , compile and install it

   cd webalizer* ./configure make make install

It is installed and you can checked the version now

webalizer --version
Webalizer V2.23-03 ********other stuff shows here***** 

WEBALIZER CONFIGURATION (single domain)

1. Create a central directory for the webalizer configuration files

   mkdir /etc/webalizer

2. Create two webalizer configuration files, a.example.conf from the sample file and put it into /etc/webalizer directory. You need to locate the webalizer.conf.sample files and copy it to /etc/webalizer directory

   cp /usr/local/etc/webalizer.conf.sample /etc/webalizer/a.example.com.conf

3. Modify LogFile, OutputDir and HostName of the webalizer config files. For example, fora.example.com.conf

   vi /etc/webalizer/a.example.com.conf

and changes the content

LogFile /var/log/httpd/access_log
into
LogFile /var/log/httpd/access_log_1  # it depend on your httpd access_log you can search it on /var/log/httpd/ directory 
OutputDir /var/www/usage/a
into
OutputDir  /home/praetorian-id.org/public_html/webalizer  # it depend on your apache virtual directory that you set on httpd.conf (/etc/httpd/conf/httpd.conf)
HostName       localhost
into
HostName       yourhostname.org  # it depend on your web hostname

Note:You may want to specify other settings specific to the domain, such as HideReferrer,HideSite, etc.

4. To process all the virtual sites, run the following command:

   for i in /etc/webalizer/*.conf; do webalizer -c $i; done

Now you can see the webalizer files on your site, example :

http://domainname.org/webalizer/usage_20100811.html

How to set Webalizer on multiple virtual domain :

Tha above tutorial are to set the webalizer for singel domain, how about if you needed to create webalizer for multiple domain ? , you need to makesure that you have create custom log on your every virtual domain setting on httpd.conf it is usually on /usr/local/apache/conf/httpd.conf , here is the eample :

<VirtualHost 34.14.212.39>
    ServerName test.com
    ServerAlias www.test.com
    DocumentRoot "/home/domains/test.com/htdocs"
    CustomLog "/home/domains/test.com/logs/access_log" common
</VirtualHost>

as you can see above, we have set the custom log on /home/domains/test.com/logs/access_log , please also make sure you have create the /home/domains/test.com/logs directory and set the permission was set for domains users

-rw-r--r-- 1 domains users 677485 Dec 1 14:45 access_log then set the log path on your webalizer conf on /etc/webalizer/

# vi /etc/webalizer/test.conf

LogFile /home/domains/test.com/logs/access_log then set it per virtual domain that you have :)

and last thing , restart the httpd , if you are using rpm

# service httpd restart

You should see the size of the access_logs on /home/domains/test.com/logs/access_log increase , if not checked the permission and webalizer config on /etc/webalizer/test.com.conf

Do not forget to add cron so the webalizer can be udpated automaticly

crontab -e

add

1 * * * * /root/dowebalizer >>/dev/null 2>&1

i will set the crontab every 1 hour to execute the shell script on /root/dowebalizer

Now create the shell script on /root/dowebalizer

vi /root/dowebalizer

enter the following:

#!/usr/bin/perl
use strict;

my @files = </etc/webalizer/*.conf>;

foreach my $file (@files) {
        chomp($file);
        system("/usr/local/bin/webalizer -c $file");
}

hit escape and save with w!

did this help you? if so - leave a comment, grade it - and tag it :-)

Configure auditd to monitor this file. It will tell you who accessed it.