For years we have run our own homebrew web stats application out of frustration with currently available tools. Well, web stats packages have certainly improved over the years, and we decided to implement AWStats on one of our servers. We are doing this on a Red Hat Enterprise 4 box with Apache 2, so some items may be specific, but this is mostly generic. The first thing to adjust is the log format. We used the standard combined log format:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
|
This is a line in httpd.conf. Note that your virtual hosts area needs to define combined as the log format as well. For instance:
ErrorLog logs/example.com-error_log CustomLog logs/example.com-access_log combined |
Restart Apache to make the changes active, and get rid of the old format entries if you are changing:
[root@webserver]# cat /dev/null > /path/to/apache/logs/access_log |
If you do this at the beginning, you will be able to hit your logs with a few entries by the time you finish configuring. We used the RPM of AWStats and the default installer:
[root@webserver awstats]# tools/awstats_configure.pl ----- AWStats awstats_configure 1.0 (build 1.6) (c) Laurent Destailleur ----- This tool will help you to configure AWStats to analyze statistics for one web server. You can try to use it to let it do all that is possible in AWStats setup, however following the step by step manual setup documentation (docs/index.html) is often a better idea. Above all if: - You are not an administrator user, - You want to analyze downloaded log files without web server, - You want to analyze mail or ftp log files instead of web log files, - You need to analyze load balanced servers log files, - You want to 'understand' all possible ways to use AWStats... Read the AWStats documentation (docs/index.html). -----> Running OS detected: Linux, BSD or Unix -----> Check for web server install Found Web server Apache config file '/path/to/apache/conf/httpd.conf' -----> Check and complete web server config file '/path/to/apache/conf/httpd.conf' Add 'Alias /awstatsclasses "/usr/local/awstats/wwwroot/classes/"' Add 'Alias /awstatscss "/usr/local/awstats/wwwroot/css/"' Add 'Alias /awstatsicons "/usr/local/awstats/wwwroot/icon/"' Add 'ScriptAlias /awstats/ "/usr/local/awstats/wwwroot/cgi-bin/"' Add '' directive AWStats directives added to Apache config file. -----> Update model config file '/etc/awstats/awstats.model.conf' File awstats.model.conf updated. -----> Need to create a new config file ? Do you want me to build a new AWStats config/profile file (required if first install) [y/N] ? y -----> Define config file name to create What is the name of your web site or profile analysis ? Example: www.mysite.com Example: demo Your web site, virtual server or profile name: > www.example.com -----> Define config file path In which directory do you plan to store your config file(s) ? Default: /etc/awstats Directory path to store config file(s) (Enter for default): > -----> Create config file '/etc/awstats/awstats.www.example.com.conf' Config file /etc/awstats/awstats.www.example.com.conf created. -----> Add update process inside a scheduler Sorry, configure.pl does not support automatic add to cron yet. You can do it manually by adding the following command to your cron: /usr/local/awstats/wwwroot/cgi-bin/awstats.pl -update -config=www.example.com Or if you have several config files and prefer having only one command: /usr/local/awstats/tools/awstats_updateall.pl now Press ENTER to continue... A SIMPLE config file has been created: /etc/awstats/awstats.www.example.com.conf You should have a look inside to check and change manually main parameters. You can then manually update your statistics for 'www.example.com' with command: > perl awstats.pl -update -config=www.example.com You can also read your statistics for 'www.example.com' with URL: > http://localhost/awstats/awstats.pl?config=www.example.com Press ENTER to finish... |
The main changes we needed to make in /etc/awstats/awstats.www.example.com.conf were:
LogFile="/path/to/apache/logs/example.com-access_log" PurgeLogFile=1 |
This specifies the Apache access log file and purges that file after awstats is run. Watch out for the Alias directives that the configure program adds to httpd.conf. These can be quite dangerous. We took all of them out and copied the files manually. Generally, be very aware of these Alias directives for other programs as well. What they do is tie requests for directories off of root on your domains to some directory that isn’t in your web root. To make matters worse, the Alias is global, not specific to a single virtual domain. Next, we need to set up the directory that houses the database and run an update to make sure all is in place and happy:
[root@webserver cgi-bin]# mkdir /var/lib/awstats [root@webserver cgi-bin]# perl awstats.pl -update -config=www.example.com Update for config "/etc/awstats/awstats.www.example.com.conf" With data in log file "/path/to/apache/logs/example.com-access_log"... Phase 1 : First bypass old records, searching new record... Searching new records from beginning of log file... Phase 2 : Now process new records (Flush history on disk after 20000 hosts)... Jumped lines in file: 0 Parsed lines in file: 536 Found 0 dropped records, Found 0 corrupted records, Found 0 old records, Found 536 new qualified records. |
Looks good. AWStats has the option to generate the pages on the fly using CGI; however, for security reasons, we like the idea of static pages. The awstats_buildstaticpages.pl script is used for just this purpose:
[root@webserver conf]# perl /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=www.example.com -dir=/path/towwwdocs/awstatsfiles/ Launch update process : "/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -config=www.example.com -update -configdir= . . 20 files built. Main HTML page is 'awstats.www.example.com.html'. |
We couldn’t find any css stuff that was used with the static pages, so we just copied over the icon directory from /usr/local/awstats/wwwroot, and that seemed to work fine:
[root@webserver wwwroot]# cp -rf icon /path/towwwdocs/awstatsicons/ |
A good time to update stats is at midnight, both for load, and to get the stats to match full days. We added the build script to our crontab:
[root@webserver conf]# crontab -l 0 0 * * * /usr/bin/perl /usr/local/awstats/tools/awstats_buildstaticpages.pl -update -config=www.example.com -dir=/path/towwwdocs/awstatsfiles/ > /dev/null 2>&1 |
