astats日志分析系统

Posted

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了astats日志分析系统相关的知识，希望对你有一定的参考价值。

Awstats是一个免费非常简洁而且强大有个性的网站日志分析工具。

功能：

一：访问量，访问次数，页面浏览量，点击数，数据流量等
二：精确到每月、每日、每小时的数据
三：访问者国家
四：访问者IP
五：Robots/Spiders的统计
六：访客持续时间
七：对不同Files type 的统计信息
八：Pages-URL的统计
九：访客操作系统浏览器等信息
十：其它信息（搜索关键字等等）
 
1.安装httpd
[[email protected] /]# tar zxvf httpd-2.2.17.tar.gz -C /usr/src/
[[email protected] /]# unzip awstats-7.0.zip
[[email protected] /]# ./configure --prefix=/usr/local/httpd --enable-so --enable-rewrite --enable-charset-lite --enable-cgi && make && make install
  • --prefix：指定安装目录
  • --enable-so：启用动态加载模块支持
  • --enable-rewrite：启用网站重写功能
  • --enable-charset-lite：启动字符集支持
  • --enable-cgi：启用CGI脚本程序支持 
    [[email protected] httpd-2.2.17]# ln -s /usr/local/httpd/bin/* /usr/local/bin/            //优化执行路径
[[email protected] ~]# cp /usr/local/httpd/bin/apachectl /etc/init.d/httpd
[[email protected] ~]# vim /etc/init.d/httpd
#chkconfig: 35 85 15
#description: Apache HTTP Server

 

[[email protected] ~]# chkconfig --add httpd 
[[email protected] ~]# chkconfig --level 35 httpd on
[[email protected] ~]# vim /usr/local/httpd/conf/httpd.conf
ServerName www.aaa.com:80                    //添加网站域名
[[email protected] ~]# httpd –t                            //检查配置文件错误
Syntax OK
[[email protected] ~]# /etc/init.d/httpd start
[[email protected] ~]# netstat -anpt |grep 80
tcp        0      0 :::80                       :::*                        LISTEN      54881/httpd
[[email protected] ~]# elinks http://192.168.1.200
It works!

基于不同的域名创建虚拟Web主机

为两个站点创建网站的文档目录

[[email protected] ~]# mkdir /usr/local/httpd/htdocs/aaacom
[[email protected] ~]# mkdir /usr/local/httpd/htdocs/bbbcom
[[email protected] ~]# echo ‘<h1>www.aaa.com</h1>‘ > /usr/local/httpd/htdocs/aaacom/index.html
[[email protected] ~]# echo ‘<h1>www.bbb.com</h1>‘ > /usr/local/httpd/htdocs/bbbcom/index.html
修改配置文件httpd.conf和httpd-vhosts.conf
[[email protected] ~]# cd /usr/local/httpd/conf/extra/
[[email protected] extra]# vim httpd-vhosts.conf
NameVirtualHost 192.168.1.200:80

<VirtualHost 192.168.1.200:80>
    DocumentRoot "/usr/local/httpd/htdocs/aaacom/"
    ServerName www.aaa.com
    ErrorLog "logs/www.aaa.com-error_log"
    CustomLog "logs/www.aaa.com-access_log" common
</VirtualHost>

<VirtualHost 192.168.1.200:80>
    DocumentRoot "/usr/local/httpd/htdocs/bbbcom/"
    ServerName www.bbb.com
    ErrorLog "logs/www.bbb.com-error_log"
    CustomLog "logs/www.bbb.com-access_log" common
</VirtualHost>
[[email protected] extra]# cd ..
[[email protected] conf]# vim httpd.conf
Include conf/extra/httpd-vhosts.conf                //将此行前面的＂＃＂去掉
[[email protected] ~]# vim /etc/hosts
192.168.1.200   www.aaa.com www.bbb.com
[[email protected] ~]# /etc/init.d/httpd restart
[[email protected] ~]# elinks http://www.aaa.com
www.aaa.com
[[email protected] ~]# elinks http://www.bbb.com
www.bbb.com

二、

部署安装awstats日志分析系统

[[email protected] /]# unzip awstats-7.0.zip 
[[email protected] /]# mv awstats-7.0 /usr/local/awstats 
[[email protected] /]# cd /usr/local/awstats/tools/
[[email protected] tools]# ./awstats_configure.pl
----- AWStats awstats_configure 1.0 (build 1.9) (c) Laurent Destailleur -----
This tool will help you to configure AWStats to analyze statistics for
one web server. You can try to use it to let it do all that is possible
in AWStats setup, however following the step by step manual setup
documentation (docs/index.html) is often a better idea. Above all if:
- You are not an administrator user,
- You want to analyze downloaded log files without web server,
- You want to analyze mail or ftp log files instead of web log files,
- You need to analyze load balanced servers log files,
- You want to ‘understand‘ all possible ways to use AWStats...
Read the AWStats documentation (docs/index.html).

-----> Running OS detected: Linux, BSD or Unix

-----> Check for web server install

Enter full config file path of your Web server.
Example: /etc/httpd/httpd.conf
Example: /usr/local/apache2/conf/httpd.conf
Example: c:\Program files\apache group\apache\conf\httpd.conf
Config file path (‘none‘ to skip web server setup):
> /usr/local/httpd/conf/httpd.conf				//httpd主配置文件路径

-----> Check and complete web server config file ‘/usr/local/httpd/conf/httpd.conf‘
Warning: You Apache config file contains directives to write ‘common‘ log files
This means that some features can‘t work (os, browsers and keywords detection).
Do you want me to setup Apache to write ‘combined‘ log files [y/N] ? y		//设置日志文件类型
  Add ‘Alias /awstatsclasses "/usr/local/awstats/wwwroot/classes/"‘
  Add ‘Alias /awstatscss "/usr/local/awstats/wwwroot/css/"‘
  Add ‘Alias /awstatsicons "/usr/local/awstats/wwwroot/icon/"‘
  Add ‘ScriptAlias /awstats/ "/usr/local/awstats/wwwroot/cgi-bin/"‘
  Add ‘<Directory>‘ directive
  AWStats directives added to Apache config file.

-----> Update model config file ‘/usr/local/awstats/wwwroot/cgi-bin/awstats.model.conf‘
  File awstats.model.conf updated.

-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
file (required if first install) [y/N] ? y					//设置站点配置文件

-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
> www.aaa.com									//统计的站点名称
往后默认配置按回车

 

修改awstats.www.aaa.com.conf配置文件

[[email protected] ~]# mkdir /var/lib/awstats
[[email protected] ~]# vim /etc/awstats/awstats.www.aaa.com.conf
LogFile="/usr/local/httpd/logs/access_log"
DirData="/var/lib/awstats"

 

创建数据库的存放目录，并更新数据库

[email protected] ~]# /usr/local/awstats/tools/awstats_updateall.pl now			//执行更新脚本
Running ‘"/usr/local/awstats/wwwroot/cgi-bin/awstats.pl" -update -config=www.aaa.com -configdir="/etc/awstats"‘ to update config www.aaa.com
Create/Update database for config "/etc/awstats/awstats.www.aaa.com.conf" by AWStats version 7.0 (build 1.971)
From data in log file "/usr/local/httpd/logs/access_log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Jumped lines in file: 0
Parsed lines in file: 2
 Found 0 dropped records,
 Found 0 comments,
 Found 0 blank records,
 Found 2 corrupted records,
 Found 0 old records,
 	 Found 0 new qualified records.

 

创建周期性计划任务

[[email protected] ~]# crontab -e
*/5 * * * * /usr/local/awstats/tools/awstats_updateall.pl now

创建网站跳转页

[[email protected] /] vim aws.html
<html>
<head>
<meta http-equiv=refresh content="0;
url=http://www.aaa.com/awstats/awstats.pl?config=www.aaa.com">
</head>
<body><body>
</html>
[[email protected] /]# mv aws.html /usr/local/httpd/htdocs/        //将跳转页移动到网站根目录下
[[email protected] ~]# /etc/init.d/httpd restart
重启apache访问awstats主页

为AWSstats进行访问权限的控制

客户机地址限制

[[email protected] /]# vim /usr/local/httpd/conf/httpd.conf
<Directory "/usr/local/awstats/wwwroot">
    Options None
    AllowOverride None
    Order deny,allow
    Deny from 192.168.1.234                //拒绝IP为192.168.1.234的主机访问
</Directory>
[[email protected] /]# /etc/init.d/httpd restart

此时192.168.1.234客户机将无法访问www.aaa.com/aws.html

用户身份认证限制

首先使用htpasswd工具创建用户认证数据文件

[[email protected] /]# htpasswd -c /usr/local/httpd/conf/.awspwd xuhao
New password: 
Re-type new password: 
Adding password for user xuhao
[[email protected] /]# cat /usr/local/httpd/conf/.awspwd 
xuhao:eE3EHFDzN4A5c
修改httpd.conf配置文件，添加用户授权
[[email protected] /]# vim /usr/local/httpd/conf/httpd.conf
<Directory "/usr/local/awstats/wwwroot">
    Options None
    AllowOverride None
    Order allow,deny
    Allow from all
    AuthName "AWStats Directory"                //定义受保护的领域名称
    AuthType Basic                            //设置认证的类型
    AuthUserFile /usr/local/httpd/conf/.awspwd        //设置用于保存用户账号、密码文件的路径
    require valid-user                            //合法用户才能访问
</Directory>
[[email protected] /]# /etc/init.d/httpd restart
    登录客户端访问测试

 

以上是关于astats日志分析系统的主要内容，如果未能解决你的问题，请参考以下文章

python分析apache和nginx日志文件输出访客ip列表的代码

argparse 代码片段只打印部分日志

常用python日期日志获取内容循环的代码片段

App性能：TraceView分析法

我的Android进阶之旅NDK开发之在C++代码中使用Android Log打印日志，打印出C++的函数耗时以及代码片段耗时详情

大数据综合项目--网站流量日志数据分析系统(详细步骤和代码)