nginx日志统计分析

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了nginx日志统计分析相关的知识,希望对你有一定的参考价值。

 本文主要使用的是grep,awk,cut等工具来对nginx日志进行统计和分析,具体如下:


1,列出当天访问最多次数的ip地址

cut -d- -f 1 /usr/local/nginx/logs/20160329/access_2016032913.log |uniq -c | sort -rn | head -20 

[[email protected] 20160329]# cut -d- -f 1 /usr/local/nginx/logs/20160329/access_2016032913.log |uniq -c | sort -rn | head -20  
     69 180.116.214.31 
     45 180.116.214.31 
     45 180.116.214.31 
     36 49.80.54.111 
     35 183.206.185.204 
     35 180.116.214.31 
     32 49.80.54.111 
     32 49.80.54.111 
     32 180.116.214.31 
     31 117.136.45.101 
     29 180.116.214.31 
     28 218.205.19.112 
     28 180.116.214.31 
     28 180.116.214.31 
     27 49.80.54.111 
     27 222.185.248.242 
     24 49.80.54.111 
     24 175.0.8.161 
     23 49.80.54.111 
     23 49.80.54.111


2,查看某一个页面被访问的次数

[[email protected] 20160329]#grep "/index.php" log_file | wc -l


3,查看每一个IP访问了多少页面并排序

awk ‘{++S[$1]} END {for (a in S) print a,S[a]}‘ access_2016032913.log |uniq|sort -rn|more

[[email protected] 20160329]# awk ‘{++S[$1]} END {for (a in S) print a,S[a]}‘ access_2016032913.log |uniq|sort -rn|more
223.94.229.51 148
223.73.166.191 1
223.68.252.103 156
223.68.167.66 2
223.68.106.138 43
223.67.99.72 7
223.67.153.173 12
223.66.93.152 15
223.66.38.31 103
223.65.191.181 1
223.65.191.135 11
223.65.190.71 13
223.65.141.78 3
223.64.63.71 31
223.64.63.229 7
223.64.62.242 59
223.64.62.23 27
223.64.62.216 1
223.64.62.160 40
223.64.61.136 28
223.64.60.80 13
223.64.60.21 12
223.64.237.37 187
223.64.209.247 2
223.64.158.4 15

其中,sort -rn 按照数字从大到小排序,uniq 将重复行去除。


4,查看某一个ip访问了那些页面:grep ^xx.xx.xx.xx log_file |awk ‘{print $1,$7}‘

[[email protected] 20160329]# grep ^223.147.39.194 17 access_2016032913.log |awk ‘{print $1,$7}‘          grep: 17: No such file or directory
access_2016032913.log:223.147.39.194 //customer/customerInfo/getCustUnReadMsgInfo.json
access_2016032913.log:223.147.39.194 //customer/customerInfo/getCustUnReadMsgInfo.json
access_2016032913.log:223.147.39.194 //remind/redDot/checkRedDot.json
access_2016032913.log:223.147.39.194 //remind/redDot/checkRedDot.json
access_2016032913.log:223.147.39.194 //thirdpartyapi/appaction/app_action/action_send_batch.json
access_2016032913.log:223.147.39.194 //customer/customerInfo/getCustUnReadMsgInfo.json
access_2016032913.log:223.147.39.194 //customer/customerInfo/getCustUnReadMsgInfo.json
access_2016032913.log:223.147.39.194 //remind/redDot/checkRedDot.json
access_2016032913.log:223.147.39.194 //remind/redDot/checkRedDot.json
access_2016032913.log:223.147.39.194 //customer/customerInfo/getCustUnReadMsgInfo.json
access_2016032913.log:223.147.39.194 //customer/customerInfo/getCustUnReadMsgInfo.json
access_2016032913.log:223.147.39.194 //remind/redDot/checkRedDot.json
access_2016032913.log:223.147.39.194 //remind/redDot/checkRedDot.json
access_2016032913.log:223.147.39.194 //customer/customerInfo/getCustUnReadMsgInfo.json
access_2016032913.log:223.147.39.194 //customer/customerInfo/getCustUnReadMsgInfo.json
access_2016032913.log:223.147.39.194 //remind/redDot/checkRedDot.json
access_2016032913.log:223.147.39.194 //remind/redDot/checkRedDot.json

5,去掉搜索引擎统计当天的页面:awk ‘{print $12,$1}‘ access_2016032913.log | grep ^\"Mozilla | awk ‘{print $2}‘ |sort | uniq | wc -l      

[[email protected] 20160329]# awk ‘{print $12,$1}‘ access_2016032913.log | grep ^\"Mozilla | awk ‘{print $2}‘ |sort | uniq | wc -l      
35


6,查看一个小时内有多少ip访问:

[[email protected] 20160329]# awk ‘{print $4,$1}‘ access_2016032913.log | grep 29/Mar/2016:13 | awk ‘{print $2}‘| sort | uniq | wc -l   
1926


本文出自 “清风明月” 博客,请务必保留此出处http://liqingbiao.blog.51cto.com/3044896/1758009

以上是关于nginx日志统计分析的主要内容,如果未能解决你的问题,请参考以下文章

Shell脚本切割日志

多维度分析统计nginx访问日志

Nginx 日志分析及性能排查

Nginx 日志分析及性能排查

Nginx 日志分析及性能排查

Oracle数据库从RMAN备份集片段还原指定单个归档日志进行日志挖掘分析