Logstash 之 Nginx日志收集分析

Posted 2021-09-04 wx592bc92b285c7

一、作用

通过收集nginx Access Log 日志，分析用户的访问量，请求状态，响应时间等数据

二、设置Nginx日志为json格式（便于logstash的快速收集）

log_format access_json \'{"@timestamp":"$time_iso8601",\'
                           \'"host":"$server_addr",\'
                           \'"clientip":"$remote_addr",\'
                           \'"size":$body_bytes_sent,\'
                           \'"responsetime":$request_time,\'
                           \'"upstreamhost":"$upstream_addr",\'
                           \'"http_host":"$host",\'
                           \'"url":"$request",\'
                           \'"domain":"$host",\'
                           \'"referer":"$http_referer",\'
                           \'"status":"$status"}\';
access_log  /var/log/nginx/access.log  access_json;

三、配置logstash的日志收集配置文件

input {
    file {
        path => ["/var/log/nginx/access.log"]
        type => "ansible-nginx"
        codec => "json"
    }
}
output {
  elasticsearch {
        hosts => ["http://10.224.14.14:9200","http://10.224.14.15:9200","http://10.224.14.16:9200"]     #elasticsearch集群主机
        index => "logstash-%{type}-%{+YYYY.MM.dd}"           #索引的名称，用日期切割，减 少单个索引大小，同时对过旧的索引数据可删除
    }
}

四、运行logstash监听NGINX日志信息

/usr/share/logstash/bin/logstash -f /usr/share/logstash/ansible-nginx.conf &

五、在Kibana上创建相关索引