ELK收集nginx日志

Posted 2021-12-29 青衫解衣

json格式化nginx日志

备注:key是自定义的

    log_format log_json "@timestamp": "$time_local", 
                        "remote_addr": "$remote_addr", 
                        "referer": "$http_referer", 
                        "request": "$request", 
                        "status": $status, 
                        "bytes": $body_bytes_sent, 
                        "agent": "$http_user_agent", 
                        "x_forwarded": "$http_x_forwarded_for", 
                        "up_addr": "$upstream_addr",
                        "up_host": "$upstream_http_host",
                        "up_resp_time": "$upstream_response_time",
                        "request_time": "$request_time"
                         ;
                        
    access_log  /var/log/nginx/access.log  log_json;

重启服务

[root@es1 <sub>]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[root@es1 </sub>]# vim /etc/nginx/nginx.conf 
[root@es1 ~]# systemctl start nginx

配置logstash

input里面可以嵌套多个file收集；output也可以嵌套多个if判断，输出到不同的es

[root@es1 conf.d]# cat nginx.conf 
input 
  file 
  type => "nginxlog"
  path => "/var/log/nginx/access.log"
  start_position => "beginning"
  stat_interval => 5
  codec => "json"
  
  file 
  type => "messageslog"
  path => "/var/log/messages"
  start_position => "beginning"
  stat_interval => 5
  



output 
  if [type] == "nginxlog" 
  elasticsearch 
  hosts => ["192.168.1.9:9200"]
  index => "logstash-nginx-log-%+YYYY.MM.dd"
    
  
  if [type] == "messageslog" 
  elasticsearch 
  hosts => ["192.168.1.10:9200"]
  index => "logstash-system-log-%+YYYY.MM.dd"

配置kibana：

[logstash-nginx-log]-YYYY.MM.DD

刷新kibana索引：

目的是第一配置缺少参数需要kibana重新加载下索引。