elk-logstash-kibana

Posted 2021-01-01 zhaojingyu

一、修改logstash.yml

 unzip logstash-6.3.2

vim config/logstash.yml

#添加：检查所有ip

http.host: "0.0.0.0"

二、在logstash下的config下创建（根据自己log日志地址设置：安装nginx等）

[[email protected] config]$ cat syslog.conf 
input {
  syslog {
    type => "system-syslog"
    port => 10514
  }
}
output {
  elasticsearch {
    hosts => ["139.199.124.136:9200"]  # 定义es服务器的ip
    index => "system-syslog-%{+YYYY.MM.dd}" # 定义索引
  }
}
input {
  file {  # 指定一个文件作为输入源
    path => "/home/elk/nginx/logs/elk_access.log"  # 指定文件的路径
    start_position => "beginning"  # 指定何时开始收集
    type => "nginx"  # 定义日志类型，可自定义
  }
}
filter {  # 配置过滤器
    grok {
        match => { "message" => "%{IPORHOST:http_host} %{IPORHOST:clientip} - %{USERNAME:remote_user} [%{HTTPDATE:timestamp}] "(?:%{WORD:http_verb} %{NOTSPACE:http_request}(?: HTTP/%{NUMBER:http_version})?|%{DATA:raw_http_request})" %{NUMBER:response} (?:%{NUMBER:bytes_read}|-) %{QS:referrer} %{QS:agent} %{QS:xforwardedfor} %{NUMBER:request_time:float}"}  # 定义日志的输出格式
    }
    geoip {
        source => "clientip"
    }
}
output {
    stdout { codec => rubydebug }
    elasticsearch {
        hosts => ["139.199.124.136:9200"]
        index => "nginx-test-%{+YYYY.MM.dd}"
  }
}

nginx日志编译二进制：

nginx添加：

    log_format  main  ‘$remote_addr - $remote_user [$time_local] "$request" ‘
                      ‘$status $body_bytes_sent "$http_referer" ‘
                      ‘"$http_user_agent" "$http_x_forwarded_for"‘;

    access_log  logs/access.log  main;

三、vim   kibana.yml

tar xf kibana-6.3.2-linux-x86_64.tar.gz

#vim kibana-6.3.2-linux-x86_64/config/kibana.yml

elasticsearch.url: "http://139.199.124.136:9200"    #ela地址
server.host: 0.0.0.0
server.port: 5601
logging.dest: /elk/logs/kibana.log  #日志存放目录，自定义　

四、配置kibana服务器的ip以及配置的监听端口：

vim /etc/rsyslog.conf

*.* @@192.168.77.130:10514

重启rsyslog

systemctl restart rsyslog

-----------启动服务查看

[[email protected] ~]# netstat -lntup|egrep ‘9600|5601‘

tcp        0      0 0.0.0.0:9600                0.0.0.0:*                   LISTEN      5076/java           
tcp        0      0 0.0.0.0:5601                0.0.0.0:*                   LISTEN      5073/node 

五、启动服务汇总

#cat /etc/rc.local
/home/elk/nginx/sbin/nginx
su -c ‘/home/elk/elasticsearch-6.3.2/bin/elasticsearch -d >>/dev/null 2>&1 &‘ elk
#su -c ‘cd /home/elk/elasticsearch-head-master;grunt server >>/dev/null 2>&1 &‘ elk
su -c ‘/home/elk/kibana-6.3.2-linux-x86_64//bin/kibana  >>/dev/null 2>&1 &‘ elk
su -c ‘/home/elk/logstash-6.3.2/bin/logstash -f /home/elk/logstash-6.3.2/config/syslog.conf  >>/dev/null 2>&1 &‘ elk
#su -c ‘/home/elk/filebeat-6.3.2-linux-x86_64/filebeat &‘ elk