ELK 5.5.2 分布式日志实战

Posted 2020-12-30 woodylau

一.  ELK 分布式日志实战介绍

　　此实战方案以 Elk 5.5.2 版本为准，分布式日志将以下图分布进行安装部署以及配置。

二. Filebeat 插件安装以及配置

1.下载Filebeat插件 5.5.2 版本

wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-5.5.2-linux-x86_64.tar.gz

2.解压filebeat-5.5.2-linux-x86_64.tar.gz文件至/tools/elk/目录下

1 tar -zxvf filebeat-5.5.2-linux-x86_64.tar.gz -C /tools/elk/
2 cd /tools/elk/
3 mv filebeat-5.5.2-linux-x86_64 filebeat-5.5.2

3.配置filebeat.yml文件

1 cd /tools/elk/filebeat-5.5.2
2 vi filebeat.yml

　　filebeat.yml 配置

 1 filebeat.prospectors:
 2 - input_type: log
 3   paths:
 4     - /data/applog/app.info.log
 5   encoding: utf-8
 6   document_type: app-info
 7   fields:
 8       type: app-info
 9   fields_under_root: true
10   scan_frequency: 10s
11   harvester_buffer_size: 16384
12   max_bytes: 10485760
13   tail_files: true
14 
15 - input_type: log
16   paths:
17     - /data/applog/app.error.log
18   encoding: utf-8
19   document_type: app-error
20   fields:
21       type: app-error
22   fields_under_root: true
23   scan_frequency: 10s
24   harvester_buffer_size: 16384
25   max_bytes: 10485760
26   tail_files: true
27 
28 output.kafka:
29   enabled: true
30   hosts: ["192.168.20.21:9092","192.168.20.22:9092","192.168.20.23:9092"]
31   topic: elk-%{[type]}
32   worker: 2
33   max_retries: 3
34   bulk_max_size: 2048
35   timeout: 30s
36   broker_timeout: 10s
37   channel_buffer_size: 256
38   keep_alive: 60
39   compression: gzip
40   max_message_bytes: 1000000
41   required_acks: 1
42   client_id: beats
43   partition.hash:
44      reachable_only: true
45 logging.to_files: true