五分钟带你玩转docker（十三）实战elk环境——logstash搭建

Posted 2021-07-15 小黄鸡1992

1.下载镜像

 docker pull logstash:7.5.1

2.新建映射文件

mkdir -p /usr/local/logstash/conf.d
mkdir -p /usr/local/logstash/config
mkdir -p /usr/local/logstash/logs

3.赋权

chmod -777 /usr/local/logstash

4.设置配置文件

将logstash.yml放入/usr/local/logstash/config/中

logstash.yml

注意 http.host: "0.0.0.0" 而不是指定ip

http.host: "0.0.0.0"
xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.hosts: "http://192.168.xx.xx:9200"  #es地址
xpack.monitoring.elasticsearch.username: "elastic"  #es xpack账号密码
xpack.monitoring.elasticsearch.password: "xxxx"     #es xpack账号密码
path.config: /usr/share/logstash/config/conf.d/*.conf
path.logs: /usr/share/logstash/logs

将log_to_es.conf放入/usr/local/logstash/conf.d/

log_to_es.conf（详细解释 请参考：https://blog.csdn.net/qq_20143059/category_10040348.html?spm=1001.2014.3001.5482）

input{
        tcp {
			    mode => "server"
				host => "0.0.0.0"
                port => 5000
                codec => json_lines
				type=> "datalog"
        }
		tcp {
			    mode => "server"
				host => "0.0.0.0"
                port => 4999
                codec => json_lines
				type=> "loginlog"
 
        }
}
filter{
			if[type] == "loginlog"{
			    grok {
					match => {"message" => "\\|%{GREEDYDATA:loginMsg}\\|%{GREEDYDATA:timeFormat}\\|%{GREEDYDATA:userName}"}
				}
				if([message] =~  "^(?!.*?登录系统).*$") {
					### 丢弃
					drop{}
				   }
				}
			if[type] == "datalog"{	
				grok {
					match => {"message" => "\\|%{DATA:userName}\\|%{GREEDYDATA:operationName}\\|%{DATA:timeFormat}\\|%{DATA:ip}\\|%{DATA:systemType}\\|%{GREEDYDATA:logType}\\|%{GREEDYDATA:method}\\|%{GREEDYDATA:input}"}
				}	
			}
            ruby {
				code => "event['time'] = event['@timestamp']"
			}
			mutate
			{
				add_field => ["time", "%{@timestamp}"]
			}				 
}
output{
       
				if[type] == "datalog"{
				    elasticsearch{
						hosts=>["192.168.xx.xx:9200"]
						user => "elastic"
						password => "xxxx"
						index => "xxxx-%{+YYYY.MM.dd}"
						    }
						}
				if[type] == "loginlog"{
				    elasticsearch{
						hosts=>["192.168.xx.xx:9200"]
						user => "elastic"
						password => "xxxx"
						index => "xxxx-%{+YYYY.MM.dd}"
					        }					
                        }
}

5.启动

docker run -p 5044:5044 -p 5000:5000-p 4999:4999--name=logstash \\
  --restart=always --privileged=true\\
  -e ES_JAVA_OPTS="-Xms1g -Xmx2g" \\
  -v /usr/local/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml \\
  -v /usr/local/logstash/conf.d:/usr/share/logstash/config/conf.d \\
  -v /usr/local/logstash/logs:/usr/share/logstash/logs  \\
  -d logstash:7.5.1

docker run -p 5044:5044 -p 5000:5000-p 4999:4999 ：映射的端口号 这里与上文log_to_es.conf input中一定要相同！！！！额外价格一个5044 为logstash地址
  --name=logstash \\
  --restart=always --privileged=true\\
  -e ES_JAVA_OPTS="-Xms1g -Xmx2g" \\
  -v /usr/local/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml \\
  -v /usr/local/logstash/conf.d:/usr/share/logstash/config/conf.d \\
  -v /usr/local/logstash/logs:/usr/share/logstash/logs  \\
  -d logstash:7.5.1

访问kibana可以看到是否连接成功