CentOS7??????Elasticsearch+ Logstash+kibana??????????????????????????????

Posted 2020-10-13

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了CentOS7??????Elasticsearch+ Logstash+kibana??????????????????????????????相关的知识，希望对你有一定的参考价值。

?????????centos7??????elasticsearch+ logstash+kibana??????????????????????????????

CentOS7??????Elasticsearch+ Logstash+kibana??????????????????????????????

?????????

??????logstash,elasticsearch,kibana??????????????????????????????????????????????????????????????????

???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

Elasticsearch??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? restful ??????????????????????????????????????????????????????

Logstash ????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

kibana??????????????????????????????????????????Kibana ????????? Logstash ??? ElasticSearch ?????????????????????????????? Web ??????????????????????????????????????????????????????????????????

????????????elastic:

1?????????java???

[[email protected] elasticsearch-5.6.3]# yum install -y *jdk*

2?????????????????????????????????

[[email protected] src]# cat/etc/security/limits.conf

* soft nofile 65536

* hard nofile 131072

* soft nproc 2048

* hard nproc 4096

2??????????????????????????????log,data???????????????

[[email protected]]# mkdir data

[[email protected]]# mkdir log

[[email protected]]# mkdir logs ##????????????????????????????????????????????????

3?????????elastic.yaml:????????????

[[email protected] elasticsearch-5.6.3]# vimconfig/elasticsearch.yml

node.name: cml3

network.host: 192.168.5.104

discovery.zen.minimum_master_nodes: 1 ##????????????????????????????????????????????????????????????????????????1???

4?????????elastic??????????????????elasticsearch

[[email protected] elasticsearch-5.6.3]# useraddelastic

[[email protected] elasticsearch-5.6.3]# ls

bin config data lib LICENSE.txt log logs modules NOTICE.txt plugins README.textile

##????????????????????????????????????????????????????????????

[[email protected] logs]# ll

total 4

-rw-rw-r-- 1 elastic elastic 0 Nov 3 19:52 elasticsearch_deprecation.log

-rw-rw-r-- 1 elastic elastic 0 Nov 3 19:52 elasticsearch_index_indexing_slowlog.log

-rw-rw-r-- 1 elastic elastic 0 Nov 3 19:52 elasticsearch_index_search_slowlog.log

##??????5.0?????????????????????????????????root?????????????????????elasticsearch-5.6.3?????????elastic??????????????????????????????????????????

##??????haed:?????????

http.cors.enabled:true

http.cors.allow-origin:"*"

##????????????????????????????????????

OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should conf...CThreads=N

OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x0000000085330000, 2060255232, 0) failed; error=???Cannot a ...???(errno=12)

##???????????????

?????????????????????????????????????????????????????????

##????????????????????????????????????

[[email protected] bin]$ ./elasticsearch

Exception in thread "main"2017-11-03 20:38:47,194 main ERROR No log4j2 configuration file found. Usingdefault configuration: logging only errors to the console. Set system property???log4j2.debug??? to show Log4j2 internal initialization logging.

2017-11-03 20:38:47,646 main ERROR Couldnot register mbeans java.security.AccessControlException: access denied("javax.management.MBeanTrustPermission" "register")

##?????????????????????????????????log4j??????ok???

[[email protected] elasticsearch-5.6.3]# yuminstall -y log4j*

5?????????elastic:?????????curl??????9200??????????????????

[[email protected] elasticsearch-5.6.3]# netstat-ntlp

Active Internet connections (only servers)

tcp6 0 0 192.168.5.104:9200 :::* LISTEN 3749/java

tcp6 0 0 192.168.5.104:9300 :::* LISTEN 3749/java

????????????head?????????

1????????????

????????????es???????????????????????????????????????head?????????????????????????????????????????????????????????

5.0??????head????????????????????????????????????????????????

2???????????????????????????????????????

#mkdir head ##???????????????????????????

#cd head

#git clonegit://github.com/mobz/elasticsearch-head.git

#cd elasticsearch-head

yum install -y npm* ##??????npm??????????????????ok???

#npm install ##??????????????????????????????????????????????????????????????????

#npm install -g grunt-cli

#grunt server

3??????????????????localhost??????????????????????????????

[[email protected] head]# cdelasticsearch-head/

[[email protected] elasticsearch-head]#vim Gruntfile.js

??????????????????

server: {

options: {

port: 9100,

hostname: ???0.0.0.0???, #####???????????????

base: ???.???,

keepalive: true

}

??????es???????????????????????????

[[email protected]]# vim elasticsearch.yml

http.cors.enabled:true

http.cors.allow-origin:"*"

#####????????????es??????

4?????????head?????????????????????web???ip??????????????????

#gruntserver

##???????????????????????????????????????????????????????????????????????????????????????????????????

??????????????????logstash?????????????????????

1???????????????logstash???

????????????: https://artifacts.elastic.co/downloads/logstash/logstash-5.6.3.tar.gz

[[email protected] src]# tar -xflogstash-5.6.3.tar.gz

2?????????conf?????????

[[email protected] ~]# mkdir /logstash/

[[email protected] logstash]# viminput_flter_output.conf

input {

file{

path=> "/usr/local/nginx/logs/cml.log" ##nginx?????????????????????

type=> "cml" ##???????????????

start_position=> "beginning" ##???????????????????????????????????????

}

stdin{}

}

filter{

grok {

match => {

"message" =>"(?<remote_IP>\d+.\d+.\d+.\d+)\s-\s-\s\[(?<DATA>\d+/\w+/\d+:\d+:\d+:\d+)[[:space:]](?<time_zone>\+\d+)\]\s\"(?<action>\w+)%{URIPATHPARAM:request} (?<Version>\w+/\d+.\d+)\"\s(?<status>\w+)\s(?<web_size>\w+)\s\"(?<check>\S+)\"\s"

##?????????grok

}

output{

elasticsearch{

action=> "index"

hosts=> "192.168.5.104:9200" ##?????????elasticsearch??????

index=> "log-%{+yyyy.MM.dd}" ##????????????log-???????????????

}

stdout {codec=>rubydebug} ##??????????????????????????????

}

3?????????logstash

[[email protected] logstash]# /usr/local/src/logstash-5.6.3/bin/logstash-f input_flter_output.conf

????????????kibana?????????????????????ip??????????????????

1???????????????kibana???

[[email protected] logstash]# cd /usr/local/src/

[[email protected] src]# wget https://artifacts.elastic.co/downloads/kibana/kibana-5.6.3-x86_64.rpm

[[email protected] src]# rpm -ivhkibana-5.6.3-x86_64.rpm

2?????????kibana???????????????

[[email protected] config]# vim kibana.yml

server.host: "192.168.63.246"

elasticsearch.url: http://192.168.63.246:9200

elasticsearch.username: "elastic"

elasticsearch.password: "changeme ##x-pack?????????????????????????????????(es???kibana?????????????????????????????????????????????????????????????????????????????????????????????????????????)

3?????????kibana

[[email protected] src]# systemctl start kibana

???????????????http://IP:5601??????input_flter_output.conf??????????????????????????????????????????kibana???