webserver Elasticsearch Cluster

Posted 天行健 - Lindows

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了webserver Elasticsearch Cluster相关的知识，希望对你有一定的参考价值。

S

https://www.elastic.co

id:mail / pwd:com + string + num

elasticsearch 7.10.0 下载

https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.0-windows-x86_64.zip

https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.0-linux-x86_64.tar.gz

https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.10.0-x86_64.rpm

 

elasticsearch 5.6.9 下载

https://www.elastic.co/guide/en/elasticsearch/reference/5.6/release-notes-5.6.9.html

https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.9.tar.gz

https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.9.rpm

https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.9.msi

https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.6.9.zip

 

 

Elasticsearch 之 x-pack 安装与使用（5.6.3）

https://blog.csdn.net/qq_33772352/article/details/105724577

elasticsearch-5.6插件x-pack破解

https://blog.51cto.com/mstools/2119168

 

x-pack破解并安装

https://www.cnblogs.com/benwu/articles/6648471.html

https://www.elastic.co/guide/en/elasticsearch/reference/5.6/setup-xpack.html

安装方法如下

[esadmin@sctssitapp254 soft]$ /opt/elasticsearch-5.6.9/bin/elasticsearch-plugin install file:///opt/soft/x-pack-5.6.9.zip

破解免费1年期为50年

http://8dx.pc6.com/wwb6/luyten054.zip

https://www.elastic.co/guide/en/x-pack/5.6/installing-license.html

Elasticsearch 7.x -- https://www.elastic.co/guide/en/elastic-stack-overview/7.1/license-management.html
Elasticsearch 6.x -- https://www.elastic.co/guide/en/x-pack/current/license-management.html
Elasticsearch 5.x -- https://www.elastic.co/guide/en/x-pack/5.6/installing-license.html
Elasticsearch 2.x -- https://www.elastic.co/guide/en/marvel/current/license-management.html
Elasticsearch 1.x -- Use license code \'1010\' to register
{"license":{"uid":"e6a4569d-2b07-494b-aebe-7468b7bbe6bd","type":"basic","issue_date_in_millis":1605830400000,"expiry_date_in_millis":1637452799999,"max_nodes":100,"issued_to":"etang lindows (lindows.com)","issuer":"Web Form","signature":"AAAAAwAAAA3Jc+CLfbuTWPxJqW9tAAABmC9ZN0hjZDBGYnVyRXpCOW5Bb3FjZDAxOWpSbTVoMVZwUzRxVk1PSmkxaktJRVl5MUYvUWh3bHZVUTllbXNPbzBUemtnbWpBbmlWRmRZb25KNFlBR2x0TXc2K2p1Y1VtMG1UQU9TRGZVSGRwaEJGUjE3bXd3LzRqZ05iLzRteWFNekdxRGpIYlFwYkJiNUs0U1hTVlJKNVlXekMrSlVUdFIvV0FNeWdOYnlESDc3MWhlY3hSQmdKSjJ2ZTcvYlBFOHhPQlV3ZHdDQ0tHcG5uOElCaDJ4K1hob29xSG85N0kvTWV3THhlQk9NL01VMFRjNDZpZEVXeUtUMXIyMlIveFpJUkk2WUdveEZaME9XWitGUi9WNTZVQW1FMG1DenhZU0ZmeXlZakVEMjZFT2NvOWxpZGlqVmlHNC8rWVVUYzMwRGVySHpIdURzKzFiRDl4TmM1TUp2VTBOUlJZUlAyV0ZVL2kvVk10L0NsbXNFYVZwT3NSU082dFNNa2prQ0ZsclZ4NTltbU1CVE5lR09Bck93V2J1Y3c9PQAAAQCN8LFbgkGQ8vr/AneOP8rFQYPCbQAt1puJdjztoeEBamW7zjJzi7au0Pw5URBjugJ6KqnnS3lXk3CmYPMTgTBy7M/NmA1S1Z0mF2BQf5FBYnm04tQrbWZgn+3jSrhtwxllPj7pyBWIDO9mTmKM7PkUtyD0T1HnU0eRuwXUYPfAw6iLOqbLo7x1byuwZrlSr+kQUQUtQtZG4c/gO4c51mf3eLiEqErgFkIqSvIL3MkRZL+3UWi47a7RLeu8ddgybZLIJnoPfdcPuCJNSSsd9bh+RD89wl6BoTO9xubpP+B5HhJhLI4X6cxKaYdNRiA8rsNP7lx5UO1Y9agx7W7aPxCB","start_date_in_millis":1605830400000}}

 

 

 

 

使用nginx代理kibana并设置身份验证

https://blog.csdn.net/wyl9527/article/details/72598112

使用nginx代理kibana并设置身份验证

https://www.cnblogs.com/keithtt/p/6593866.html

nginx机器执行shell命令，生成用户名和密码。

[root@sftssitweb05 ~]# printf "admin:$(openssl passwd -crypt admin)\\n" >> /user/local/nginx/conf/SFTS_ElasticSearch_Nginx_Passwd

 

Kibana采用nginx做登陆认证

https://blog.csdn.net/dwyane__wade/article/details/80190771

mkdir -p /usr/local/test/passwd

cd /usr/local/test/passwd

htpasswd -c -b kibana.passwd user <weifan>

编辑nginx配置文件nginx.conf:

    server {
        listen       80;
        server_name  172.30.1.45:80;

        location / {
            auth_basic "kibana login auth";
            auth_basic_user_file /usr/local/test/passwd/kibana.passwd;
            proxy_pass http://172.30.1.45:5601;
            proxy_redirect off;
        }
    }

 

 

 

 

Elasticsearch集群部署

https://www.cnblogs.com/aubin/p/8012840.html

CentOS7下Elasticsearch集群部署记录

https://www.cnblogs.com/kevingrace/p/7693422.html

Elasticsearch集群  Master - Slave

http://blog.51cto.com/4925054/2084176

井底之蛙：教你快速搭建Elasticsearch搜索集群，So Easy！

https://baijiahao.baidu.com/s?id=1592190741615317567

集群配置

下载软件：http://localhost:8080/files_pre/sfts_dmz/elasticsearch-5.6.9.tar.gz 
释放路径：/opt/elasticsearch-5.6.9
部署机器："192.168.0.1", "192.168.0.2", "192.168.0.3"
修改文件：/opt/elasticsearch-5.6.9/config/elasticsearch.yml
新增内容：
cluster.name: lindows-es-cluster
#node.name: lindows-node
bootstrap.system_call_filter: false
network.host: 0.0.0.0
http.port: 9200
path.data: /data/elasticsearch-5.6.9/data
path.logs: /data/elasticsearch-5.6.9/logs
discovery.zen.ping.unicast.hosts: ["192.168.0.1", "192.168.0.2", "192.168.0.3"]
下载软件http://10.27.188.107:8080/files_pre/sfts_prd/elasticsearch-analysis-ik-5.6.9.zip 
释放上面文件到/opt/elasticsearch-5.6.9/plugins/ik目录下即可。

 

集群状态查询

http://sctsesxgpre.cns*****.com:9200/_cluster/health?pretty 

中文分词验证方法一

[mqadmin@sctssitapp214 ~]$  curl http://sctsessit.cns*****.com:9200/flow_record/_analyze?analyzer=ik_max_word -d 世界如此之大

中文分词验证方法二

 

 

 

 

 

Elasticsearch5.0 安装问题集锦 

https://www.cnblogs.com/sloveling/p/elasticsearch.html

问题一：警告提示
[2016-11-06T16:27:21,712][WARN ][o.e.b.JNANatives ] unable to install syscall filter:
java.lang.UnsupportedOperationException: seccomp unavailable: requires kernel 3.5+ with CONFIG_SECCOMP and CONFIG_SECCOMP_FILTER compiled in
at org.elasticsearch.bootstrap.Seccomp.linuxImpl(Seccomp.java:349) ~[elasticsearch-5.0.0.jar:5.0.0]
at org.elasticsearch.bootstrap.Seccomp.init(Seccomp.java:630) ~[elasticsearch-5.0.0.jar:5.0.0]
报了一大串错误，其实只是一个警告。
解决：使用心得linux版本，就不会出现此类问题了。

问题二：ERROR: bootstrap checks failed
max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]
max number of threads [1024] for user [lishang] likely too low, increase to at least [2048]
解决：切换到root用户，编辑limits.conf 添加类似如下内容
vi /etc/security/limits.conf
添加如下内容:
* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096

问题三：max number of threads [1024] for user [lish] likely too low, increase to at least [2048]
解决：切换到root用户，进入limits.d目录下修改配置文件。
vi /etc/security/limits.d/90-nproc.conf
修改如下内容：
* soft nproc 1024
#修改为
* soft nproc 2048

问题四：max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144]
解决：切换到root用户修改配置sysctl.conf
vi /etc/sysctl.conf
添加下面配置：
vm.max_map_count=655360
并执行命令：
sysctl -p

问题五：max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]
解决：修改切换到root用户修改配置limits.conf 添加下面两行
命令:vi /etc/security/limits.conf
* hard nofile 65536
* soft nofile 65536
切换到es的用户。

文件："/etc/security/limits.d/20-nproc.conf"

内容：

*          soft    nproc     11000
*          hard    nproc     11000
# #*          soft    nofile     65536
# #*          hard    nofile     1310723
root       soft    nproc     unlimited

然后，重新启动elasticsearch，即可启动成功。

六、后台运行
最后还有一个小问题，如果你在服务器上安装Elasticsearch，而你想在本地机器上进行开发，这时候，你很可能需要在关闭终端的时候，让Elasticsearch继续保持运行。最简单的方法就是使用nohup。先按Ctrl + C，停止当前运行的Elasticsearch，改用下面的命令运行Elasticsearch
nohup./bin/elasticsearch&
这样，你就可以放心地关闭服务器终端，而不用担心Elasticsearch也跟着关闭了。

问题七：unable to install syscall filter: 

[2018-10-29T14:50:27,468][WARN ][o.e.b.JNANatives ] unable to install syscall filter:
java.lang.UnsupportedOperationException: seccomp unavailable:
at org.elasticsearch.bootstrap.SystemCallFilter.linuxImpl(SystemCallFilter.java:329) ~[elasticsearch-5.6.9.jar:5.6.9]
at org.elasticsearch.bootstrap.SystemCallFilter.init(SystemCallFilter.java:617) ~[elasticsearch-5.6.9.jar:5.6.9]
at org.elasticsearch.bootstrap.JNANatives.tryInstallSystemCallFilter(JNANatives.java:258) [elasticsearch-5.6.9.jar:5.6.9]
at org.elasticsearch.bootstrap.Natives.tryInstallSystemCallFilter(Natives.java:113) [elasticsearch-5.6.9.jar:5.6.9]
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:111) [elasticsearch-5.6.9.jar:5.6.9]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:195) [elasticsearch-5.6.9.jar:5.6.9]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:342) [elasticsearch-5.6.9.jar:5.6.9]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:132) [elasticsearch-5.6.9.jar:5.6.9]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:123) [elasticsearch-5.6.9.jar:5.6.9]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:70) [elasticsearch-5.6.9.jar:5.6.9]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) [elasticsearch-5.6.9.jar:5.6.9]
at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-5.6.9.jar:5.6.9]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) [elasticsearch-5.6.9.jar:5.6.9]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) [elasticsearch-5.6.9.jar:5.6.9]

解决方案：

修改文件：[root@sftspstapp10 opt]# vim /opt/elasticsearch-5.6.9/config/elasticsearch.yml

39 # ----------------------------------- Memory -----------------------------------
40 #
41 # Lock the memory on startup:
42 #
43 #bootstrap.memory_lock: true
44 bootstrap.system_call_filter: false  #增加这行即可解决

　　

kibana 启动失败异常问题

 

[esadmin@sftspstapp10 opt]$ /opt/kibana-5.6.9-linux-x86_64/bin/kibana
  log   [08:12:25.790] [info][status][plugin:kibana@5.6.9] Status changed from uninitialized to green - Ready
  log   [08:12:25.858] [info][status][plugin:elasticsearch@5.6.9] Status changed from uninitialized to yellow - Waiting for Elasticsearch
  log   [08:12:25.894] [info][status][plugin:console@5.6.9] Status changed from uninitialized to green - Ready
  log   [08:12:25.929] [info][status][plugin:metrics@5.6.9] Status changed from uninitialized to green - Ready
  log   [08:12:25.951] [info][status][plugin:elasticsearch@5.6.9] Status changed from yellow to green - Kibana index ready
  log   [08:12:26.106] [info][status][plugin:timelion@5.6.9] Status changed from uninitialized to green - Ready
  log   [08:12:26.110] [info][listening] Server running at http://localhost:5601
  log   [08:12:26.111] [info][status][ui settings] Status changed from uninitialized to green - Ready　　

 解决方案：

修改文件：(添加如下内容后重新启动即可)

/opt/kibana-5.6.9-linux-x86_64/config/kibana.yml

server.port: 5601
server.host: "10.244.167.148"  #填写当前服务器的正确IP即可
elasticsearch.url: "http://10.244.167.148:9200"  #填写当前服务器的正确IP即可

kibana启动成功

 

验证中文分词搜索安装

http://10.49.2.188:5601/app/kibana#/dev_tools/console?_g=()

发送请求：

GET _analyze?pretty
{
  "analyzer": "ik_smart",
  "text": "中华人民共和国国歌"
}

 

输出结果如下：

 

{
  "tokens": [
    {
      "token": "中华人民共和国",
      "start_offset": 0,
      "end_offset": 7,
      "type": "CN_WORD",
      "position": 0
    },
    {
      "token": "国歌",
      "start_offset": 7,
      "end_offset": 9,
      "type": "CN_WORD",
      "position": 1
    }
  ]
}

 

问题6：验证中文分词插件是否生效

http://10.27.188.107:8080/files_pre/sfts_prd/elasticsearch-analysis-ik-5.6.9.zip

释放上面文件到/opt/elasticsearch-5.6.9/plugins/ik目录下即可。

解决6：提交POST请求

[root@sctssitapp234 ~]#  curl http://sctsessit.cns*****.com:9200/flow_record/_analyze?analyzer=ik_max_word -d 世界如此之大

 

问题7：关于搜索elasticsearch的数据条数大于10000的坑 max_result_window的两种设置方式

https://www.cnblogs.com/snake23/p/9469584.html

解决7：

第一种：[root@sctssitapp234 ~]#  curl -XPUT http://127.0.0.1:9200/_settings -d \'{ "index" : { "max_result_window" : 100000000}}\'

第二种：在/opt/elasticsearch-5.6.9/config/elasticsearch.yml文件中的最后加上index.max_result_window: 100000000，但是这种方法要注意在最前面加上空格

问题8：elasticsearch process is too low, increase to at least [65536] 

解决8：https://www.cnblogs.com/zhaohz/p/7814966.html

每次更新环境变量的时候limits.conf的hard nofile 131072设置被覆盖掉了
这就好办了，vi /etc/profile 将 ulimit -n 65535 行注释掉，退出重新进入当前用户，再使用 ulimit -Hn 查看当前值，已经是131072了，设置成功！

vi /etc/profile
# ulimit -n 65535 
ulimit -Hn
131072

 

end

以上是关于webserver Elasticsearch Cluster的主要内容，如果未能解决你的问题，请参考以下文章

c_cpp esp8266-webserver.ino

c_cpp init_webserver.ino

libwebsockets libwebsockets-webserver.c hacking

windows c webserver上的独占锁文件

CXF 简单创建Webserver 例子

如何快速启动WebDev.WebServer