搭建spring boot+elasticsearch+activemq服务
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了搭建spring boot+elasticsearch+activemq服务相关的知识，希望对你有一定的参考价值。
目前时间是：2017-01-24
本文不涉及activemq的安装
需求
activemq实时传递数据至服务 elasticsearch做索引 对外开放查询接口 完成全文检索
环境
jdk：1.8
spirng boot：1.4.3.RELEASE
elasticsearch：2.4.3
activemq：5.13.2
ES插件
head：版本好像无太大差别 能查数据就行
analysis-ik：1.10.3
search-guard-2：2.4.3.9
search-guard-ssl：2.4.3.19
注意
作者遇到的最大问题就是版本兼容 网上资料少且版本较低 故列出以下版本矩阵 方便查阅
spring boot与elasticsearch：
| Spring Boot Version (x) | Spring Data Elasticsearch Version (y) | Elasticsearch Version (z) |
| x <= 1.3.5 | y <= 1.3.4 | z <= 1.7.2* |
| x >= 1.4.x | 2.0.0 <=y < 5.0.0** | 2.0.0 <= z < 5.0.0** |
矩阵来源以及更多版本兼容：https://github.com/spring-projects/spring-data-elasticsearch
elasticsearch与ik：我的ES版本为2.x 对应如下
| IK version | ES version |
|---|---|
| master | 2.4.0 -> master |
| 1.10.3 | 2.4.3 |
| 1.9.5 | 2.3.5 |
| 1.9.4 | 2.3.4 |
| 1.9.3 | 2.3.3 |
| 1.9.0 | 2.3.0 |
| 1.8.1 | 2.2.1 |
| 1.7.0 | 2.1.1 |
| 1.5.0 | 2.0.0 |
| 1.2.6 | 1.0.0 |
| 1.2.5 | 0.90.x |
| 1.1.3 | 0.20.x |
| 1.0.0 | 0.16.2 -> 0.19.0 |
矩阵来源以及更多版本兼容：https://github.com/medcl/elasticsearch-analysis-ik
elasticsearch与search-guard以及search-guard-ssl：
| Elasticsearch Version | Latest Search Guard Version | Search Guard SSL Version | Commercial support available |
|---|---|---|---|
| 1.x.y | not available | - | - |
| 2.0.x | not available | - | - |
| 2.1.x | not available | - | - |
| 2.2.0 | 2.2.0.7 | 2.2.0.16 | Yes |
| 2.3.1 | available upon request | - | - |
| 2.3.2 | available upon request | - | - |
| 2.3.3 | 2.3.3.10 | 2.3.3.19 | YES |
| 2.3.4 | 2.3.4.10 | 2.3.4.19 | YES |
| 2.3.5 | 2.3.5.10 | 2.3.5.19 | YES |
| 2.4.0 | 2.4.0.10 | 2.4.0.19 | YES |
| 2.4.1 | 2.4.1.10 | 2.4.1.19 | YES |
| 2.4.2 | 2.4.2.10 | 2.4.2.19 | YES |
| 2.4.3 | 2.4.3.10 | 2.4.3.19 | YES |
| 2.4.4 | 2.4.4.10 | 2.4.4.19 | YES |
| 5.0.0 | 5.0.0-10 | (comes bundled since SG 5) | YES |
| 5.0.1 | 5.0.1-10 | (comes bundled since SG 5) | YES |
| 5.0.2 | 5.0.2-10 | (comes bundled since SG 5) | YES |
| 5.1.1 | 5.1.1-10 | (comes bundled since SG 5) | YES |
| 5.1.2 | 5.1.2-10 | (comes bundled since SG 5) | YES |
矩阵来源以及更多版本兼容：https://github.com/floragunncom/search-guard/wiki
开始
安装elasticsearch
我的安装目录：/usr/local
注意：elasticsearch不能用root用户运行 所以创建你的用户组和用户 切换到新用户再安装 如何创建切换 请自行搜索
wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.4.3/elasticsearch-2.4.3.tar.gz
tar -zxvf elasticsearch-2.4.3.tar.gz
cd elasticsearch-2.4.3/config/
vim elasticsearch.yml
# ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please see the documentation for further information on configuration options: # <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html> # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: 你的集群名称 # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # # node.name: node-1 # # Add custom attributes to the node: # # node.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # # path.data: /path/to/data # # Path to log files: # # path.logs: /path/to/logs # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # # bootstrap.memory_lock: true # # Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory # available on the system and that the owner of the process is allowed to use this limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 0.0.0.0 # # Set a custom port for HTTP: # # http.port: 9200 # # For more information, see the documentation at: # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html> # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when new node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # # discovery.zen.ping.unicast.hosts: ["host1", "host2"] # # Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1): # # discovery.zen.minimum_master_nodes: 3 # # For more information, see the documentation at: # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html> # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # # gateway.recover_after_nodes: 3 # # For more information, see the documentation at: # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html> # # ---------------------------------- Various ----------------------------------- # # Disable starting multiple nodes on a single system: # # node.max_local_storage_nodes: 1 # # Require explicit names when deleting indices: # # action.destructive_requires_name: true
我修改了两个地方 cluster.name和network.host
cd ../bin/
./elasticsearch
[2017-01-24 10:02:49,627][INFO ][node ] [Ariel] version[2.4.3], pid[23274], build[d38a34e/2016-12-07T16:28:56Z] [2017-01-24 10:02:49,628][INFO ][node ] [Ariel] initializing ... [2017-01-24 10:02:50,259][INFO ][plugins ] [Ariel] modules [reindex, lang-expression, lang-groovy], plugins [], sites [] [2017-01-24 10:02:50,279][INFO ][env ] [Ariel] using [1] data paths, mounts [[/ (overlay)]], net usable_space [75.3gb], net total_space [113.9gb], spins? [possibly], types [overlay] [2017-01-24 10:02:50,279][INFO ][env ] [Ariel] heap size [990.7mb], compressed ordinary object pointers [true] [2017-01-24 10:02:52,051][INFO ][node ] [Ariel] initialized [2017-01-24 10:02:52,051][INFO ][node ] [Ariel] starting ... [2017-01-24 10:02:52,110][INFO ][transport ] [Ariel] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300} [2017-01-24 10:02:52,116][INFO ][discovery ] [Ariel] elasticsearch/MI21JVBWSbKfj9nC1V6N9w [2017-01-24 10:02:55,166][INFO ][cluster.service ] [Ariel] new_master {Ariel}{MI21JVBWSbKfj9nC1V6N9w}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-join(elected_as_master, [0] joins received) [2017-01-24 10:02:55,197][INFO ][http ] [Ariel] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200} [2017-01-24 10:02:55,197][INFO ][node ] [Ariel] started
出现类似这样的信息 说明安装成功
安装head插件
elasticsearch根目录执行
bin/plugin install mobz/elasticsearch-head
注意：2.0以下版本应该是 -install
访问地址：http://ip:9200/_plugin/head/
安装analysis-ik插件
下载地址：https://github.com/medcl/elasticsearch-analysis-ik/tree/v1.10.3
使用maven打包：mvn clean package
生成的zip包在target/releases下
在elasticsearch-2.4.3/plugins下创建ik目录
将zip包放到该目录下并解压 解压出如下文件
编辑elasticsearch-2.4.3/config/elasticsearch.yml配置文件 添加如下内容
index: analysis: analyzer: ik: alias: [ik_analyzer] type: org.elasticsearch.index.analysis.IkAnalyzerProvider ik_max_word: type: ik use_smart: false ik_smart: type: ik use_smart: true
或者
index.analysis.analyzer.ik.type : “ik”
测试：http://ip:9200/_analyze?analyzer=ik&pretty=true&text=我是中国人
安装searchguard
elasticsearch根目录执行
bin/plugin install -b com.floragunn/search-guard-2/2.4.3.9
bin/plugin install -b com.floragunn/search-guard-ssl/2.4.3.19
下载 searchguard-ssl 的包，里面包含自动创建证书的脚本：
wget https://github.com/floragunncom/search-guard-ssl/archive/v2.4.3.19.zip
unzip v2.4.3.19.zip
cd search-guard-ssl-2.4.3.19/example-pki-scripts/
有三个脚本
gen_client_node_cert.sh 创建客户端证书
gen_node_cert.sh 创建节点证书
gen_root_ca.sh 创建根证书
编辑脚本 vim example.sh
#!/bin/bash set -e ./clean.sh ./gen_root_ca.sh password password ./gen_node_cert.sh 0 password password ./gen_node_cert.sh 1 password password ./gen_client_node_cert.sh admin password password cp truststore.jks node-0-keystore.jks /usr/local/elasticsearch-2.4.3/config/ cp truststore.jks admin-keystore.jks /usr/local/elasticsearch-2.4.3/plugins/search-guard-2/sgconfig/
./example.sh
可以发现 最后两句就是将证书cp到相应目录
编辑elasticsearch-2.4.3/config/elasticsearch.yml配置文件 添加如下内容
############################################################################################# # SEARCH GUARD # # Configuration # ############################################################################################# security.manager.enabled: false searchguard.authcz.admin_dn: - "CN=admin, OU=client, O=client, L=Test, C=DE" ############################################################################################# # SEARCH GUARD SSL # # Configuration # ############################################################################################# ############################################################################################# # Transport layer SSL # # # ############################################################################################# # Enable or disable node-to-node ssl encryption (default: true) # searchguard.ssl.transport.enabled: true # JKS or PKCS12 (default: JKS) #searchguard.ssl.transport.keystore_type: PKCS12 # Relative path to the keystore file (mandatory, this stores the server certificates), must be placed under the config/ dir searchguard.ssl.transport.keystore_filepath: node-0-keystore.jks # Alias name (default: first alias which could be found) #searchguard.ssl.transport.keystore_alias: my_alias # Keystore password (default: changeit) searchguard.ssl.transport.keystore_password: password # JKS or PKCS12 (default: JKS) #searchguard.ssl.transport.truststore_type: PKCS12 # Relative path to the truststore file (mandatory, this stores the client/root certificates), must be placed under the config/ dir searchguard.ssl.transport.truststore_filepath: truststore.jks # Alias name (default: first alias which could be found) #searchguard.ssl.transport.truststore_alias: my_alias # Truststore password (default: changeit) searchguard.ssl.transport.truststore_password: password # Enforce hostname verification (default: true) # searchguard.ssl.transport.enforce_hostname_verification: true # If hostname verification specify if hostname should be resolved (default: true) # searchguard.ssl.transport.resolve_hostname: true # Use native Open SSL instead of JDK SSL if available (default: true) # searchguard.ssl.transport.enable_openssl_if_available: false
在elasticsearch根目录 执行命令 将配置插入
./plugins/search-guard-2/tools/sgadmin.sh -cn 集群名称 -h hostname -cd plugins/search-guard-2/sgconfig -ks plugins/search-guard-2/sgconfig/admin-keystore.jks -kspass password -ts plugins/search-guard-2/sgconfig/truststore.jks -tspass password -nhnv
注意：elasticsearch的服务必须是运行状态
elasticsearch-2.4.3/plugins/search-guard-2/sgconfig下的配置文件是管理用户角色的
安装配置成功后 任何客户端访问elasticsearch 需提供用户名及密码
至此服务端安装结束
客户端将以源码方式提供 为公司信息安全着想 仅提供关键性代码供参考 无法运行
以上是关于搭建spring boot+elasticsearch+activemq服务的主要内容，如果未能解决你的问题，请参考以下文章
Spring Cloud 分布式链路跟踪 Sleuth + Zipkin + Elasticsear
Spring Boot基础：Spring Boot简介与快速搭建