java调用elasticSearch api

Posted 阿无，

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了java调用elasticSearch api相关的知识，希望对你有一定的参考价值。

java操作es有两种方式

  1. 通过操作es的9300端口，9300是tcp端口，集群节点之间通信也是通过9300端口，会通过9300和es建立一个长连接，下面的es的依赖可以直接操作

    但是随着es的版本的提升spring-data需要封装不同版本的es的jar包，好像还没封装到这个版本（2019），另外官方也不推荐通过9300来操作es，而且这种方式在es8以后将被废弃

  2. 通过9200操作，发送http请求

    1. JestClient，非官方，更新慢
    2. RestTemplate（springboot），模拟发http请求，es很多操作需要自己封装，麻烦
    3. HttpCLient，同上
    4. Elasticsearch-Rest-Client，官方RestClient，封装了ES操作，API层次分明，上手简单

我们在浏览官方文档的时候发现，js可以直接操作es，那为什么我们不直接用js来操作es呢？

  1. 出于安全，因为es集群属于后端集群服务器，端口一般不对外暴露，如果对外暴露，会被别人恶意利用

  2. js对es支持度有些低，我们如果用js操作的话，不需要通过官网提供的api，我们直接发送ajax请求，用原生es语句即可

其中，官网的java api是通过9300来操作的，java rest api是通过9200来操作的

官网中有Java Low Level REST Client和Java High Level REST Client，关系就和mybatis和jdbc一样

Elasticsearch-Rest-Client（官方，推荐）

这个不是专门看视频学习的，是谷粒商城的时候，跟着老师敲的，所以其实就是一个对谷粒商城涉及到这块儿的一个总结，版本什么的自然也就是用的它的。

这算是我总结的一个api，没有真实对照的使用过，只是为了理清思路。

maven

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.5.5</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.atlinxi.gulimall</groupId>
    <artifactId>gulimall-search</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>gulimall-search</name>
    <description>elasticsearch检索服务</description>
    <properties>
        <java.version>1.8</java.version>
        <elasticsearch.version>7.4.2</elasticsearch.version>
        <spring-cloud.version>2020.0.4</spring-cloud.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>com.atlinxi.gulimall</groupId>
            <artifactId>gulimall-common</artifactId>
            <version>0.0.1-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>7.4.2</version>
        </dependency>
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.47</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>$spring-cloud.version</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

配置文件

spring.cloud.nacos.discovery.server-addr=127.0.0.1:8848
spring.application.name=gulimall-search

es配置类

package com.atlinxi.gulimall.search.config;

import org.apache.http.HttpHost;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * es配置类，给容器中注入一个RestHighLevelClient
 */
@Configuration
public class GulimallElasticSearchConfig 

    // 后端访问es的时候，出于安全考虑，可以携带一个请求头
    // 现在暂时不用
    public static final RequestOptions COMMON_OPTIONS;
    static 
        RequestOptions.Builder builder = RequestOptions.DEFAULT.toBuilder();
//        builder.addHeader("Authorization", "Bearer " + TOKEN);
//        builder.setHttpAsyncResponseConsumerFactory(
//                new HttpAsyncResponseConsumerFactory
//                        .HeapBufferedResponseConsumerFactory(30 * 1024 * 1024 * 1024));
        COMMON_OPTIONS = builder.build();
    


    @Bean
    public RestHighLevelClient esRestClient()

        RestHighLevelClient client = new RestHighLevelClient(
                RestClient.builder(
                        new HttpHost("192.168.56.10", 9200, "http")
//                        new HttpHost("localhost", 9201, "http")
                ));
        return client;

导包

package com.atlinxi.gulimall.search;

import com.alibaba.fastjson.JSON;
import com.atlinxi.gulimall.search.config.GulimallElasticSearchConfig;
import lombok.Data;
import lombok.ToString;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.xcontent.XContentType;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.aggregations.AggregationBuilders;
import org.elasticsearch.search.aggregations.Aggregations;
import org.elasticsearch.search.aggregations.bucket.terms.Terms;
import org.elasticsearch.search.aggregations.bucket.terms.TermsAggregationBuilder;
import org.elasticsearch.search.aggregations.metrics.Avg;
import org.elasticsearch.search.aggregations.metrics.AvgAggregationBuilder;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;

import java.io.IOException;

@Autowired
RestHighLevelClient restHighLevelClient;

api

查询

// 1. 创建检索请求
SearchRequest searchRequest = new SearchRequest();

// 2. 指定索引
searchRequest.indices("bank");


// 3. 指定DSL，检索条件
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();

// 3.1 构造检索条件
        // 所有的函数名都对应原生es DSL语句
//        searchSourceBuilder.query();
//        searchSourceBuilder.from();
//        searchSourceBuilder.size();
//        searchSourceBuilder.aggregation();
BoolQueryBuilder boolQuery = QueryBuilders.boolQuery();
boolQuery.must(QueryBuilders.matchQuery("skuTitle", param.getKeyword()));
boolQuery.filter(QueryBuilders.termQuery("catalogId", param.getCatalog3Id()));
boolQuery.filter(QueryBuilders.termsQuery("brandId", param.getBrandId()));

BoolQueryBuilder nestedBoolQuery = QueryBuilders.boolQuery();
QueryBuilders.nestedQuery("attrs", nestedBoolQuery, ScoreMode.None);
boolQuery.filter(nestedQuery);

QueryBuilders.rangeQuery("skuPrice");
boolQuery.filter(rangeQuery);

searchSourceBuilder.query(QueryBuilders.matchQuery("address","mill"));
searchSourceBuilder.query(boolQuery);
searchSourceBuilder.sort(field, order);
sourceBuilder.from(0);
sourceBuilder.size(10);

HighlightBuilder builder = new HighlightBuilder();
builder.field("skuTitle");
builder.preTags("<b style='color:red'>");
builder.postTags("</b>");
sourceBuilder.highlighter(builder);

// 3.2 聚合
// 按照年龄的值分布进行聚合
TermsAggregationBuilder ageAgg = AggregationBuilders.terms("ageAgg").field("age").size(10);
// 计算平均薪资
AvgAggregationBuilder balanceAvg = AggregationBuilders.avg("balanceAvg").field("balance");
AggregationBuilders.nested("attr_agg", "attrs");
        
searchSourceBuilder.aggregation(balanceAvg);
searchSourceBuilder.aggregation(ageAgg);



// 4. 执行检索请求
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = restHighLevelClient.search(searchRequest, GulimallElasticSearchConfig.COMMON_OPTIONS);


// 5.获取响应结果
SearchHits hits = searchResponse.getHits();
SearchHit[] searchHits = hits.getHits();
searchHit.getSourceAsString();


// 3.1 获取聚合结果
Aggregations aggregations = searchResponse.getAggregations();
Terms ageAgg1 = aggregations.get("ageAgg");
// 返回值为List<? extends Terms.Bucket>
ageAgg1.getBuckets()
bucket.getKeyAsString();

aggregations.get("balanceAvg");

保存更新

// 添加数据有多种方式，例如hashmap、直接将json粘在这儿
IndexRequest request = new IndexRequest("users");
request.id("1");
//        request.source("userName","zhangsan","age",12,"gender","男");

//        String jsonString = "" +
//                "\\"user\\":\\"kimchy\\"," +
//                "\\"postDate\\":\\"2013-01-30\\"," +
//                "\\"message\\":\\"trying out Elasticsearch\\"" +
//                "";
//        request.source(jsonString, XContentType.JSON);

User user = new User();
user.setUserName("zs");
user.setAge(12);
user.setGender("man");
String jsonString = JSON.toJSONString(user);
request.source(jsonString, XContentType.JSON);

// 执行保存/更新操作
IndexResponse index = restHighLevelClient.index(request, GulimallElasticSearchConfig.COMMON_OPTIONS);





// 批量保存
// 1. 建立索引 product 建立好映射关系（kibana操作）
// 2. 给es中保存这些数据
BulkRequest bulkRequest = new BulkRequest();
IndexRequest indexRequest = new IndexRequest(EsConstant.Product_INDEX);
indexRequest.id(skuEsModel.getSkuId().toString());
String s = JSON.toJSONString(skuEsModel);
indexRequest.source(s, XContentType.JSON);
bulkRequest.add(indexRequest);
BulkResponse bulk = restHighLevelClient.bulk(bulkRequest, GulimallElasticSearchConfig.COMMON_OPTIONS);

// todo 如果批量错误，处理错误
boolean b = bulk.hasFailures();
bulk.getItems()

Spring Data ElasticSearch

Spring Data可以极大的简化JPA的写法，可以在几乎不用写实现的情况下，实现对数据的访问和操作。除了CRUD外，还包括如分页、排序等一些常用的功能。

配置文件

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:es="http://www.springframework.org/schema/data/elasticsearch"


       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="
       http://www.springframework.org/schema/beans
       http://www.springframework.org/schema/beans/spring-beans.xsd


       http://www.springframework.org/schema/data/elasticsearch
       http://www.springframework.org/schema/data/elasticsearch/spring-elasticsearch.xsd

">
    <!--
        如果你希望 进一步了解 xsd相关的知识 请求百度去 百度不着了  http://www.jk1123.com/?p=124
    -->

    <!--
        配置 client 连上 es

       配置 dao层的扫描

       配置其他  一个叫做 esTemplate  就是一个简单对应client封装


    -->


     <es:transport-client id="client" cluster-nodes="127.0.0.1:9300" cluster-name="my-elasticsearch"/>

    <es:repositories base-package="com.itheima.dao"></es:repositories>

    <bean id="elasticsearchTemplate" class="org.springframework.data.elasticsearch.core.ElasticsearchTemplate">
        <constructor-arg name="client" ref="client"></constructor-arg>
    </bean>

</beans>

实体类

实体类的无参构造必须有，否则查询出来的对象无法映射到实体类

  • @Document(indexName=“blob3”,type=“article”)：
    indexName：索引的名称（必填项）
    type：索引的类型

  • @Id：主键的唯一标识

  • @Field(index=true,analyzer=“ik_smart”,store=true,searchAnalyzer=“ik_smart”,type = FieldType.text)
    analyzer：存储时使用的分词器
    searchAnalyze：搜索时使用的分词器
    store：是否存储
    type: 数据类型

Elasticsearch 5.4.3实战--Java API调用：搜索建议


 Elasticsearch 5.4.3实战--Java API调用：搜索


 shell脚本中ssh命令远程调用java启动脚本无法启动java进程


 java调用elasticSearch api


 用ChatGPT编写的一个调用ElasticSearch的maven的spring elasticsearch demo案例


 谷粒商城ES调用(十九)