微服务实用篇6-分布式搜索elasticsearch篇2

Posted nuist__NJUPT

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了微服务实用篇6-分布式搜索elasticsearch篇2相关的知识，希望对你有一定的参考价值。

今天我们继续学习分布式搜索引擎elasticsearch，今天主要学习四个模块，分别为DSL查询文档，搜索结果处理，RestClient查询文档，还有最好演示一个旅游案例。下面开始今天的学习吧。

目录

一、DSL查询文档

1.1、DSL查询分类

1.2、DSL查询之复合查询function_score

1.3、DSL查询之复合查询boolean

二、ES搜索结果处理

2.1、搜索结果处理-排序

2.2、搜索结果处理-分页

2.3、搜索结果处理-高亮

三、RestClient查询文档

3.1、快速入门

3.2、RestClient查询文档-match、term、range、bool

3.3、RestClient搜索结果处理-排序和分页

3.4、RestClient搜索结果处理-高亮显示

四、旅游案例

4.1、搜索与分页

4.2、条件过滤

4.3、附近的酒店距离升序排序

一、DSL查询文档

1.1、DSL查询分类

常见的DSL查询分为如下几类：查询所有、全文查询、精确查询、地理查询、复合查询。


#查询所有
GET /hotel/_search

  "query": 
    "match_all": 
  


#全文检索-match单字段查询
GET /hotel/_search

  "query": 
    "match": 
      "name": "上海"
    
  


#全文检索-multi_match多字段查询
GET /hotel/_search

  "query": 
    "multi_match": 
      "query": "上海如家外滩"
      , "fields": ["brand","name","business"]
    
  


#精确查询-term查询
GET /hotel/_search

  "query": 
    "term": 
      "city.keyword": 
       "value": "上海"
       
    
  


#精确查询-range查询
GET /hotel/_search

  "query": 
    "range": 
      "price": 
       "gt": 100
       , "lt": 10000
       
    
  


#distance查询
GET /hotel/_search

  "query": 
    "geo_distance": 
      "distance": "30km",
      "location": "31, 121"

1.2、DSL查询之复合查询function_score

function_score的查询方式可以修改文档的相关性分，根据新的得分进行排序，主要包含四个部分，分别为：原始查询条件、过滤条件、算分函数、加权模式。

我们看下面的一个案例，给名为如家的酒店排名设计靠前一点，具体如下：

我们查询所有名称包含“外滩”的酒店名，然后使用算分函数对brand为如家的进行过滤，将其排名靠前一些，如下所示。

#function-score查询
GET /hotel/_search

  "query": 
    "function_score": 
      "query": 
        "match": 
          "name": "外滩"
        
      
      , "functions": [
        
          "filter": 
            "term": 
              "brand.keyword": "如家"
            
          ,
          "weight": 12
        
      ],
      "boost_mode": "sum"

1.3、DSL查询之复合查询boolean

boolean查询和function_score都是复合查询，但是boolean查询是不参与算分的，只返回是否满足，must是必须满足、should是满足其中之一、must_not是必须不满足、filter是过滤。

我们看下面的案例，首先是must必须包含如家，价格大于400取反，用filter过滤在坐标范围的10km内的酒店，具体如下：

 

二、ES搜索结果处理

2.1、搜索结果处理-排序

我们先看第一个案例，使用sort对酒店先按照评价进行降序，再按照价格进行升序排序。

 

#sort排序
GET /hotel/_search

  "query": 
    "match_all":  
    ,
    "sort":[
      "score": "desc"
    ,
    
      "price": "asc"
    ]

下面演示按地理位置进行升序排序，具体如下：

#sort排序
GET /hotel/_search

  "query": 
    "match_all":  
    ,
    "sort":[
      
     "_geo_distance": 
       "location": 
         "lat": 22.507276,
         "lon": 113.931251
        ,
       "order": "asc"
       , "unit": "km"
     
    
    ]

2.2、搜索结果处理-分页

ES通过修改from和size的参数来控制返回分页的结果，具体如下：按价格进行升序排序，然后返回10条数据，如下所示：

因为ES是分布式搜索，因此会面临深度分页的问题，他需要把所有的如下1000条数据都查询并聚合后重新排序，再从中截取10条文档数据，这样的话，可能会导致搜索页数过深，对内存和CPU的消耗也很大。

 ES分页一般有如下三种分页方式，目前用的比较多的还是from+size的搜索分页方式。

2.3、搜索结果处理-高亮

高亮：即对搜索的结果进行关键字突出显示，给高亮的字段加标签即可。

 将名称为如家的字段进行高亮显示，具体如下，标签可以省略，自动默认em标签：

GET /hotel/_search

  "query": 
    "match": 
      "name": "如家"
    
  ,
  "highlight": 
    "fields": 
      "name":

三、RestClient查询文档

3.1、快速入门

Match_all查询：主要包括两部分，DSL请求的发送，以及Json结果的解析，具体如下：


    @Test
    void testMatchAll() throws IOException 
        //1.创建request对象
        SearchRequest searchRequest = new SearchRequest("hotel") ;
        //2.准备DSL
        searchRequest.source().query(QueryBuilders.matchAllQuery()) ;
        //3.发送请求
        SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT) ;
        //4.解析响应
        SearchHits searchHits = response.getHits() ;
        //获取总条数
        long total = searchHits.getTotalHits().value ;
        System.out.println("一共搜索到了" + total + "条数据！");
        //获取文档数组
        SearchHit [] searchHits1 = searchHits.getHits() ;
        for(SearchHit searchHits2 : searchHits1)
            String json = searchHits2.getSourceAsString() ;
            //反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
            System.out.println(hotelDoc);

3.2、RestClient查询文档-match、term、range、bool

下面演示mtach查询，查询name中包含“如家”的酒店名称，具体如下，其中我把解析响应单独抽取出来作为一个方法，直接调用方法进行解析即可，不用重复撰写。


    @Test
    void testMatch() throws IOException 
        //1.创建request对象
        SearchRequest searchRequest = new SearchRequest("hotel") ;
        //2.准备DSL
        searchRequest.source().query(QueryBuilders.matchQuery("name","如家")) ;
        //3.发送请求
        SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT) ;
        handleResponse(response);
    

    private void handleResponse(SearchResponse response) 
        //4.解析响应
        SearchHits searchHits = response.getHits() ;
        //获取总条数
        long total = searchHits.getTotalHits().value ;
        System.out.println("一共搜索到了" + total + "条数据！");
        //获取文档数组
        SearchHit[] searchHits1 = searchHits.getHits() ;
        for(SearchHit searchHits2 : searchHits1)
            String json = searchHits2.getSourceAsString() ;
            //反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
            System.out.println(hotelDoc);

下面演示bool查询，使用了must和filter，同时其中包含term精准查询和range查询，具体如下：

    @Test
    void testAll() throws IOException 
        //1.创建request对象
        SearchRequest searchRequest = new SearchRequest("hotel") ;
        //2.准备DSL
        BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery() ;
        //添加term
        boolQueryBuilder.must(QueryBuilders.termQuery("city.keyword","上海")) ;
        //添加range
        boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").lte(500)) ;
        searchRequest.source().query(boolQueryBuilder) ;
        //3.发送请求
        SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT) ;
        handleResponse(response);

3.3、RestClient搜索结果处理-排序和分页

定义页码和页面大小，每次将查询到的对象按照price升序排序，然后进行分页，每5个一页，具体如下所示：

    @Test
    void testPageAndSort() throws IOException 
        int page = 1 , pageSize = 5 ;

        //1.创建request对象
        SearchRequest searchRequest = new SearchRequest("hotel") ;
        //2.准备DSL
        searchRequest.source().query(QueryBuilders.matchAllQuery()) ;
        //排序
        searchRequest.source().sort("price", SortOrder.ASC) ;
        //分页
        searchRequest.source().from((page-1)*pageSize).size(pageSize) ;
        //3.发送请求
        SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT) ;
        //4.解析响应
        handleResponse(response);

3.4、RestClient搜索结果处理-高亮显示

将查询的结果进行高亮显示，查询到name为如家的，进行高亮显示，对处理的结果进行覆盖，然后就可以形成高亮显示了。

 @Test
    void testHighlight() throws IOException 

        //1.创建request对象
        SearchRequest searchRequest = new SearchRequest("hotel") ;
        //2.准备DSL
        searchRequest.source().query(QueryBuilders.matchQuery("name","如家")) ;
        //高亮显示
        searchRequest.source().highlighter(new HighlightBuilder().field("name").requireFieldMatch(false)) ;
        //3.发送请求
        SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT) ;
        //4.解析响应
        handleResponse1(response);
    

    private void handleResponse1(SearchResponse response) 
        SearchHits searchHits = response.getHits() ;
        //获取总条数
        long total = searchHits.getTotalHits().value ;
        System.out.println("一共搜索到了" + total + "条数据！");
        //获取文档数组
        SearchHit[] searchHits1 = searchHits.getHits() ;
        for(SearchHit searchHits2 : searchHits1)
            String json = searchHits2.getSourceAsString() ;
            //反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
            //获取高亮的结果
            Map<String, HighlightField> highlightFields = searchHits2.getHighlightFields();
            if(!CollectionUtils.isEmpty(highlightFields))
                HighlightField highlightField = highlightFields.get("name") ;
                if(highlightField != null)
                    String name = highlightField.getFragments()[0].string() ;
                    hotelDoc.setName(name);
                
            
            System.out.println(hotelDoc);

四、旅游案例

4.1、搜索与分页

实现旅游网站的酒店搜索功能，完成关键字搜索和分页，具体包括下面的三个步骤：

 定义实体类如下：


import lombok.Data;

@Data
public class RequestParams 
    private String key ;
    private Integer page ;
    private Integer size ;
    private String sortBy ;
    



import java.util.List;

@Data
public class PageResult 
    private Long total ;
    private List<HotelDoc> hotels ;

    public PageResult() 
    

    public PageResult(Long total, List<HotelDoc> hotels) 
        this.total = total;
        this.hotels = hotels;

定义表现层controller，接收前端页面请求，调用业务层方法实现查找和分页。

import cn.itcast.hotel.pojo.PageResult;
import cn.itcast.hotel.pojo.RequestParams;
import cn.itcast.hotel.service.IHotelService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/hotel")
public class HotelController 

    @Autowired
    private IHotelService iHotelService ;

    @PostMapping("/list")
    public PageResult search(@RequestBody RequestParams params)
        return iHotelService.search(params) ;

在业务层定义接口和实现类，如下，在实现类中完成查询和分页业务，将解析得到的结果进行封装后返回。其中注入的client在启动类中注解为bean，变成配置类交给spring管理。


import cn.itcast.hotel.mapper.HotelMapper;
import cn.itcast.hotel.pojo.Hotel;
import cn.itcast.hotel.pojo.HotelDoc;
import cn.itcast.hotel.pojo.PageResult;
import cn.itcast.hotel.pojo.RequestParams;
import cn.itcast.hotel.service.IHotelService;
import com.alibaba.fastjson.JSON;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

@Service
public class HotelService extends ServiceImpl<HotelMapper, Hotel> implements IHotelService 

    @Autowired
    private RestHighLevelClient client ;

    @Override
    public PageResult search(RequestParams params) 

        try 
            //1.创建request对象
            SearchRequest searchRequest = new SearchRequest("hotel");
            //2.准备DSL
            String key = params.getKey();
            if (key == null || "".equals(key)) 
                searchRequest.source().query(QueryBuilders.matchAllQuery());
             else 
                searchRequest.source().query(QueryBuilders.matchQuery("name", key));
            
            //分页
            int page = params.getPage();
            int pageSize = params.getSize();
            searchRequest.source().from((page - 1) * pageSize).size(pageSize);
            //3.发送请求
            SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
            //4.解析响应
            return handleResponse(response);
        catch (IOException e)
            throw new RuntimeException(e) ;
        

    

    private PageResult handleResponse(SearchResponse response) 
        //4.解析响应
        SearchHits searchHits = response.getHits() ;
        //获取总条数
        long total = searchHits.getTotalHits().value ;
        //获取文档数组
        SearchHit[] searchHits1 = searchHits.getHits() ;
        //遍历
        List<HotelDoc> list = new ArrayList<>() ;
        for(SearchHit searchHits2 : searchHits1)
            String json = searchHits2.getSourceAsString() ;
            //反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
            list.add(hotelDoc) ;
        
        return new PageResult(total, list) ;

4.2、条件过滤

添加品牌、价格、星级、城市等条件过滤功能，具体实现步骤如下：

修改实体类，添加一些参数，如下：


import lombok.Data;

@Data
public class RequestParams 
    private String key ;
    private Integer page ;
    private Integer size ;
    private String sortBy ;
    private String city ;
    private String brand ;
    private String starName ;
    private Integer minPrice ;
    private Integer maxPrice ;

然后在业务层的实现方法中进行条件过滤即可，具体如下：


import cn.itcast.hotel.mapper.HotelMapper;
import cn.itcast.hotel.pojo.Hotel;
import cn.itcast.hotel.pojo.HotelDoc;
import cn.itcast.hotel.pojo.PageResult;
import cn.itcast.hotel.pojo.RequestParams;
import cn.itcast.hotel.service.IHotelService;
import com.alibaba.fastjson.JSON;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

@Service
public class HotelService extends ServiceImpl<HotelMapper, Hotel> implements IHotelService 

    @Autowired
    private RestHighLevelClient client ;

    @Override
    public PageResult search(RequestParams params) 

        try 
            //1.创建request对象
            SearchRequest searchRequest = new SearchRequest("hotel");
            buildBasicQuery(params, searchRequest);
            //分页
            int page = params.getPage();
            int pageSize = params.getSize();
            searchRequest.source().from((page - 1) * pageSize).size(pageSize);
            //3.发送请求
            SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
            //4.解析响应
            return handleResponse(response);
        catch (IOException e)
            throw new RuntimeException(e) ;
        

    

    private void buildBasicQuery(RequestParams params, SearchRequest searchRequest) 
        //2.关键字搜索
        BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery() ;
        String key = params.getKey();
        if (key == null || "".equals(key)) 
            boolQueryBuilder.must(QueryBuilders.matchAllQuery());
         else 
            boolQueryBuilder.must(QueryBuilders.matchQuery("name", key));
        
        //城市条件搜索
        if(params.getCity() != null && !"".equals(params.getCity()))
            boolQueryBuilder.filter(QueryBuilders.termQuery("city.keyword", params.getCity())) ;
        
        //品牌条件搜索
        if(params.getBrand() != null && !"".equals(params.getBrand()))
            boolQueryBuilder.filter(QueryBuilders.termQuery("brand.keyword", params.getBrand())) ;
        
        //星级条件搜索
        if(params.getStarName() != null && !"".equals(params.getStarName()))
            boolQueryBuilder.filter(QueryBuilders.termQuery("starName.keyword", params.getStarName())) ;
        
        //价格条件搜索
        if(params.getMinPrice() != null && params.getMaxPrice() != null)
            boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").
                    gte(params.getMinPrice()).lte(params.getMaxPrice())) ;
        

        searchRequest.source().query(boolQueryBuilder) ;

    

    private PageResult handleResponse(SearchResponse response) 
        //4.解析响应
        SearchHits searchHits = response.getHits() ;
        //获取总条数
        long total = searchHits.getTotalHits().value ;
        //获取文档数组
        SearchHit[] searchHits1 = searchHits.getHits() ;
        //遍历
        List<HotelDoc> list = new ArrayList<>() ;
        for(SearchHit searchHits2 : searchHits1)
            String json = searchHits2.getSourceAsString() ;
            //反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
            list.add(hotelDoc) ;
        
        return new PageResult(total, list) ;

4.3、附近的酒店距离升序排序

获取前端的位置数据，根据计算得到的位置升序排序，找出最近的酒店位置，如下：

import cn.itcast.hotel.mapper.HotelMapper;
import cn.itcast.hotel.pojo.Hotel;
import cn.itcast.hotel.pojo.HotelDoc;
import cn.itcast.hotel.pojo.PageResult;
import cn.itcast.hotel.pojo.RequestParams;
import cn.itcast.hotel.service.IHotelService;
import com.alibaba.fastjson.JSON;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.geo.GeoPoint;
import org.elasticsearch.common.unit.DistanceUnit;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.sort.SortBuilder;
import org.elasticsearch.search.sort.SortBuilders;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

@Service
public class HotelService extends ServiceImpl<HotelMapper, Hotel> implements IHotelService 

    @Autowired
    private RestHighLevelClient client ;

    @Override
    public PageResult search(RequestParams params) 

        try 
            //1.创建request对象
            SearchRequest searchRequest = new SearchRequest("hotel");
            buildBasicQuery(params, searchRequest);
            //分页
            int page = params.getPage();
            int pageSize = params.getSize();
            searchRequest.source().from((page - 1) * pageSize).size(pageSize);
            //排序
            String location = params.getLocation() ;
            if(location != null && !"".equals(location))
                searchRequest.source().sort(SortBuilders.
                        geoDistanceSort("location", new GeoPoint(location)).
                        order(SortOrder.ASC).unit(DistanceUnit.KILOMETERS)) ;
            
            //3.发送请求
            SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
            //4.解析响应
            return handleResponse(response);
        catch (IOException e)
            throw new RuntimeException(e) ;
        

    

    private void buildBasicQuery(RequestParams params, SearchRequest searchRequest) 
        //2.关键字搜索
        BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery() ;
        String key = params.getKey();
        if (key == null || "".equals(key)) 
            boolQueryBuilder.must(QueryBuilders.matchAllQuery());
         else 
            boolQueryBuilder.must(QueryBuilders.matchQuery("name", key));
        
        //城市条件搜索
        if(params.getCity() != null && !"".equals(params.getCity()))
            boolQueryBuilder.filter(QueryBuilders.termQuery("city.keyword", params.getCity())) ;
        
        //品牌条件搜索
        if(params.getBrand() != null && !"".equals(params.getBrand()))
            boolQueryBuilder.filter(QueryBuilders.termQuery("brand.keyword", params.getBrand())) ;
        
        //星级条件搜索
        if(params.getStarName() != null && !"".equals(params.getStarName()))
            boolQueryBuilder.filter(QueryBuilders.termQuery("starName.keyword", params.getStarName())) ;
        
        //价格条件搜索
        if(params.getMinPrice() != null && params.getMaxPrice() != null)
            boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").
                    gte(params.getMinPrice()).lte(params.getMaxPrice())) ;
        

        searchRequest.source().query(boolQueryBuilder) ;

    

    private PageResult handleResponse(SearchResponse response) 
        //4.解析响应
        SearchHits searchHits = response.getHits() ;
        //获取总条数
        long total = searchHits.getTotalHits().value ;
        //获取文档数组
        SearchHit[] searchHits1 = searchHits.getHits() ;
        //遍历
        List<HotelDoc> list = new ArrayList<>() ;
        for(SearchHit searchHits2 : searchHits1)
            String json = searchHits2.getSourceAsString() ;
            //反序列化
            HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
            //获取排序值
            Object [] objects = searchHits2.getSortValues() ;
            if(objects.length > 0)
                Object value = objects[0] ;
                hotelDoc.setDistance(value);
            
            list.add(hotelDoc) ;
        
        return new PageResult(total, list) ;

以上是关于微服务实用篇6-分布式搜索elasticsearch篇2的主要内容，如果未能解决你的问题，请参考以下文章

微服务实用篇5-分布式搜索elasticsearch篇1

微服务实用篇5-分布式搜索elasticsearch篇1

微服务实用篇--学习笔记

实用篇SpringCloud+RabbitMQ+Docker+Redis+搜索+分布式，系统详解springcloud分布式

SpringCloud之微服务实用篇1

SpringCloud之微服务实用篇1