微服务实用篇6-分布式搜索elasticsearch篇2
Posted nuist__NJUPT
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了微服务实用篇6-分布式搜索elasticsearch篇2相关的知识,希望对你有一定的参考价值。
今天我们继续学习分布式搜索引擎elasticsearch,今天主要学习四个模块,分别为DSL查询文档,搜索结果处理,RestClient查询文档,还有最好演示一个旅游案例。下面开始今天的学习吧。
目录
3.2、RestClient查询文档-match、term、range、bool
一、DSL查询文档
1.1、DSL查询分类
常见的DSL查询分为如下几类:查询所有、全文查询、精确查询、地理查询、复合查询。
#查询所有
GET /hotel/_search
"query":
"match_all":
#全文检索-match单字段查询
GET /hotel/_search
"query":
"match":
"name": "上海"
#全文检索-multi_match多字段查询
GET /hotel/_search
"query":
"multi_match":
"query": "上海如家外滩"
, "fields": ["brand","name","business"]
#精确查询-term查询
GET /hotel/_search
"query":
"term":
"city.keyword":
"value": "上海"
#精确查询-range查询
GET /hotel/_search
"query":
"range":
"price":
"gt": 100
, "lt": 10000
#distance查询
GET /hotel/_search
"query":
"geo_distance":
"distance": "30km",
"location": "31, 121"
1.2、DSL查询之复合查询function_score
function_score的查询方式可以修改文档的相关性分,根据新的得分进行排序,主要包含四个部分,分别为:原始查询条件、过滤条件、算分函数、加权模式。
我们看下面的一个案例,给名为如家的酒店排名设计靠前一点,具体如下:
我们查询所有名称包含“外滩”的酒店名,然后使用算分函数对brand为如家的进行过滤,将其排名靠前一些,如下所示。
#function-score查询
GET /hotel/_search
"query":
"function_score":
"query":
"match":
"name": "外滩"
, "functions": [
"filter":
"term":
"brand.keyword": "如家"
,
"weight": 12
],
"boost_mode": "sum"
1.3、DSL查询之复合查询boolean
boolean查询和function_score都是复合查询,但是boolean查询是不参与算分的,只返回是否满足,must是必须满足、should是满足其中之一、must_not是必须不满足、filter是过滤。
我们看下面的案例,首先是must必须包含如家,价格大于400取反,用filter过滤在坐标范围的10km内的酒店,具体如下:
二、ES搜索结果处理
2.1、搜索结果处理-排序
我们先看第一个案例,使用sort对酒店先按照评价进行降序,再按照价格进行升序排序。
#sort排序
GET /hotel/_search
"query":
"match_all":
,
"sort":[
"score": "desc"
,
"price": "asc"
]
下面演示按地理位置进行升序排序,具体如下:
#sort排序
GET /hotel/_search
"query":
"match_all":
,
"sort":[
"_geo_distance":
"location":
"lat": 22.507276,
"lon": 113.931251
,
"order": "asc"
, "unit": "km"
]
2.2、搜索结果处理-分页
ES通过修改from和size的参数来控制返回分页的结果,具体如下:按价格进行升序排序,然后返回10条数据,如下所示:
因为ES是分布式搜索,因此会面临深度分页的问题,他需要把所有的如下1000条数据都查询并聚合后重新排序,再从中截取10条文档数据,这样的话,可能会导致搜索页数过深,对内存和CPU的消耗也很大。
ES分页一般有如下三种分页方式,目前用的比较多的还是from+size的搜索分页方式。
2.3、搜索结果处理-高亮
高亮:即对搜索的结果进行关键字突出显示,给高亮的字段加标签即可。
将名称为如家的字段进行高亮显示,具体如下,标签可以省略,自动默认em标签:
GET /hotel/_search
"query":
"match":
"name": "如家"
,
"highlight":
"fields":
"name":
三、RestClient查询文档
3.1、快速入门
Match_all查询:主要包括两部分,DSL请求的发送,以及Json结果的解析,具体如下:
@Test
void testMatchAll() throws IOException
//1.创建request对象
SearchRequest searchRequest = new SearchRequest("hotel") ;
//2.准备DSL
searchRequest.source().query(QueryBuilders.matchAllQuery()) ;
//3.发送请求
SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT) ;
//4.解析响应
SearchHits searchHits = response.getHits() ;
//获取总条数
long total = searchHits.getTotalHits().value ;
System.out.println("一共搜索到了" + total + "条数据!");
//获取文档数组
SearchHit [] searchHits1 = searchHits.getHits() ;
for(SearchHit searchHits2 : searchHits1)
String json = searchHits2.getSourceAsString() ;
//反序列化
HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
System.out.println(hotelDoc);
3.2、RestClient查询文档-match、term、range、bool
下面演示mtach查询,查询name中包含“如家”的酒店名称,具体如下,其中我把解析响应单独抽取出来作为一个方法,直接调用方法进行解析即可,不用重复撰写。
@Test
void testMatch() throws IOException
//1.创建request对象
SearchRequest searchRequest = new SearchRequest("hotel") ;
//2.准备DSL
searchRequest.source().query(QueryBuilders.matchQuery("name","如家")) ;
//3.发送请求
SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT) ;
handleResponse(response);
private void handleResponse(SearchResponse response)
//4.解析响应
SearchHits searchHits = response.getHits() ;
//获取总条数
long total = searchHits.getTotalHits().value ;
System.out.println("一共搜索到了" + total + "条数据!");
//获取文档数组
SearchHit[] searchHits1 = searchHits.getHits() ;
for(SearchHit searchHits2 : searchHits1)
String json = searchHits2.getSourceAsString() ;
//反序列化
HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
System.out.println(hotelDoc);
下面演示bool查询,使用了must和filter,同时其中包含term精准查询和range查询,具体如下:
@Test
void testAll() throws IOException
//1.创建request对象
SearchRequest searchRequest = new SearchRequest("hotel") ;
//2.准备DSL
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery() ;
//添加term
boolQueryBuilder.must(QueryBuilders.termQuery("city.keyword","上海")) ;
//添加range
boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").lte(500)) ;
searchRequest.source().query(boolQueryBuilder) ;
//3.发送请求
SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT) ;
handleResponse(response);
3.3、RestClient搜索结果处理-排序和分页
定义页码和页面大小,每次将查询到的对象按照price升序排序,然后进行分页,每5个一页,具体如下所示:
@Test
void testPageAndSort() throws IOException
int page = 1 , pageSize = 5 ;
//1.创建request对象
SearchRequest searchRequest = new SearchRequest("hotel") ;
//2.准备DSL
searchRequest.source().query(QueryBuilders.matchAllQuery()) ;
//排序
searchRequest.source().sort("price", SortOrder.ASC) ;
//分页
searchRequest.source().from((page-1)*pageSize).size(pageSize) ;
//3.发送请求
SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT) ;
//4.解析响应
handleResponse(response);
3.4、RestClient搜索结果处理-高亮显示
将查询的结果进行高亮显示,查询到name为如家的,进行高亮显示,对处理的结果进行覆盖,然后就可以形成高亮显示了。
@Test
void testHighlight() throws IOException
//1.创建request对象
SearchRequest searchRequest = new SearchRequest("hotel") ;
//2.准备DSL
searchRequest.source().query(QueryBuilders.matchQuery("name","如家")) ;
//高亮显示
searchRequest.source().highlighter(new HighlightBuilder().field("name").requireFieldMatch(false)) ;
//3.发送请求
SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT) ;
//4.解析响应
handleResponse1(response);
private void handleResponse1(SearchResponse response)
SearchHits searchHits = response.getHits() ;
//获取总条数
long total = searchHits.getTotalHits().value ;
System.out.println("一共搜索到了" + total + "条数据!");
//获取文档数组
SearchHit[] searchHits1 = searchHits.getHits() ;
for(SearchHit searchHits2 : searchHits1)
String json = searchHits2.getSourceAsString() ;
//反序列化
HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
//获取高亮的结果
Map<String, HighlightField> highlightFields = searchHits2.getHighlightFields();
if(!CollectionUtils.isEmpty(highlightFields))
HighlightField highlightField = highlightFields.get("name") ;
if(highlightField != null)
String name = highlightField.getFragments()[0].string() ;
hotelDoc.setName(name);
System.out.println(hotelDoc);
四、旅游案例
4.1、搜索与分页
实现旅游网站的酒店搜索功能,完成关键字搜索和分页,具体包括下面的三个步骤:
定义实体类如下:
import lombok.Data;
@Data
public class RequestParams
private String key ;
private Integer page ;
private Integer size ;
private String sortBy ;
import java.util.List;
@Data
public class PageResult
private Long total ;
private List<HotelDoc> hotels ;
public PageResult()
public PageResult(Long total, List<HotelDoc> hotels)
this.total = total;
this.hotels = hotels;
定义表现层controller,接收前端页面请求,调用业务层方法实现查找和分页。
import cn.itcast.hotel.pojo.PageResult;
import cn.itcast.hotel.pojo.RequestParams;
import cn.itcast.hotel.service.IHotelService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
@RequestMapping("/hotel")
public class HotelController
@Autowired
private IHotelService iHotelService ;
@PostMapping("/list")
public PageResult search(@RequestBody RequestParams params)
return iHotelService.search(params) ;
在业务层定义接口和实现类,如下,在实现类中完成查询和分页业务,将解析得到的结果进行封装后返回。其中注入的client在启动类中注解为bean,变成配置类交给spring管理。
import cn.itcast.hotel.mapper.HotelMapper;
import cn.itcast.hotel.pojo.Hotel;
import cn.itcast.hotel.pojo.HotelDoc;
import cn.itcast.hotel.pojo.PageResult;
import cn.itcast.hotel.pojo.RequestParams;
import cn.itcast.hotel.service.IHotelService;
import com.alibaba.fastjson.JSON;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
@Service
public class HotelService extends ServiceImpl<HotelMapper, Hotel> implements IHotelService
@Autowired
private RestHighLevelClient client ;
@Override
public PageResult search(RequestParams params)
try
//1.创建request对象
SearchRequest searchRequest = new SearchRequest("hotel");
//2.准备DSL
String key = params.getKey();
if (key == null || "".equals(key))
searchRequest.source().query(QueryBuilders.matchAllQuery());
else
searchRequest.source().query(QueryBuilders.matchQuery("name", key));
//分页
int page = params.getPage();
int pageSize = params.getSize();
searchRequest.source().from((page - 1) * pageSize).size(pageSize);
//3.发送请求
SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
//4.解析响应
return handleResponse(response);
catch (IOException e)
throw new RuntimeException(e) ;
private PageResult handleResponse(SearchResponse response)
//4.解析响应
SearchHits searchHits = response.getHits() ;
//获取总条数
long total = searchHits.getTotalHits().value ;
//获取文档数组
SearchHit[] searchHits1 = searchHits.getHits() ;
//遍历
List<HotelDoc> list = new ArrayList<>() ;
for(SearchHit searchHits2 : searchHits1)
String json = searchHits2.getSourceAsString() ;
//反序列化
HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
list.add(hotelDoc) ;
return new PageResult(total, list) ;
4.2、条件过滤
添加品牌、价格、星级、城市等条件过滤功能,具体实现步骤如下:
修改实体类,添加一些参数,如下:
import lombok.Data;
@Data
public class RequestParams
private String key ;
private Integer page ;
private Integer size ;
private String sortBy ;
private String city ;
private String brand ;
private String starName ;
private Integer minPrice ;
private Integer maxPrice ;
然后在业务层的实现方法中进行条件过滤即可,具体如下:
import cn.itcast.hotel.mapper.HotelMapper;
import cn.itcast.hotel.pojo.Hotel;
import cn.itcast.hotel.pojo.HotelDoc;
import cn.itcast.hotel.pojo.PageResult;
import cn.itcast.hotel.pojo.RequestParams;
import cn.itcast.hotel.service.IHotelService;
import com.alibaba.fastjson.JSON;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
@Service
public class HotelService extends ServiceImpl<HotelMapper, Hotel> implements IHotelService
@Autowired
private RestHighLevelClient client ;
@Override
public PageResult search(RequestParams params)
try
//1.创建request对象
SearchRequest searchRequest = new SearchRequest("hotel");
buildBasicQuery(params, searchRequest);
//分页
int page = params.getPage();
int pageSize = params.getSize();
searchRequest.source().from((page - 1) * pageSize).size(pageSize);
//3.发送请求
SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
//4.解析响应
return handleResponse(response);
catch (IOException e)
throw new RuntimeException(e) ;
private void buildBasicQuery(RequestParams params, SearchRequest searchRequest)
//2.关键字搜索
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery() ;
String key = params.getKey();
if (key == null || "".equals(key))
boolQueryBuilder.must(QueryBuilders.matchAllQuery());
else
boolQueryBuilder.must(QueryBuilders.matchQuery("name", key));
//城市条件搜索
if(params.getCity() != null && !"".equals(params.getCity()))
boolQueryBuilder.filter(QueryBuilders.termQuery("city.keyword", params.getCity())) ;
//品牌条件搜索
if(params.getBrand() != null && !"".equals(params.getBrand()))
boolQueryBuilder.filter(QueryBuilders.termQuery("brand.keyword", params.getBrand())) ;
//星级条件搜索
if(params.getStarName() != null && !"".equals(params.getStarName()))
boolQueryBuilder.filter(QueryBuilders.termQuery("starName.keyword", params.getStarName())) ;
//价格条件搜索
if(params.getMinPrice() != null && params.getMaxPrice() != null)
boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").
gte(params.getMinPrice()).lte(params.getMaxPrice())) ;
searchRequest.source().query(boolQueryBuilder) ;
private PageResult handleResponse(SearchResponse response)
//4.解析响应
SearchHits searchHits = response.getHits() ;
//获取总条数
long total = searchHits.getTotalHits().value ;
//获取文档数组
SearchHit[] searchHits1 = searchHits.getHits() ;
//遍历
List<HotelDoc> list = new ArrayList<>() ;
for(SearchHit searchHits2 : searchHits1)
String json = searchHits2.getSourceAsString() ;
//反序列化
HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
list.add(hotelDoc) ;
return new PageResult(total, list) ;
4.3、附近的酒店距离升序排序
获取前端的位置数据,根据计算得到的位置升序排序,找出最近的酒店位置,如下:
import cn.itcast.hotel.mapper.HotelMapper;
import cn.itcast.hotel.pojo.Hotel;
import cn.itcast.hotel.pojo.HotelDoc;
import cn.itcast.hotel.pojo.PageResult;
import cn.itcast.hotel.pojo.RequestParams;
import cn.itcast.hotel.service.IHotelService;
import com.alibaba.fastjson.JSON;
import com.baomidou.mybatisplus.extension.service.impl.ServiceImpl;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.geo.GeoPoint;
import org.elasticsearch.common.unit.DistanceUnit;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.sort.SortBuilder;
import org.elasticsearch.search.sort.SortBuilders;
import org.elasticsearch.search.sort.SortOrder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
@Service
public class HotelService extends ServiceImpl<HotelMapper, Hotel> implements IHotelService
@Autowired
private RestHighLevelClient client ;
@Override
public PageResult search(RequestParams params)
try
//1.创建request对象
SearchRequest searchRequest = new SearchRequest("hotel");
buildBasicQuery(params, searchRequest);
//分页
int page = params.getPage();
int pageSize = params.getSize();
searchRequest.source().from((page - 1) * pageSize).size(pageSize);
//排序
String location = params.getLocation() ;
if(location != null && !"".equals(location))
searchRequest.source().sort(SortBuilders.
geoDistanceSort("location", new GeoPoint(location)).
order(SortOrder.ASC).unit(DistanceUnit.KILOMETERS)) ;
//3.发送请求
SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
//4.解析响应
return handleResponse(response);
catch (IOException e)
throw new RuntimeException(e) ;
private void buildBasicQuery(RequestParams params, SearchRequest searchRequest)
//2.关键字搜索
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery() ;
String key = params.getKey();
if (key == null || "".equals(key))
boolQueryBuilder.must(QueryBuilders.matchAllQuery());
else
boolQueryBuilder.must(QueryBuilders.matchQuery("name", key));
//城市条件搜索
if(params.getCity() != null && !"".equals(params.getCity()))
boolQueryBuilder.filter(QueryBuilders.termQuery("city.keyword", params.getCity())) ;
//品牌条件搜索
if(params.getBrand() != null && !"".equals(params.getBrand()))
boolQueryBuilder.filter(QueryBuilders.termQuery("brand.keyword", params.getBrand())) ;
//星级条件搜索
if(params.getStarName() != null && !"".equals(params.getStarName()))
boolQueryBuilder.filter(QueryBuilders.termQuery("starName.keyword", params.getStarName())) ;
//价格条件搜索
if(params.getMinPrice() != null && params.getMaxPrice() != null)
boolQueryBuilder.filter(QueryBuilders.rangeQuery("price").
gte(params.getMinPrice()).lte(params.getMaxPrice())) ;
searchRequest.source().query(boolQueryBuilder) ;
private PageResult handleResponse(SearchResponse response)
//4.解析响应
SearchHits searchHits = response.getHits() ;
//获取总条数
long total = searchHits.getTotalHits().value ;
//获取文档数组
SearchHit[] searchHits1 = searchHits.getHits() ;
//遍历
List<HotelDoc> list = new ArrayList<>() ;
for(SearchHit searchHits2 : searchHits1)
String json = searchHits2.getSourceAsString() ;
//反序列化
HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class) ;
//获取排序值
Object [] objects = searchHits2.getSortValues() ;
if(objects.length > 0)
Object value = objects[0] ;
hotelDoc.setDistance(value);
list.add(hotelDoc) ;
return new PageResult(total, list) ;
以上是关于微服务实用篇6-分布式搜索elasticsearch篇2的主要内容,如果未能解决你的问题,请参考以下文章
实用篇SpringCloud+RabbitMQ+Docker+Redis+搜索+分布式,系统详解springcloud分布式