es多字段聚合,聚合后分页,聚合后having等操作

Posted 2023-01-14 CatServer

ES使用场景

es聚合的应用场景是很多,不过,有时候,还会有多个字段聚合,聚合后分页,聚合后having的需求

多个字段聚合

		//构造查询对象
		SearchRequest baseSubOrderIndexRequest = CloudBaseQueryBuilder.getBaseSubOrderIndexRequest();
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        BoolQueryBuilder boolQuery = CloudBaseQueryBuilder.getBaseSubOrderDimensionBoolQuery(vo);
        // 采购方式
        if(vo.getBugTypes()!=null && vo.getBugTypes().size()>0)
            boolQuery.filter(QueryBuilders.termsQuery("buyType",vo.getBugTypes()));
        
        if(StringUtils.isNotEmpty(vo.getSupplierName()))
            boolQuery.filter(QueryBuilders.termQuery("supplierName",vo.getSupplierName()));
        
        if(StringUtils.isNotEmpty(vo.getUnitName()))
            boolQuery.filter(QueryBuilders.termQuery("unitName",vo.getUnitName()));
        
        sourceBuilder.query(boolQuery);
		//最外层的聚合
		TermsAggregationBuilder percentageInfo = AggregationBuilders.terms("percentage_info").field(vo.getType().equals("1")?"unitName.keyword":"supplierName.keyword").size(Integer.MAX_VALUE).order(BucketOrder.aggregation(getTurnOverSumName(),false));
		//在第一层聚合的基础上
		percentageInfo.subAggregation(  AggregationBuilders.terms("provinceName")
                    .field("provinceName.keyword")
                    .size(1));
            percentageInfo.subAggregation(  AggregationBuilders.terms("cityName")
                    .field("cityName.keyword")
                    .size(1));
            percentageInfo.subAggregation(  AggregationBuilders.terms("areaName")
                    .field("areaName.keyword")
                    .size(1));
            percentageInfo.subAggregation(  AggregationBuilders.terms("cgUnitIndustry")
                    .field("cgUnitIndustry.keyword")
                    .size(1));

聚合后分页

Elasticsearch 聚合后是不支持分页的
性能角度——聚合分页会在大量的记录中产生性能问题。
正确性角度——聚合的文档计数不准确.
但是,公司的一些要求,要进行分页,大致有两种思路

1. 聚合排序后的结果放到list中,利用list进行分页,这种方式,每次查询都是查询全部,并且所有的数据都放到内存中,是有性能问题的,适合数据量小的场景
2. 利用Elasticsearch 的BucketSortPipelineAggregationBuilder类,这个类时用来构造
   bucket_sort管道聚合来实现分页
   中文释义：一个父管道聚合，对其父多桶聚合的桶进行排序。可以指定零个或多个排序字段以及相应的排序顺序。每个桶可以根据它的_key、_count或它的子集合进行排序。此外，可以设置参数from和size，以便截断结果桶。

	    // 聚合分页
        if ((!ObjectUtils.isEmpty(vo.getPageNo())) && (!ObjectUtils.isEmpty(vo.getPageSize()))) 
            percentageInfo.subAggregation(new BucketSortPipelineAggregationBuilder("bucket_field", Arrays.asList(new FieldSortBuilder(getTurnOverSumName()).order(SortOrder.DESC))).from((vo.getPageNo() - 1) * vo.getPageSize()).size(vo.getPageSize()));
        
        //分组总条数
        List<? extends Terms.Bucket> totalBuckets = ((Terms) masterAggregations.get("percentage_info_total")).getBuckets();
        //分组总条数
        totalBuckets.size();
        percentageInfo.subAggregation(AggregationBuilders
                .sum("turnOverSum")
                .field("totalPrice"));

聚合后having

聚合之后还不能满足我们的需求,比如要过滤出价格大于某个值的数据
Elasticsearch 提供了bucketSelector的写法

	
		if(StringUtils.isNotEmpty(vo.getTotalPrice()))
            //(1) 设置脚本
            Script script = new Script("params.turnOverSum >="+vo.getTotalPrice());

            //(2) 声明BucketPath，用于后面的bucket筛选
            Map<String, String> bucketsPathsMap = new HashMap<>(2);
            bucketsPathsMap.put("turnOverSum", "turnOverSum");

            //(3) 构建bucket selector 实现having条件筛选过滤
            BucketSelectorPipelineAggregationBuilder bs =
                    PipelineAggregatorBuilders.bucketSelector("having", bucketsPathsMap,script);
            percentageInfo.subAggregation(bs);
        

最后

		sourceBuilder.aggregation(percentageInfo);
        baseSubOrderIndexRequest.source(sourceBuilder);

        SearchResponse response = elasticsearchClient.search(baseSubOrderIndexRequest, RequestOptions.DEFAULT);
        log.debug(()->sourceBuilder.toString());
        List<DataReportDTO> list=new ArrayList<>();
        Aggregations masterAggregations = response.getAggregations();

分页参考链接
having参考链接