QueryDsl SpringData Jpa findAll 如何避免count()

Posted 2023-02-26

【中文标题】QueryDsl SpringData Jpa findAll 如何避免count()

【英文标题】：QueryDsl SpringData Jpa findAll how to avoid count()

【发布时间】：2016-09-12 06:29:31

【问题描述】：

我正在尝试将 QueryDSL 与 Spring Data JPA 一起使用，我想将 findAll 与分页一起使用，但始终执行计数，如果返回类型为 List 也是如此。 我不需要这个计数，因为它真的很慢，而且我可能会失去分页的好处。

有解决这个问题的办法吗？

这是count()，在 mysql 上大约需要 30 秒：

Mysql too slow on simple query between two tables

在任何情况下，我都不想为我需要的每个页面重复计数，此信息仅在第一次调用时需要。

【问题讨论】：

可以和我们分享一些代码吗？ count 查询通常是分页的一部分，以便您可以在 UI 中显示总页数/结果数。为什么在您的情况下 count 部分很慢？

同样的问题，我们有无限滚动，所以不需要总数。这是一个浪费的查询

【参考方案1】：

由于QueryDslPredicateExecutor 不支持将Slice 作为findAll(Predicate, Pageable) 的返回值返回，所以计数查询 似乎是不可避免的。但是您可以定义一个新的基本存储库接口并以不发出分页计数查询的方式实现findAll 方法。对于初学者，您应该定义一个接口，该接口将用作所有其他 Repositories 的基本接口：

/**
 * Interface for adding one method to all repositories.
 *
 * <p>The main motivation of this interface is to provide a way
 * to paginate list of items without issuing a count query
 * beforehand. Basically we're going to get one element more
 * than requested and form a @link Page object out of it.</p>
 */
@NoRepositoryBean
public interface SliceableRepository<T, ID extends Serializable>
        extends JpaRepository<T, ID>,
        QueryDslPredicateExecutor<T> 

    Page<T> findAll(Predicate predicate, Pageable pageable);

然后，像这样实现这个接口：

public class SliceableRepositoryImpl<T, ID extends Serializable>
        extends QueryDslJpaRepository<T, ID>
        implements SliceableRepository<T, ID> 
    private static final EntityPathResolver DEFAULT_ENTITY_PATH_RESOLVER = SimpleEntityPathResolver.INSTANCE;
    private final EntityPath<T> path;
    private final PathBuilder<T> builder;
    private final Querydsl querydsl;

    public SliceableRepositoryImpl(JpaEntityInformation<T, ID> entityInformation, EntityManager entityManager) 
        super(entityInformation, entityManager);
        path = DEFAULT_ENTITY_PATH_RESOLVER.createPath(entityInformation.getJavaType());
        this.builder = new PathBuilder<>(path.getType(), path.getMetadata());
        this.querydsl = new Querydsl(entityManager, builder);
    

    @Override
    public Page<T> findAll(Predicate predicate, Pageable pageable) 
        int oneMore = pageable.getPageSize() + 1;
        JPQLQuery query = createQuery(predicate)
                .offset(pageable.getOffset())
                .limit(oneMore);

        Sort sort = pageable.getSort();
        query = querydsl.applySorting(sort, query);

        List<T> entities = query.list(path);

        int size = entities.size();
        if (size > pageable.getPageSize())
            entities.remove(size - 1);

        return new PageImpl<>(entities, pageable, pageable.getOffset() + size);
    

基本上，此实现将获取比请求大小多一个的元素，并将结果用于构造Page。然后你应该告诉 Spring Data 使用这个实现作为存储库基类：

@EnableJpaRepositories(repositoryBaseClass = SliceableRepositoryImpl.class)

最后将SliceableRepository 扩展为您的基本接口：

public SomeRepository extends SliceableRepository<Some, SomeID> 

【讨论】：

感谢您的回答，它解决了这个问题，但最后似乎总是需要对 querydsl + Spring Data JPA 进行某种优化，我一直在寻找一种更标准化的方法。再次感谢

谢谢。请记住 QueryDslPredicateExecutor 现在是 SpringBoot 2 中的 QuerydslPredicateExecutor

使用 Mongo 时如何做到这一点？

【参考方案2】：

仅供参考，有一个 spring jira 问题：

https://jira.spring.io/browse/DATAJPA-289

让我们为这项改进投票

【参考方案3】：

如果有人来到这里寻找如何在 Spring Data MongoDB 中实现与 Ali 为 Spring Data JPA 所做的相同的效果，这里是我的解决方案，以他为模型：

import java.io.Serializable;
import java.util.List;

import org.springframework.data.domain.Page;
import org.springframework.data.domain.PageImpl;
import org.springframework.data.domain.Pageable;
import org.springframework.data.domain.Slice;
import org.springframework.data.domain.Sort;
import org.springframework.data.domain.Sort.Order;
import org.springframework.data.mongodb.core.MongoOperations;
import org.springframework.data.mongodb.repository.query.MongoEntityInformation;
import org.springframework.data.mongodb.repository.support.QueryDslMongoRepository;
import org.springframework.data.mongodb.repository.support.SpringDataMongodbQuery;
import org.springframework.data.querydsl.EntityPathResolver;
import org.springframework.data.querydsl.QSort;
import org.springframework.data.querydsl.QueryDslPredicateExecutor;
import org.springframework.data.querydsl.SimpleEntityPathResolver;
import org.springframework.data.repository.core.EntityInformation;

import com.querydsl.core.types.EntityPath;
import com.querydsl.core.types.Expression;
import com.querydsl.core.types.OrderSpecifier;
import com.querydsl.core.types.Predicate;
import com.querydsl.core.types.dsl.PathBuilder;
import com.querydsl.mongodb.AbstractMongodbQuery;

/**
 * Custom extension of @link QueryDslMongoRepository that avoids unnecessary MongoDB "count"
 * operations
 * <p>
 * @link QueryDslPredicateExecutor#findAll(Predicate, Pageable) returns a @link Page at
 * potentially great expense because determining the @link Page's "totalElements" property
 * requires doing a potentially expensive MongoDB "count" operation. We'd prefer a "findAll"-like
 * method that returns a @link Slice (which doesn't have a "totalElements" property) but no such
 * method exists. See @link #findAll(Predicate, Pageable) for more details.
 *
 * @see https://github.com/spring-projects/spring-data-commons/issues/1011
 * @see https://***.com/questions/37254385/querydsl-springdata-jpa-findall-how-to-avoid-count
 */
public class MyQueryDslMongoRepository<T, ID extends Serializable> extends QueryDslMongoRepository<T, ID>
            implements MyAbstractRepository<T, ID> 
    private final PathBuilder<T> builder;
    private final EntityInformation<T, ID> entityInformation;
    private final MongoOperations mongoOperations;

    public BTQueryDslMongoRepository(MongoEntityInformation<T, ID> entityInformation, MongoOperations mongoOperations) 
        this(entityInformation, mongoOperations, SimpleEntityPathResolver.INSTANCE);
    

    public BTQueryDslMongoRepository(MongoEntityInformation<T, ID> entityInformation, MongoOperations mongoOperations,
            EntityPathResolver resolver) 
        super(entityInformation, mongoOperations, resolver);
        EntityPath<T> path = resolver.createPath(entityInformation.getJavaType());
        this.builder = new PathBuilder<T>(path.getType(), path.getMetadata());
        this.entityInformation = entityInformation;
        this.mongoOperations = mongoOperations;
    

    /**
     * An override of our superclass method to return a fake but cheaper-to-compute @link Page
     * that's adequate for our purposes.
     */
    @Override
    public Page<T> findAll(Predicate predicate, Pageable pageable) 
        int pageSize = pageable.getPageSize();
        SpringDataMongodbQuery<T> query = new SpringDataMongodbQuery<T>(mongoOperations, entityInformation.getJavaType())
                .where(predicate)
                .offset(pageable.getOffset())
                .limit(pageSize + 1);
        applySorting(query, pageable.getSort());

        List<T> entities = query.fetch();

        int numFetched = entities.size();
        if (numFetched > pageSize) 
            entities.remove(numFetched - 1);
        

        return new PageImpl<T>(entities, pageable, pageable.getOffset() + numFetched);
    

    /**
     * Applies the given @link Sort to the given @link MongodbQuery.
     * <p>
     * Copied from @link QueryDslMongoRepository
     */
    private AbstractMongodbQuery<T, SpringDataMongodbQuery<T>> applySorting(
            AbstractMongodbQuery<T, SpringDataMongodbQuery<T>> query, Sort sort) 

        if (sort == null) 
            return query;
        

        // TODO: find better solution than instanceof check
        if (sort instanceof QSort) 

            List<OrderSpecifier<?>> orderSpecifiers = ((QSort) sort).getOrderSpecifiers();
            query.orderBy(orderSpecifiers.toArray(new OrderSpecifier<?>[orderSpecifiers.size()]));

            return query;
        

        for (Order order : sort) 
            query.orderBy(toOrder(order));
        

        return query;
    
    /**
     * Transforms a plain @link Order into a QueryDsl specific @link OrderSpecifier.
     * <p>
     * Copied from @link QueryDslMongoRepository
     */
    @SuppressWarnings( "rawtypes", "unchecked" )
    private OrderSpecifier<?> toOrder(Order order) 

        Expression<Object> property = builder.get(order.getProperty());

        return new OrderSpecifier(
                order.isAscending() ? com.querydsl.core.types.Order.ASC : com.querydsl.core.types.Order.DESC, property);
    


@NoRepositoryBean
public interface MyAbstractRepository<T, ID extends Serializable> extends Repository<T, ID>,
        QueryDslPredicateExecutor<T> 

    @Override
    Page<T> findAll(Predicate predicate, Pageable pageable);

以上适用于 Spring Data MongoDB 1.10.23，但我认为可以进行修改以使其适用于更现代的版本。

【参考方案4】：

根据 Ali Dehghani 的回答，我们为 querydsl 4.2.1 构建了以下内容，因为 querydsl 语法在当前版本 4.x 中发生了变化

存储库接口：

import com.querydsl.core.types.EntityPath;
import com.querydsl.core.types.Predicate;
import org.springframework.data.domain.Pageable;
import org.springframework.data.domain.Slice;

public interface SliceableRepository<T> 

  Slice<T> findAllSliced(EntityPath<T> entityPath, Predicate predicate, Pageable pageable);

存储库实现： （必须命名为“Impl”）

import com.querydsl.core.types.EntityPath;
import com.querydsl.core.types.Predicate;
import com.querydsl.core.types.dsl.PathBuilder;
import com.querydsl.jpa.JPQLQuery;
import com.querydsl.jpa.impl.JPAQuery;
import com.querydsl.jpa.impl.JPAQueryFactory;
import java.util.List;
import javax.persistence.EntityManager;
import org.springframework.data.domain.Pageable;
import org.springframework.data.domain.Slice;
import org.springframework.data.domain.SliceImpl;
import org.springframework.data.jpa.repository.support.Querydsl;


public class SliceableRepositoryImpl<T> implements SliceableRepository<T> 

  private final EntityManager entityManager;
  private final JPAQueryFactory jpaQueryFactory;

  public SliceableRepositoryImpl(EntityManager entityManager) 
    this.entityManager = entityManager;
    this.jpaQueryFactory = new JPAQueryFactory(entityManager);
  

  @Override
  public Slice<T> findAllSliced(final EntityPath<T> entityPath, final Predicate predicate,
      final Pageable pageable) 

    final Querydsl querydsl = new Querydsl(entityManager,
        new PathBuilder<>(entityPath.getType(), entityPath.getMetadata()));

    final int oneMore = pageable.getPageSize() + 1;

    final JPAQuery<T> query = this.jpaQueryFactory.selectFrom(entityPath)
        .where(predicate)
        .offset(pageable.getOffset())
        .limit(oneMore);

    final JPQLQuery<T> querySorted = querydsl.applySorting(pageable.getSort(), query);

    final List<T> entities = querySorted.fetch();

    final int size = entities.size();
    // If there was one more result than requested from the pageable,
    // then the slice gets "hasNext"=true
    final boolean hasNext = size > pageable.getPageSize();
    if (hasNext) 
      entities.remove(size - 1);
    
    return new SliceImpl<>(entities, pageable, hasNext);
  

将新存储库用作其他存储库中的片段：

public SomeRepository extends JpaRepository<Some, Long>, SliceableRepository<Some> 

@EnableJpaRepositories(repositoryBaseClass = SliceableRepositoryImpl.class)不需要

然后像这样使用它：

public class MyService 
  @Autowired
  private final SomeRepository someRepository;

  public void doSomething() 
    Predicate predicate = ...
    Pageable pageable = ...
     // QSome is the generated model class from querydsl
    Slice<Some> result = someRepository.findAllSliced(QSome.some, predicate, pageable);