mongdb报OutOfMemoryError: Java heap space或OutOfMemoryError: GC overhead limit exceeded的解决方法

Posted 2022-03-01 秋9

【现象】

failed; nested exception is java.lang.OutOfMemoryError: Java heap space] with root cause

java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:3210) ~[na:1.8.0_181]
    at java.util.Arrays.copyOf(Arrays.java:3181) ~[na:1.8.0_181]
    at java.util.ArrayList.grow(ArrayList.java:265) ~[na:1.8.0_181]
    at java.util.ArrayList.ensureExplicitCapacity(ArrayList.java:239) ~[na:1.8.0_181]
    at java.util.ArrayList.ensureCapacityInternal(ArrayList.java:231) ~[na:1.8.0_181]
    at java.util.ArrayList.add(ArrayList.java:462) ~[na:1.8.0_181]
    at org.springframework.data.mongodb.core.MongoTemplate.executeFindMultiInternal(MongoTemplate.java:2813) ~[spring-data-mongodb-3.2.6.jar:3.2.6]
    at org.springframework.data.mongodb.core.MongoTemplate.findAll(MongoTemplate.java:1779) ~[spring-data-mongodb-3.2.6.jar:3.2.6]
    at org.springframework.data.mongodb.core.MongoTemplate.findAll(MongoTemplate.java:1774) ~[spring-data-mongodb-3.2.6.jar:3.2.6]
java.lang.OutOfMemoryError: GC overhead limit exceeded
    at java.util.LinkedHashMap$LinkedKeySet.iterator(LinkedHashMap.java:543) ~[na:1.8.0_181]
    at java.util.HashSet.iterator(HashSet.java:173) ~[na:1.8.0_181]
    at java.util.AbstractCollection.toArray(AbstractCollection.java:137) ~[na:1.8.0_181]
    at java.util.ArrayList.addAll(ArrayList.java:581) ~[na:1.8.0_181]
    at org.springframework.context.event.AbstractApplicationEventMulticaster$CachedListenerRetriever.getApplicationListeners(AbstractApplicationEventMulticaster.java:462) ~[spring-context-5.3.12.jar:5.3.12]
    at org.springframework.context.event.AbstractApplicationEventMulticaster.getApplicationListeners(AbstractApplicationEventMulticaster.java:213) ~[spring-context-5.3.12.jar:5.3.12]
    at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:138) ~[spring-context-5.3.12.jar:5.3.12]
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:421) ~[spring-context-5.3.12.jar:5.3.12]
    at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:378) ~[spring-context-5.3.12.jar:5.3.12]
    at org.springframework.data.mongodb.core.MongoTemplate.maybeEmitEvent(MongoTemplate.java:2353) ~[spring-data-mongodb-3.2.6.jar:3.2.6]
    at org.springframework.data.mongodb.core.MongoTemplate$ReadDocumentCallback.doWith(MongoTemplate.java:3182) ~[spring-data-mongodb-3.2.6.jar:3.2.6]
    at org.springframework.data.mongodb.core.MongoTemplate.executeFindMultiInternal(MongoTemplate.java:2813) ~[spring-data-mongodb-3.2.6.jar:3.2.6]
    at org.springframework.data.mongodb.core.MongoTemplate.findAll(MongoTemplate.java:1779) ~[spring-data-mongodb-3.2.6.jar:3.2.6]
    at org.springframework.data.mongodb.core.MongoTemplate.findAll(MongoTemplate.java:1774) ~[spring-data-mongodb-3.2.6.jar:3.2.6]    

【原因】查询的数据量太大

【报错代码如下】

     public List<User> getAllUsers()
        return mongoTemplate.findAll(User.class);
   

一下子查询出1000多万条数据，导致的错误，代码逻辑修改为分页查询，例如

 public Map<String, Object> query(String userName,  int page,  int size) 

        // 条件
        Criteria criteria1 = Criteria.where("name").is(userName);
        Query query = new Query();
        if (userName!= null) 
            query.addCriteria(criteria1);
        

        // 数量
        long total = mongoTemplate.count(query, User.class);

        // 分页
        query.skip((page - 1) * size).limit(size);

        List<User> data = mongoTemplate.find(query, User.class);
        Map<String, Object> map = new HashMap<String, Object>();

        map.put("data", data);
        map.put("total", total);

        return map;