将 GCP 订阅者与 Spring Boot 集成会导致内存致命错误

Posted

技术标签:

【中文标题】将 GCP 订阅者与 Spring Boot 集成会导致内存致命错误【英文标题】:Integrating GCP subscriber with Spring boot giving memory fatal error 【发布时间】:2021-02-07 03:08:30 【问题描述】:

我正在尝试将 GCP pub-sub 与我的 spring boot 项目集成,并在日志中出现以下内存致命错误并且应用程序崩溃

2020-10-24 10:35:51.032 INFO  GcpContextAutoConfiguration  The default project ID is <project-id>
2020-10-24 10:35:51.111 INFO  DefaultCredentialsProvider  Default credentials provider for service account <credentials>@<domain>
2020-10-24 10:35:51.111 INFO  DefaultCredentialsProvider  Scopes in use by default credentials: [https://www.googleapis.com/auth/pubsub, https://www.googleapis.com/auth/spanner.admin,https://www.googleapis.com/auth/spanner.data,http://www.googleapis.com/auth/datastore,https://www.googleapis.com/auth/sqlservice.admin,https://www.googleapis.com/auth/devstorage.read_only,https://www.googleapis.com/auth/devstorage.read_write, https://www.googleapis.com/auth/cloudruntimeconfig,https://www.googleapis.com/auth/trace.append,https://www.googleapis.com/auth/cloud-platform, https://www.googleapis.com/auth/cloud-vision,https://www.googleapis.com/auth/bigquery,https://www.googleapis.com/auth/monitoring.write]
2020-10-24 10:35:51.128 INFO  ThreadPoolTaskScheduler  Initializing ExecutorService 'pubsubPublisherThreadPool'
2020-10-24 10:35:51.209 INFO  ThreadPoolTaskScheduler  Initializing ExecutorService 'pubsubSubscriberThreadPool'
2020-10-24 10:35:51.227 INFO  ThreadPoolTaskExecutor  Initializing ExecutorService 'pubSubAcknowledgementExecutor'
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x000000012a1c7b30, pid=49538, tid=0x0000000000001203
#
# JRE version: OpenJDK Runtime Environment (8.0_232-b09) (build 1.8.0_232-b09)
# Java VM: OpenJDK 64-Bit Server VM (25.232-b09 mixed mode bsd-amd64 compressed oops)
# Problematic frame:
# C  0x000000012a1c7b30
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try 
"ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# <path-where-log-is-generated>/hs_err_pid49538.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
#

错误报告可以在这里找到hs_err_pid49538.log 这是完整的pom

    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.2.4.RELEASE</version>
    </parent>
    <groupId><some-group-id></groupId>
    <artifactId><some-artifact-id></artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name><some-name></name>
    <properties>
        <java.version>1.8</java.version>
        <jacoco.version>0.8.2</jacoco.version>
        <code.coverage>0%</code.coverage>
        <jmockit.version>1.38</jmockit.version>
    </properties>

    <dependencies>

        <dependency>
            <groupId>org.jmockit</groupId>
            <artifactId>jmockit</artifactId>
            <version>$jmockit.version</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
            <exclusions>
                <exclusion>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-starter-logging</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>

        <dependency>
            <groupId>io.springfox</groupId>
            <artifactId>springfox-swagger2</artifactId>
            <version>2.9.2</version>
        </dependency>
        <dependency>
            <groupId>io.springfox</groupId>
            <artifactId>springfox-swagger-ui</artifactId>
            <version>2.9.2</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <version>2.2.4.RELEASE</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-sleuth</artifactId>
            <version>2.2.3.RELEASE</version>
        </dependency>

        <!-- https://mvnrepository.com/artifact/org.apache.commons/commons-lang3 -->
        <dependency>
            <groupId>org.apache.commons</groupId>
            <artifactId>commons-lang3</artifactId>
        </dependency>

        <dependency>
            <groupId>com.microsoft.azure</groupId>
            <artifactId>spring-cloud-azure-servicebus-topic-stream-binder</artifactId>
            <version>1.1.0.RC5</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-mongodb</artifactId>
        </dependency>

        <dependency>
            <groupId>com.microsoft.azure</groupId>
            <artifactId>applicationinsights-logging-logback</artifactId>
            <version>2.3.1</version>
        </dependency>

        <dependency>
            <groupId>org.apache.httpcomponents</groupId>
            <artifactId>httpclient</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-gcp-starter-pubsub</artifactId>
            <version>1.2.5.RELEASE</version>
        </dependency>


    </dependencies>

    <build>
        <plugins>
            <plugin>

                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>

            </plugin>
            <plugin>
                <groupId>org.jacoco</groupId>
                <artifactId>jacoco-maven-plugin</artifactId>
                <version>$jacoco.version</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>prepare-agent</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>jacoco-report</id>
                        <phase>test</phase>
                        <goals>
                            <goal>report</goal>
                        </goals>
                    </execution>
                    <!-- Add this checking -->
                    <execution>
                        <id>jacoco-check</id>
                        <goals>
                            <goal>check</goal>
                        </goals>
                        <configuration>
                            <rules>
                                <rule>
                                    <element>BUNDLE</element>
                                    <limits>
                                        <limit>
                                            <counter>INSTRUCTION</counter>
                                            <value>COVEREDRATIO</value>
                                            <minimum>$code.coverage</minimum>
                                        </limit>
                                    </limits>
                                </rule>
                            </rules>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <configuration>
                    <argLine>-javaagent:$settings.localRepository/org/jmockit/jmockit/$jmockit.version/jmockit-$jmockit.version.jar</argLine>
                    <useSystemClassLoader>true</useSystemClassLoader>
                </configuration>
            </plugin>

        </plugins>
    </build>
</project>

这些是我最后完成的以下代码更改

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.cloud.gcp.pubsub.core.PubSubTemplate;
import org.springframework.cloud.gcp.pubsub.integration.AckMode;
import org.springframework.cloud.gcp.pubsub.integration.inbound.PubSubInboundChannelAdapter;
import org.springframework.cloud.gcp.pubsub.support.BasicAcknowledgeablePubsubMessage;
import org.springframework.cloud.gcp.pubsub.support.GcpPubSubHeaders;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.integration.annotation.ServiceActivator;
import org.springframework.integration.channel.DirectChannel;
import org.springframework.messaging.MessageChannel;
import org.springframework.messaging.handler.annotation.Header;

import <package>.OrderEventObject;

import lombok.extern.slf4j.Slf4j;

@Configuration
@Slf4j
public class GCPReceiverConfiguration 

    @Autowired
    private PropertiesConfig config;

    @Bean
    public DirectChannel pubSubInputChannel() 
        return new DirectChannel();
    

    @Bean
    public PubSubInboundChannelAdapter messageChannelAdapter(
            @Qualifier("pubSubInputChannel") MessageChannel inputChannel, PubSubTemplate pubSubTemplate) 
        PubSubInboundChannelAdapter adapter = new PubSubInboundChannelAdapter(pubSubTemplate, config.getGcpSubscriptionId());
        adapter.setOutputChannel(inputChannel);
        adapter.setAckMode(AckMode.MANUAL);
        adapter.setPayloadType(OrderEventObject.class);
        return adapter;
    

    @ServiceActivator(inputChannel = "pubSubInputChannel")
    public void messageReceiver(OrderEventObject payload,
            @Header(GcpPubSubHeaders.ORIGINAL_MESSAGE) BasicAcknowledgeablePubsubMessage message) 
        log.info("Message arrived! Payload: " + payload);
        message.ack();
    


在 application.properties 中所做的更改

spring.cloud.gcp.project-id=<project-id>
spring.cloud.gcp.subscription-id=<subscription-id>
spring.cloud.gcp.credentials.location=classpath:<credentials-file>.json

方法 config.getGcpSubscriptionId() 返回属性文件中存在的订阅 ID 以进行连接。

我已参考以下链接进行集成 Springs Documentation for GCP PubSub

我还注意到,即使我删除了 GCPReceiverConfiguration 类,这个错误仍然存​​在。一旦我添加了这个依赖,这个错误就会出现

<dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-gcp-starter-pubsub</artifactId>
            <version>1.2.5.RELEASE</version>
        </dependency>

我的操作系统详情

谁能帮我看看这意味着什么错误以及如何解决?

【问题讨论】:

这个问题正在 Spring Pub/Sub 集成的 GitHub 存储库上进行跟踪和解决:github.com/spring-guides/gs-messaging-gcp-pubsub/issues/29 【参考方案1】:

将此作为答案重新发布:此问题正在 Spring Pub/Sub 集成的 GitHub 存储库上进行跟踪和解决:https://github.com/spring-guides/gs-messaging-gcp-pubsub/issues/29

【讨论】:

以上是关于将 GCP 订阅者与 Spring Boot 集成会导致内存致命错误的主要内容,如果未能解决你的问题,请参考以下文章

spring bootredisspring boot 集成redis的发布订阅机制

将 Spring Data JPA 与 GCP Spanner 集成

无法将 Spring Boot 指标发布到 GCP 堆栈驱动程序

java——spring boot集成RabbitMQ——发布/订阅模式——应用场景

在 GCP PubSub 和 Spring Boot 中捕获发布错误

Spring Boot GCP:将 PubSub 应用程序部署到 App Engine 标准环境时出现“Google 凭据”异常