使用Spring启动的DB2和MongoDB的Spring批处理

Posted

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了使用Spring启动的DB2和MongoDB的Spring批处理相关的知识，希望对你有一定的参考价值。

我需要读取DB2数据库并使用Spring Batch将数据复制到mongoDB。因为我要将数据写入mongoDB，所以我不需要事务。我想将mongoDB中的元数据表脚本保留在DB2中，但是我找不到mongoDB的元数据表脚本。在服务器启动时，spring boot期望Db2中的batch_job_instance表而不是mongoDB。我将mongoDB注释为主要但仍然是抛出错误。

有人可以帮我这个。提前致谢。

mongo config.Java:

@Configuration
@EnableMongoRepositories("com.test.mongodb")
public class MongoConfig extends AbstractMongoConfiguration {

    private final Logger log = LoggerFactory.getLogger(MongoConfig.class);

    @Value("${spring.data.mongodb.host}")
    private String host;

    @Value("${spring.data.mongodb.port}")
    private Integer port;

    @Value("${spring.data.mongodb.username}")
    private String username;

    @Value("${spring.data.mongodb.database}")
    private String database;

    @Value("${spring.data.mongodb.password}")
    private String password;

    @Bean
    public ValidatingMongoEventListener validatingMongoEventListener() {
        return new ValidatingMongoEventListener(validator());
    }
    @Bean
    public LocalValidatorFactoryBean validator() {
        return new LocalValidatorFactoryBean();
    }

    @Override
    public String getDatabaseName() {
        return database;
    }

    @Override
    @Bean
    public Mongo mongo() throws Exception {
        return new MongoClient(singletonList(new ServerAddress(host, port)),
                singletonList(MongoCredential.createCredential(username, database, password.toCharArray())));
    }

    @Override
    @Bean
    @Primary
    @ConfigurationProperties(prefix = "spring.data.mongodb")
    public MongoTemplate mongoTemplate() throws Exception {
        return new MongoTemplate(mongo(), database);
    }
}

application.properties:

# DB2
spring.datasource.jndi-name=java:jboss/datasources/Db2XaDsn

# Mongo DB
spring.data.mongodb.host=localhost
spring.data.mongodb.port=27017
spring.data.mongodb.username=admin
spring.data.mongodb.password=admin
spring.data.mongodb.database=test

批次类：

@Configuration
@EnableBatchProcessing
public class ItemBatch {

    @Autowired
    private JobBuilderFactory jobBuilderFactory;

    @Autowired
    private StepBuilderFactory stepBuilderFactory;

    @Autowired
    EntityManagerFactory entityManagerFactory;

    @Autowired
    MongoTemplate mongoTemplate;

    @Bean
    public Job readDB2() {
        return jobBuilderFactory.get("readDB2").start(step1()).build();
    }

    @Bean
    public Step step1() {
        return stepBuilderFactory.get("step1").<com.model.db2.Item, Item>chunk(200).reader(reader())
                .writer(writer()).build();
    }

    @Bean
    public ItemReader<com.model.db2.Item> reader() {
        JpaPagingItemReader<com.model.db2.Item> reader = new JpaPagingItemReader<>();
        reader.setQueryString("select i from Item i");
        reader.setEntityManagerFactory(entityManagerFactory);
        return reader;
    }

    @Bean
    public MongoItemWriter<Item> writer() {
        MongoItemWriter<Item> writer = new MongoItemWriter<>();
        try {
            writer.setTemplate(mongoTemplate);
        } catch (Exception e) {
            e.printStackTrace();
        }
        writer.setCollection("item");
        return writer;
    }
}

错误：

00:44:39,484 ERROR [org.jboss.msc.service.fail] (ServerService Thread Pool -- 72) MSC000001: Failed to start service jboss.undertow.deployment.default-server.default-host./itemapi: org.jboss.msc.service.StartException in service jboss.undertow.deployment.default-server.default-host./itemapi: java.lang.RuntimeException: java.lang.IllegalStateException: Failed to execute CommandLineRunner
    at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:85)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
    at org.jboss.threads.JBossThread.run(JBossThread.java:320)
Caused by: java.lang.RuntimeException: java.lang.IllegalStateException: Failed to execute CommandLineRunner
    at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:231)
    at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:100)
    at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:82)
    ... 6 more
Caused by: java.lang.IllegalStateException: Failed to execute CommandLineRunner
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:735)
    at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:716)
    at org.springframework.boot.SpringApplication.afterRefresh(SpringApplication.java:703)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:304)
    at org.springframework.boot.web.support.SpringBootServletInitializer.run(SpringBootServletInitializer.java:154)
    at org.springframework.boot.web.support.SpringBootServletInitializer.createRootApplicationContext(SpringBootServletInitializer.java:134)
    at org.springframework.boot.web.support.SpringBootServletInitializer.onStartup(SpringBootServletInitializer.java:87)
    at org.springframework.web.SpringServletContainerInitializer.onStartup(SpringServletContainerInitializer.java:169)
    at io.undertow.servlet.core.DeploymentManagerImpl.deploy(DeploymentManagerImpl.java:184)
    ... 8 more
Caused by: org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [SELECT JOB_INSTANCE_ID, JOB_NAME from BATCH_JOB_INSTANCE where JOB_NAME = ? order by JOB_INSTANCE_ID desc]; nested exception is com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-204, SQLSTATE=42704, SQLERRMC=TEST.BATCH_JOB_INSTANCE, DRIVER=4.18.60
    at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:231)
    at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73)
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:649)
    at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:684)
    at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:716)
    at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:726)
    at org.springframework.batch.core.repository.dao.JdbcJobInstanceDao.getJobInstances(JdbcJobInstanceDao.java:230)
    at org.springframework.batch.core.explore.support.SimpleJobExplorer.getJobInstances(SimpleJobExplorer.java:173)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
    at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
    at com.sun.proxy.$Proxy302.getJobInstances(Unknown Source)
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.getNextJobParameters(JobLauncherCommandLineRunner.java:131)
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:212)
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:231)
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:123)
    at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:117)
    at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:732)
    ... 16 more
Caused by: com.ibm.db2.jcc.am.SqlSyntaxErrorException: DB2 SQL Error: SQLCODE=-204, SQLSTATE=42704, SQLERRMC=TEST.BATCH_JOB_INSTANCE, DRIVER=4.18.60
    at com.ibm.db2.jcc.am.kd.a(kd.java:747)
    at com.ibm.db2.jcc.am.kd.a(kd.java:66)
    at com.ibm.db2.jcc.am.kd.a(kd.java:135)
    at com.ibm.db2.jcc.am.bp.c(bp.java:2788)
    at com.ibm.db2.jcc.am.bp.d(bp.java:2776)
    at com.ibm.db2.jcc.am.bp.a(bp.java:2209)
    at com.ibm.db2.jcc.am.cp.a(cp.java:7886)
    at com.ibm.db2.jcc.t4.bb.h(bb.java:141)
    at com.ibm.db2.jcc.t4.bb.b(bb.java:41)
    at com.ibm.db2.jcc.t4.p.a(p.java:32)
    at com.ibm.db2.jcc.t4.vb.i(vb.java:145)
    at com.ibm.db2.jcc.am.bp.kb(bp.java:2178)
    at com.ibm.db2.jcc.am.cp.xc(cp.java:3686)
    at com.ibm.db2.jcc.am.cp.b(cp.java:4493)
    at com.ibm.db2.jcc.am.cp.kc(cp.java:767)
    at com.ibm.db2.jcc.am.cp.executeQuery(cp.java:732)
    at org.jboss.jca.adapters.jdbc.WrappedPreparedStatement.executeQuery(WrappedPreparedStatement.java:504)
    at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:692)
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:633)
    ... 38 more
答案

回答我自己的问题。我可以使用以下配置使用没有元数据表的spring批处理：

@Configuration
public class SpringBatchInMemoryConfiguration {

    @Bean
    public ResourcelessTransactionManager transactionManager() {
        return new ResourcelessTransactionManager();
    }

    @Bean
    public MapJobRepositoryFactoryBean mapJobRepositoryFactory(ResourcelessTransactionManager txManager)
            throws Exception {
        MapJobRepositoryFactoryBean factory = new MapJobRepositoryFactoryBean(txManager);
        factory.afterPropertiesSet();
        return factory;
    }

    @Bean
    public JobRepository jobRepository(MapJobRepositoryFactoryBean factory) throws Exception {
        return factory.getObject();
    }

    @Bean
    public JobExplorer jobExplorer(MapJobRepositoryFactoryBean factory) {
        return new SimpleJobExplorer(factory.getJobInstanceDao(), factory.getJobExecutionDao(),
                factory.getStepExecutionDao(), factory.getExecutionContextDao());
    }

    @Bean
    public SimpleJobLauncher jobLauncher(JobRepository jobRepository) throws Exception {
        SimpleJobLauncher launcher = new SimpleJobLauncher();
        launcher.setJobRepository(jobRepository);
        launcher.afterPropertiesSet();
        return launcher;
    }
}

以上是关于使用Spring启动的DB2和MongoDB的Spring批处理的主要内容，如果未能解决你的问题，请参考以下文章

Spring Boot 测试不启动上下文或加载依赖项

db2中的SP是不是等待触发器执行

如何配置两个实例mongodb使用spring boot和spring data

Spring Boot 应用程序和 MongoDB 在不同的 Docker 容器上。应用程序未启动

使用 mongoDB 启动 H2 DB

Spring saml - 在SP上启动登录时如何记住请求参数，并在IdP响应后处理它们