在作业 ETL-Load 中执行步骤 ETL-file-load 时遇到错误

Posted 2023-02-27

【中文标题】在作业 ETL-Load 中执行步骤 ETL-file-load 时遇到错误

【英文标题】：Encountered an error executing step ETL-file-load in job ETL-Load

【发布时间】：2020-07-14 10:32:06

【问题描述】：

我在 Heroku 服务器上收到此错误，但它完全在本地工作。

这是控制器，它正在从表单中获取 CSV 文件并尝试将其存储在资源文件夹中。

@PostMapping(value = "/import-csv")
    public ModelAndView uploadCSV(HttpServletRequest request, HttpServletResponse response, @RequestParam("file") MultipartFile multipartFile) throws IOException, JobParametersInvalidException, JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException 
        logger.info("Entering into import-csv controller");
        int id = Integer.parseInt(request.getParameter("listId"));
        try 
            String listId = request.getParameter("listId");
            logger.info("id is : ",id);
            String path = new ClassPathResource("tempUpload/").getURL().getPath();
            logger.info("path : ",path);
            File fileToImport = new File(path + multipartFile.getOriginalFilename());
            logger.info("full path : ", fileToImport.toString());

            System.out.println("iside cotmt:" + "file:" + fileToImport.getAbsolutePath());


            Map<String, JobParameter> maps = new HashMap<>();
            maps.put("time", new JobParameter(System.currentTimeMillis()));
            maps.put("fullPathFileName", new JobParameter(fileToImport.getAbsolutePath()));
            maps.put("listId", new JobParameter(listId));
            JobParameters parameters = new JobParameters(maps);
            JobExecution jobExecution = jobLauncher.run(job, parameters);


            logger.info("JobExecution: " ,jobExecution.getStatus());

            logger.info("Batch is Running...");
            userService.setListofUserSave(listId);
            logger.info("Exiting from import-csv controller");


         catch (Exception e) 
            logger.error("Error while uploading csv file");
            e.printStackTrace();
        
        return new ModelAndView("redirect:/view?id=" + id);
    

这是获取资源文件夹中临时存储的CSV文件路径的项目阅读器。

    @Bean
    @Scope(value = "step", proxyMode = ScopedProxyMode.TARGET_CLASS)
    public FlatFileItemReader<User> itemReader(@Value("#jobParameters[fullPathFileName]") String pathToFile) 
        logger.info("file path inside spring batch config: ", pathToFile);
        FlatFileItemReader<User> flatFileItemReader = new FlatFileItemReader<>();
        flatFileItemReader.setResource(new FileSystemResource(pathToFile));
        flatFileItemReader.setName("CSV-Reader");
        flatFileItemReader.setLinesToSkip(1);
        flatFileItemReader.setLineMapper(
        new DefaultLineMapper<User>() 
            setLineTokenizer(new DelimitedLineTokenizer() 
                setNames(new String[]"name", "email");
            );
            setFieldSetMapper(new BeanWrapperFieldSetMapper<User>() 
                setTargetType(User.class);
            );
        ) ;
        return flatFileItemReader;
    

但它在日志中显示此错误：

Apr 02 04:11:31 demo-letter app/web.1 2020-04-02 11:11:30.802  INFO 4 --- [o-57213-exec-10] c.e.demo.Configration.SpringBatchConfig  : file path inside spring batch config: /app/file:/app/target/demo-0.0.1-SNAPSHOT.jar!/BOOT-INF/classes!/tempUpload/csvtesting.csv
Apr 02 04:11:31 demo-letter app/web.1 2020-04-02 11:11:30.805 ERROR 4 --- [o-57213-exec-10] o.s.batch.core.step.AbstractStep         : Encountered an error executing step ETL-file-load in job ETL-Load
Apr 02 04:11:31 demo-letter app/web.1 
Apr 02 04:11:31 demo-letter app/web.1 org.springframework.batch.item.ItemStreamException: Failed to initialize the reader
Apr 02 04:11:31 demo-letter app/web.1   at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:153) ~[spring-batch-infrastructure-4.2.1.RELEASE.jar!/:4.2.1.RELEASE]
Apr 02 04:11:31 demo-letter app/web.1   at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader$$FastClassBySpringCGLIB$$ebb633d0.invoke(<generated>) ~[spring-batch-infrastructure-4.2.1.RELEASE.jar!/:4.2.1.RELEASE]
Apr 02 04:11:31 demo-letter app/web.1   at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.2.5.RELEASE.jar!/:5.2.5.RELEASE]
Apr 02 04:11:31 demo-letter app/web.1   at 

org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:771) ~[spring-aop-5.2.5.RELEASE.jar!/:5.2.5.RELEASE]
.
.
.
Apr 02 04:11:31 demo-letter app/web.1   at java.lang.Thread.run(Thread.java:748) [na:1.8.0_242-heroku]
Apr 02 04:11:31 demo-letter app/web.1 Caused by: java.lang.IllegalStateException: Input resource must exist (reader is in 'strict' mode): file [/app/file:/app/target/demo-0.0.1-SNAPSHOT.jar!/BOOT-INF/classes!/tempUpload/csvtesting.csv]
Apr 02 04:11:31 demo-letter app/web.1   at org.springframework.batch.item.file.FlatFileItemReader.doOpen(FlatFileItemReader.java:257) ~[spring-batch-infrastructure-4.2.1.RELEASE.jar!/:4.2.1.RELEASE]
Apr 02 04:11:31 demo-letter app/web.1   at org.springframework.batch.item.support.AbstractItemCountingItemStreamItemReader.open(AbstractItemCountingItemStreamItemReader.java:150) ~[spring-batch-infrastructure-4.2.1.RELEASE.jar!/:4.2.1.RELEASE]
Apr 02 04:11:31 demo-letter app/web.1   ... 84 common frames omitted

【参考方案1】：

查看错误：

Apr 02 04:11:31 demo-letter app/web.1 Caused by: java.lang.IllegalStateException: Input resource must exist (reader is in 'strict' mode): file [/app/file:/app/target/demo-0.0.1-SNAPSHOT.jar!/BOOT-INF/classes!/tempUpload/csvtesting.csv

我相信您的资源是从类路径中解析出来的，而不是在文件系统上。你的问题在这里：

String path = new ClassPathResource("tempUpload/").getURL().getPath();

应该是：

String path = new FileSystemResource("/absolute/path/to/tempUpload/").getURL().getPath();