SpringBatch的ItemReader,ItenWriter是如何进行读写的
Posted 指导毕业设计Monkey
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了SpringBatch的ItemReader,ItenWriter是如何进行读写的相关的知识，希望对你有一定的参考价值。
reader: 读是每次读一条数据的，直到读取到chunk（）里面任务大小就会进行写入的操作。
ItemWriter: 这个是写，chunk大小来解决的，如果你的chunk（）设置的提交间隔太小会执行很多次。
通过xxl-job进行调度
在任务管理器中进行配置好那个JobHandler以后（当然你最先配置好的是你的执行器管理），然后进行调度的时候根据框架的执行流程来：
JobLauncher------->
Job---------->Flow
----->Step------------(ItemReader,ItemProcess,ItemWriter)
problem:
实现IteamReader接口的类读取到也数据，那个ItemWriter也获取到了它的List数据：
但是在实现了ItemWriter的这个类中的重载方法writer中进行日志打印输出的时候输出了很多次。
截图如下：
2022-04-18 10:48:31.408 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryAllDayCaseBySysdocCheck : 159 ] - ==> Preparing: select lcaa.clmnum from clm_baseinfo_lcaa lcaa LEFT JOIN clm_worktask_swflwti t on lcaa.clmnum = t.clmnum and t.status = 1 where lcaa.status = 1 and lcaa.procecode='C' and lcaa.inspectflag is null and t.submitdte BETWEEN CAST(date_format(date_add(now(), interval - 1 day),'%Y%m%d') AS SIGNED integer) AND CAST(date_format(date_add(now(), interval - 1 day),'%Y%m%d') AS SIGNED integer) -- perid 是有效的督查规则那边传递过来的外包商id
2022-04-18 10:48:31.420 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryAllDayCaseBySysdocCheck : 159 ] - ==> Parameters:
2022-04-18 10:48:31.422 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryAllDayCaseBySysdocCheck : 159 ] - <== Total: 4
2022-04-18 10:48:31.422 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskReader : 49 ] - 捞取每日理赔的的案件数目：4条
2022-04-18 10:48:31.434 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为：[700190151, 700189900, 700191218]
2022-04-18 10:48:31.434 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.435 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.436 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.447 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为：[700190151, 700189900, 700191218]
2022-04-18 10:48:31.448 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.448 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.449 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.453 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为：[700190151, 700189900, 700191218]
2022-04-18 10:48:31.454 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.454 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.455 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.466 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为：[700190151]
2022-04-18 10:48:31.466 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.466 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.467 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.471 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为：[700189900]
2022-04-18 10:48:31.472 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.472 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.473 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.478 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为：[700191218]
2022-04-18 10:48:31.478 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.478 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.479 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.494 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为：[700191213]
2022-04-18 10:48:31.494 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.495 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04-18 10:48:31.496 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - <== Total: 1
2022-04-18 10:48:31.506 [ SimpleAsyncTaskExecutor-6 ] [] - [ INFO ] [ c.x.j.c.j.inspectortask.InspectorDayTaskWriter : 71 ] - 写入的List集合为：[700191213]
2022-04-18 10:48:31.506 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Preparing: SELECT rulid,ruldes,rulcon,perid,autallcyc FROM clm_sysdoc_check where isvalide =1 and status = 1 and type=?
2022-04-18 10:48:31.507 [ SimpleAsyncTaskExecutor-6 ] [] - [ DEBUG ] [ c.x.j.c.m.m.C.queryInspectTypeIsValid : 159 ] - ==> Parameters: 2(String)
2022-04以上是关于SpringBatch的ItemReader,ItenWriter是如何进行读写的的主要内容，如果未能解决你的问题，请参考以下文章
Spring Batch：根据 itemReader 中的结果执行作业
Spring Batch：用于高容量和低延迟的 ItemReader 实现