Spring数据JPA存储库saveAll不生成批量插入查询

Posted 2023-03-29

【中文标题】Spring数据JPA存储库saveAll不生成批量插入查询

【英文标题】：Spring data JPA repository saveAll is not generating bulk insert query

【发布时间】：2021-09-22 06:45:45

【问题描述】：

我正在使用 Spring data 2.2.8 和 hibernate 5.4.17.Final 版本。数据库是oracle 11g 当我使用 repository.saveAll(list) 保存数据时，它不会生成批量插入查询。 它为每条记录生成一个查询。

以下是实体序列相关信息

@Id
@GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "PAYG_RECONCILIATION_IDS")
@SequenceGenerator(name = "PAYG_RECONCILIATION_IDS", sequenceName = "PAYG_RECONCILIATION_IDS")
@Column
private Long paygReconciliationId;

下面是保存实体的代码

paygReconciliationRepository.saveAll(paygReconciliationMap.values());

下面是休眠设置

@Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() 
    LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
    em.setDataSource(dataSource);
    em.setPackagesToScan("com.cubic.cts.core.persistence.main.dto", "com.cubic.frm.persistence.main.dto");
    JpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
    em.setJpaVendorAdapter(vendorAdapter);
    Map<String, Object> properties = new HashMap<>();
    properties.put("hibernate.show_sql", true);
    properties.put("hibernate.generate_statistics", true);
    properties.put("hibernate.jdbc.batch_size", 50);
    properties.put("hibernate.order_inserts", true);
    properties.put("hibernate.order_updates", true);
    properties.put("hibernate.jdbc.batch_versioned_data", true);
    properties.put("hibernate.physical_naming_strategy", SpringPhysicalNamingStrategy.class.getName());
    properties.put("hibernate.implicit_naming_strategy", SpringImplicitNamingStrategy.class.getName());
    em.setJpaPropertyMap(properties);
    return em;

在下面刷新一批之后是生成的查询

Hibernate: insert into payg_reconciliation (version, inserted_dtm, bc_pay_sale_txn_payment_id, bc_pay_amount, bc_pay_cch_settlement_date, last_source_updated_dtm, merchant_id, payg_recon_status_id, bankcard_payment_id, declined_flag, subsystem_enum, retrieval_ref_nbr, payg_reconciliation_id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
Hibernate: insert into payg_reconciliation (version, inserted_dtm, bc_pay_sale_txn_payment_id, bc_pay_amount, bc_pay_cch_settlement_date, last_source_updated_dtm, merchant_id, payg_recon_status_id, bankcard_payment_id, declined_flag, subsystem_enum, retrieval_ref_nbr, payg_reconciliation_id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
Hibernate: insert into payg_reconciliation (version, inserted_dtm, bc_pay_sale_txn_payment_id, bc_pay_amount, bc_pay_cch_settlement_date, last_source_updated_dtm, merchant_id, payg_recon_status_id, bankcard_payment_id, declined_flag, subsystem_enum, retrieval_ref_nbr, payg_reconciliation_id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
Hibernate: insert into payg_reconciliation (version, inserted_dtm, bc_pay_sale_txn_payment_id, bc_pay_amount, bc_pay_cch_settlement_date, last_source_updated_dtm, merchant_id, payg_recon_status_id, bankcard_payment_id, declined_flag, subsystem_enum, retrieval_ref_nbr, payg_reconciliation_id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
Hibernate: insert into payg_reconciliation (version, inserted_dtm, bc_pay_sale_txn_payment_id, bc_pay_amount, bc_pay_cch_settlement_date, last_source_updated_dtm, merchant_id, payg_recon_status_id, bankcard_payment_id, declined_flag, subsystem_enum, retrieval_ref_nbr, payg_reconciliation_id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)

下面是休眠状态

    2021-07-13 13:15:26,261 [INFO ] scheduler_Worker-1 StatisticalLoggingSessionEventListener - Session Metrics 
    466400 nanoseconds spent acquiring 1 JDBC connections;
    0 nanoseconds spent releasing 0 JDBC connections;
    867200 nanoseconds spent preparing 13 JDBC statements;
    7763800 nanoseconds spent executing 11 JDBC statements;
    51541800 nanoseconds spent executing 8 JDBC batches;
    0 nanoseconds spent performing 0 L2C puts;
    0 nanoseconds spent performing 0 L2C hits;
    0 nanoseconds spent performing 0 L2C misses;
    4921200 nanoseconds spent executing 1 flushes (flushing a total of 400 entities and 400 collections);
    214407100 nanoseconds spent executing 7 partial-flushes (flushing a total of 2200 entities and 2200 collections)

在上面的 stat 7763800 中执行了 13 个 JDBC 语句，同样它有 51541800 执行了 8 个 JDBC 批处理。我认为如果我们可以让休眠模式为所有记录准备单个插入查询，我们可以存档一些性能

是否有可能对多条记录进行单批插入查询？下面是示例查询，我正在尝试存档

insert into payg_reconciliation (version, inserted_dtm, bc_pay_sale_txn_payment_id, bc_pay_amount, bc_pay_cch_settlement_date, last_source_updated_dtm, merchant_id, payg_recon_status_id, bankcard_payment_id, declined_flag, subsystem_enum, retrieval_ref_nbr, payg_reconciliation_id) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?),(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?),(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?),(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?),(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?),(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)

为什么休眠不生成单个插入查询，我是否错过了设置中的任何内容？ 提前致谢。

【参考方案1】：

在 application.properties 中设置以下属性。

spring.jpa.properties.hibernate.jdbc.batch_size=4
spring.jpa.properties.hibernate.order_inserts=true

第一个属性告诉 Hibernate 以四个为一组收集插入。 order_inserts 属性告诉 Hibernate 花时间按实体对插入进行分组，从而创建更大的批次。

【讨论】：

你可以在上面的问题中看到那些设置，我已经在hibernate属性中添加，在application.properties中添加，或者在LocalContainerEntityManagerFactoryBean中都一样

@GaaliPrabhakar 您的批处理大小为 50，它已执行 8 个批处理并保存了 400 个实体。你还期待什么。它正在分批正确执行事情

更多关于您期望记录的查询和为 50 个实体发送的正常查询是相同的并且具有相同的性能。控制台中还记录了什么？对于您的实体 ID 序列，这里也有发送到数据库的查询 - 也考虑一下

批次只有 50 个，在那个会话中，我正在更新 50 个“销售”实体并插入新的 50 个“侦察”，然后批次大小应该是 2，但是它是 8 个批次，有没有这样，我可以看到那些批次的详细信息吗？并且 DB 和实体序列中的分配大小为 50

查看日志，它在第一次刷新中保存了 400 个实体，在 7 次部分刷新中保存了 2200 个实体。如果您只想检查此 saveAll() 性能，请仅使用带有一些大数据的 saveAll 进行测试。此外，您可以使用baeldung.com/jpa-hibernate-batch-insert-update中给出的拦截器记录批量大小的查询和计数