Java/Spring JDBC：批量插入 2 个表：从第 1 批插入获取第 2 个表所需的 FK ID

Posted 2023-04-18

【中文标题】Java/Spring JDBC：批量插入 2 个表：从第 1 批插入获取第 2 个表所需的 FK ID

【英文标题】：Java/Spring JDBC: Batch Insert into 2 Tables: Obtain FK ID from 1st Batch Insert Required for 2nd Table

【发布时间】：2020-06-06 21:56:50

【问题描述】：

我正在使用jdbcTemplate 批量插入到 2 个表中。第一张桌子很简单，有一个ID。第二个表有一个 FK 参考 USER_ID，我需要在插入之前从表 1 中获取它。

假设我有这个：

主要 Java 代码（这里我分成多个批次

for(int i = 0; i < totalEntries.size(); i++) 
    // Add to Batch-Insert List; if list size ready for batch-insert, or if at the end, batch-persist & clear list
    batchInsert.add(user);

    if (batchInsert.size() == 1000 || i == totalEntries.size() - 1) 
         // 1. Batch is ready, insert into Table 1
         nativeBatchInsertUsers(jdbcTemplate, batchInsert);
         // 2. Batch is ready, insert into Table 2
         nativeBatchInsertStudyParticipants(jdbcTemplate, batchInsert);
        // Reset list
        batchInsert.clear();
    

批量插入表 1 的方法（注意，我在这里为 USERS_T 获取 Seq Val）

    private void nativeBatchInsertUsers(JdbcTemplate jdbcTemplate, final List<UsersT> batchInsert) 

        String sqlInsert_USERS_T =  "INSERT INTO PUBLIC.USERS_T (id, password, user_name) " +
                                    "VALUES (nextval('users_t_id_seq'), ?, ? " +
                                            ")";        

        // Insert into USERS_T using overridden JdbcTemplate's Native-SQL batchUpdate() on the string "sqlInsert_USERS_T"
        jdbcTemplate.batchUpdate(sqlInsert_USERS_T, new BatchPreparedStatementSetter() 

            @Override
            public int getBatchSize() 
                return batchInsert.size();
            

            @Override
            public void setValues(PreparedStatement ps, int i) throws SQLException 
                ps.setString(1, null); 
                ps.setString(2, batchInsert.get(i).getUsername()); 
                // etc.
       );

批量插入表2的方法

private void nativeBatchInsertStudyParticipants(JdbcTemplate jdbcTemplate, final List<UsersT> batchInsertUsers) 

    String sqlInsert_STUDY_PARTICIPANTS_T = 
            "INSERT INTO PUBLIC.STUDY_PARTICIPANTS_T (id, study_id, subject_id, user_id) "  
                            "VALUES (nextval('study_participants_t_id_seq'), ?, ?, ?
                                            ")";        

    // Insert into STUDY_PARTICIPANTS_T using overridden JdbcTemplate's Native-SQL batchUpdate() on the string "sqlInsert_USERS_T"
    jdbcTemplate.batchUpdate(sqlInsert_STUDY_PARTICIPANTS_T, new BatchPreparedStatementSetter() 

        @Override
        public int getBatchSize() 
            return batchInsert.size();
        

        @Override
        public void setValues(PreparedStatement ps, int i) throws SQLException 

             // PROBLEM: For Param #4, USER_ID, need to get the USERS_T.ID from Batch-Insert #1

               

    );

   

当我来到第二批插入时，其中一列是返回到 USERS_T.ID 的 FK，称为 STUDY_PARTICIPANTS_T.USER_ID。我是否可以通过保持jdbcTemplate.batchUpdate() 逻辑来获得它？

【参考方案1】：

这就是答案。

1) 如果您使用jdbcTemplate (Spring JDBC)，一种解决方案是提前保留您自己的 ID 范围。然后自己为每一行提供手动计算的 ID。例如

@Transactional(readOnly = false, rollbackFor = Exception.class)
public void doMultiTableInsert(List<String> entries) throws Exception 


    // 1. Obtain current Sequence values
    Integer currTable1SeqVal = table1DAO.getCurrentTable1SeqVal();
    Integer currTable2SeqVal = table2DAO.getCurrentTable2SeqVal();     
    // 2. Immediately update the Sequences to the calculated final value (this reserves the ID range immediately)
    table1DAO.setTable1SeqVal(currTable1SeqVal + entries.size());          
    table2DAO.setTable2SeqVal(currTable2SeqVal + entries.size());           

    for(int i = 0; i < entries.size(); i++) 
         // Prepare Domain object...
         UsersT user = new User();
         user.setID(currTable1SeqVal + 1 + i); // Set ID manually
         user.setCreatedDate(new Date());
         // etc.
         StudyParticipantsT sp = new StudyParticipantsT();
         sp.setID(currTable2SeqVal + 1 + i); // Set ID manually
         // etc.
         user.setStudyParticipant(sp);

         // Add to Batch-Insert List
         batchInsertUsers.add(user);

         // If list size ready for Batch-Insert (in this ex. 1000), or if at the end of all subjectIds, perform Batch Insert (both tables) and clear list
         if (batchInsertUsers.size() == 1000 || i == subjectIds.size() - 1) 
            // Part 1: Insert batch into USERS_T
            nativeBatchInsertUsers(jdbcTemplate, batchInsertUsers);             
            // Part 2: Insert batch into STUDY_PARTICIPANTS_T
            nativeBatchInsertStudyParticipants(jdbcTemplate, batchInsertUsers);                 
            // Reset list
            batchInsertUsers.clear();
         
    

然后是上面引用的 Batch-Insert 子方法：

1)

  private void nativeBatchInsertUsers(JdbcTemplate jdbcTemplate, final List<UsersT> batchInsertUsers) 
       String sqlInsert =   "INSERT INTO PUBLIC.USERS_T (id, password,  ... )"; // etc.
       jdbcTemplate.batchUpdate(sqlInsert, new BatchPreparedStatementSetter() 

           @Override
           public int getBatchSize() 
             return batchInsertUsers.size();
            

           @Override
           public void setValues(PreparedStatement ps, int i) throws SQLException 
              ps.setInt(1, batchInsertUsers.get(i).getId()); // ID (provided by ourselves)
              ps.setDate(2, batchInsertUsers.get(i).getCreatedDate());
              //etc.
                       
       );
    

2)

private void nativeBatchInsertStudyParticipants(JdbcTemplate jdbcTemplate, final List<UsersT> batchInsertUsers) 
   String sqlInsert =   "INSERT INTO PUBLIC.STUDY_PARTICIPANTS_T (id, ... )"; // etc.
   jdbcTemplate.batchUpdate(sqlInsert, new BatchPreparedStatementSetter() 

       @Override
       public int getBatchSize() 
         return batchInsertUsers.size();
        

       @Override
       public void setValues(PreparedStatement ps, int i) throws SQLException 
          ps.setInt(1, batchInsertUsers.get(i).getStudyParticipants().getId()); // ID (provided by ourselves)
          //etc.
                   
   );

有一些方法可以获取/设置序列值，例如在 Postgres 中是

SELECT last_value FROM users_t_id_seq;   -- GET SEQ VAL
SELECT setval('users_t_id_seq', 621938); -- SET SEQ VAL

还要注意，所有内容都在@Transactional 之下。如果方法中有任何异常，所有数据都会回滚（对于 all 异常，rollbackFor = Exception.class）。唯一不会回滚的是手动序列更新。不过没关系，序列可以有间隙。

2) 如果您愿意降级到PreparedStatement 级别，另一种解决方案是Statement.RETURN_GENERATED_KEYS：

PreparedStatement ps = con.prepareStatement(sql, Statement.RETURN_GENERATED_KEYS)

在您执行ps 后，ResultSet 将按照创建顺序包含您的 ID。您可以遍历 ResultSet 并将 ID 存储在单独的列表中。

while (rs.next()) 
   generatedIDs.add(rs.getInt(1));

请记住，在这种情况下，您负责自己的事务管理。你需要conn.setAutoCommit(false); 让批次堆积起来，没有真正的持久性，然后conn.commit(); / conn.rollback();。