Hibernate / MySQL 连接超时——尝试处理线程池执行程序在退出后不释放到 C3P0 的 Hibernate 连接

Posted 2023-02-24

【中文标题】Hibernate / MySQL 连接超时——尝试处理线程池执行程序在退出后不释放到 C3P0 的 Hibernate 连接

【英文标题】：Hibernate / MySQL connection timeouts -- Trying to deal with thread-pooled executors not releasing Hibernate connections to C3P0 after exit

【发布时间】：2013-05-19 04:55:20

【问题描述】：

我有一个使用 mysql 和 Hibernate for ORM 的 Tomcat 应用程序。我们的应用程序的性质要求我们必须为每个请求从 NoSQL 存储中提取和聚合大量分析数据，因此我们将每个请求的提取和聚合拆分为多个任务，并将这些任务委托给线程池执行器服务。

当每个线程执行一项任务时，它需要查询/更新 MySQL 的某些内容，因此它从 C3P0（我们用于连接池）借用 Hibernate 会话。

基本配置：

    <property name="current_session_context_class">thread</property>
    <property name="connection.provider_class">org.hibernate.connection.C3P0ConnectionProvider</property>
    <property name="hibernate.connection.shutdown">true</property>
    <property name="hibernate.use_sql_comments">false</property>

<!-- C3p0 Performance Improvements -->
    <property name="hibernate.c3p0.acquire_increment">1</property>
    <property name="hibernate.c3p0.idle_test_period">300</property>
    <property name="hibernate.c3p0.maxConnectionAge">3600</property>
    <property name="hibernate.c3p0.timeout">120</property>
    <property name="hibernate.c3p0.max_size">300</property>
    <property name="hibernate.c3p0.min_size">1</property>
    <property name="hibernate.c3p0.max_statements">100</property>
    <property name="hibernate.c3p0.preferredTestQuery">select 1;</property>

问题是 Hibernate 请求在 8 小时后导致 MySQL / JDBC 连接超时错误（我们配置的 MySQL 的 wait_timeout 参数值是默认值，即 8 小时）。我通过将 wait_timeout 设置为 11 分钟来复制它，但结果对于 8 小时的 wait_timeout 也是相同的：

2013-01-27 20:08:00,088 ERROR [Thread-0] (JDBCExceptionReporter.java:234) - Communications link failure

The last packet successfully received from the server was 665,943 milliseconds ago.  The last packet sent successfully to the server was 6 milliseconds ago.
org.hibernate.exception.JDBCConnectionException: could not execute query
  at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:99)
  at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
  at org.hibernate.loader.Loader.doList(Loader.java:2536)
  at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2276)
  at org.hibernate.loader.Loader.list(Loader.java:2271)
  at org.hibernate.loader.criteria.CriteriaLoader.list(CriteriaLoader.java:119)
  at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1716)
  at org.hibernate.impl.CriteriaImpl.list(CriteriaImpl.java:347) 
  .....
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 665,943 milliseconds ago.  The last packet sent successfully to the server was 6 milliseconds ago.
  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
  at java.lang.reflect.Constructor.newInstance(Unknown Source)
  at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
  at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)
  at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3102)
  at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:2991)
  at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3532)
  at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2002)
  at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2163)
  at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2624)
  at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2127)
  at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2293)
  at  com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:76)
  at org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:208)
  at org.hibernate.loader.Loader.getResultSet(Loader.java:1953)
  at org.hibernate.loader.Loader.doQuery(Loader.java:802)
  at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:274)
  at org.hibernate.loader.Loader.doList(Loader.java:2533)
  ... 9 more
Caused by: java.io.EOFException: Can not read response from server. Expected to read 4 bytes, read 0 bytes before connection was unexpectedly lost.
  at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:2552)
  at com.mysql.jdbc.MysqlIO.reuseAndReadPacket(MysqlIO.java:3002)
  ... 22 more

2013-01-27 20:19:00,179  WARN [Thread-0] (NewPooledConnection.java:487) - [c3p0] Another error has occurred [ com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 1,326,037 milliseconds ago.  The last packet sent successfully to the server was 660,100 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem. ] which will not be reported to listeners!
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 1,326,037 milliseconds ago.  The last packet sent successfully to the server was 660,100 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
  at java.lang.reflect.Constructor.newInstance(Unknown Source)
  at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
  at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1116)
  at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3364)
  at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1983)
  at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2163)
  at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2624)
  at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2127)
  at com.mysql.jdbc.PreparedStatement.executeQuery(PreparedStatement.java:2293)
  at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeQuery(NewProxyPreparedStatement.java:76)
  at org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:208)
  at org.hibernate.loader.Loader.getResultSet(Loader.java:1953)
  at org.hibernate.loader.Loader.doQuery(Loader.java:802)
  at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:274)
  at org.hibernate.loader.Loader.doList(Loader.java:2533)
  at org.hibernate.loader.Loader.listIgnoreQueryCache(Loader.java:2276)
  at org.hibernate.loader.Loader.list(Loader.java:2271)
  at org.hibernate.loader.criteria.CriteriaLoader.list(CriteriaLoader.java:119)
  at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1716)
  at org.hibernate.impl.CriteriaImpl.list(CriteriaImpl.java:347)
  ....
Caused by: java.net.SocketException: Broken pipe
  at java.net.SocketOutputStream.socketWrite0(Native Method)
  at java.net.SocketOutputStream.socketWrite(Unknown Source)
  at java.net.SocketOutputStream.write(Unknown Source)
  at java.io.BufferedOutputStream.flushBuffer(Unknown Source)
  at java.io.BufferedOutputStream.flush(Unknown Source)
  at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3345)
  ... 20 more

我认为 C3P0 没有足够频繁地驱逐过时的连接，因此在重负载期间，连接会返回到池中，过时并在驱逐前再次被借用。所以我创建了一个 c3p0.properties 文件并将“c3p0.testConnectionsOnCheckout”设置为 true，这样就不会从池中借用过时的连接。我又遇到了同样的错误。

我认为由于我的休眠会话配置了会话上下文“线程”，因此线程使用的会话在事务提交或回滚之前不会被释放 [http://docs.jboss.org/hibernate/orm/3.6/javadocs/org/hibernate/context/ThreadLocalSessionContext.html]。所以我唯一的解释是执行程序线程在没有提交的情况下进行只读数据库调用，一旦任务完成，线程就会返回到池中，并与它保持会话。

我应该在这里做什么？我们的应用程序在 Data-Access-Object 类中有休眠查询代码，每种 bean 一个。我希望避免必须更改那些进行只读数据库调用的类中的每一个方法，以便让它们不必要地“提交”。我还希望尽量减少对我们应用程序业务逻辑的更改。

有没有一种方法可以检查 sessionFactory.getCurrentSession() 每次声明/使用 Data-Access-Object 时返回的会话的新鲜度（所有 Data-Access-Object 类都从一个单一的继承基类，所以我可以更改该基类构造函数中的内容），并可能将陈旧的会话返回到池中？或者有更好的方法吗？

谢谢。

【参考方案1】：

注意：以下基于 Oracle 示例，但可以针对 MySQL 进行修改。

通过使用 Tomcat 管理的数据库连接，我已经解决了连接断开的问题。这可以通过在 META-INF 目录中创建一个 context.xml 文件来完成，如下所示：

<Context crossContext="true" docBase="myBase" path="/myBase" reloadable="false" useHttpOnly="true">
    <ResourceLink global="jdbc/dbOne" name="jdbc/dbOne" type="javax.sql.DataSource" />
    <!-- Need another DB? -->   
    <!-- <ResourceLink global="jdbc/dbTwo" name="jdbc/dbTwo" type="javax.sql.DataSource" /> -->
</Context>

要创建连接，还必须更新 Tomcat server.xml 文件。以下基于Tomcat 6.0.26：

<!-- Global JNDI resources
   Documentation at /docs/jndi-resources-howto.html
-->
<GlobalNamingResources>
<!-- Editable user database that can also be used by
     UserDatabaseRealm to authenticate users
-->
<Resource name="UserDatabase" auth="Container"
          type="org.apache.catalina.UserDatabase"
          description="User database that can be updated and saved"
          factory="org.apache.catalina.users.MemoryUserDatabaseFactory"
          pathname="conf/tomcat-users.xml" />

    <!-- START MY MODS -->
    <!-- Add the db connection(s)! The rest is standard in the server.xml file.-->        
    <Resource auth="Container" driverClassName="oracle.jdbc.driver.OracleDriver" initialSize="2" logAbandoned="true" maxActive="5" maxIdle="2" maxWait="120000" minEvictableIdleTimeMillis="1800000" minIdle="1" name="jdbc/dbOne" numTestsPerEvictionRun="3" password="openPlease" removeAbandoned="true" removeAbandonedTimeout="60" testOnBorrow="true" testOnReturn="true" testWhileIdle="true" timeBetweenEvictionRunsMillis="900000" type="javax.sql.DataSource" url="jdbc:oracle:thin:@servername:portnumber:schema" username="myUser" validationQuery="select sysdate from dual"/>  
    <!-- Need another connection? Copy and past the above making the required changes. -->
    <!-- END MY MODS -->

然后在我指定的 hibernate.cfg.xml 文件中：

<!-- Connection handling -->
<property name="connection.datasource">java:/comp/env/jdbc/dbOne</property>
<property name="dialect">org.hibernate.dialect.Oracle10gDialect</property>
<property name="connection.autoReconnect">true</property>
<property name="connection.autoReconnectForPools">true</property>
<property name="connection.is-connection-validation-required">true</property>
<property name="current_session_context_class">thread</property>

话虽如此，以下仅使用 C3P0 的配置也适用于我，但我现在更喜欢 Tomcat 托管连接，因为它们可以在我的 Tomcat 服务器上的应用程序之间共享，这在我的情况下是可取的。

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE hibernate-configuration PUBLIC
        "-//Hibernate/Hibernate Configuration DTD 3.0//EN"
        "http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">
<hibernate-configuration>
    <session-factory>
        <property name="hibernate.connection.driver_class">oracle.jdbc.driver.OracleDriver</property>
        <property name="hibernate.connection.password">@dbPassword@</property>
        <property name="hibernate.connection.url">@dbURL@</property>
        <property name="hibernate.connection.username">someUser</property>
        <property name="dialect">org.hibernate.dialect.Oracle10gDialect</property>

        <property name="show_sql">false</property>
        <property name="format_sql">false</property>
        <property name="current_session_context_class">thread</property>

        <!-- Connection handling -->
        <property name="connection.autoReconnect">true</property>
        <property name="connection.autoReconnectForPools">true</property>
        <property name="connection.is-connection-validation-required">true</property>
        <property name="current_session_context_class">thread</property>
        <property name="max_fetch_depth">1</property>

        <property name="hibernate.connection.provider_class">org.hibernate.connection.C3P0ConnectionProvider</property>  
        <property name="hibernate.c3p0.breakAfterAcquireFailure">false</property>
        <property name="hibernate.c3p0.acquireRetryAttempts">-1</property>
        <property name="hibernate.c3p0.acquireRetryDelay">3000</property>
        <property name="hibernate.c3p0.automaticTestTable">C3P0_TEST</property> <!-- c3p0 uses this table for testing connection. -->
        <property name="hibernate.c3p0.initialPoolSize">2</property> <!-- 3 is c3p0 default. -->
        <property name="hibernate.c3p0.minPoolSize">2</property> <!-- 1 is Hibernate default. -->
        <property name="hibernate.c3p0.maxPoolSize">6</property> <!-- 100 is Hibernate default. -->
        <property name="hibernate.c3p0.acquireIncrement">2</property> <!-- 3 is c3p0 default. -->
        <property name="hibernate.c3p0.maxIdleTimeExcessConnections">600</property> <!-- 0 seconds is c3p0 default and means do not check. This is the num of seconds a connections in excess of minPoolSize are permitted to remain idle in the pool before being culled. -->
        <property name="hibernate.c3p0.idleConnectionTestPeriod">30</property> <!-- 0 seconds is the c3p0 default and means do not check. (i.e. never test). Using only this provides the best performance even if individuals may occasionally receive an error message due to a connection being dropped. -->

        <!-- Cache setup -->
        <property name="hibernate.cache.use_second_level_cache">true</property>
        <property name="hibernate.cache.use_query_cache">true</property>
        <property name="hibernate.cache.region.factory_class">net.sf.ehcache.hibernate.EhCacheRegionFactory</property>
       <!-- <property name="hibernate.cache.provider_class">net.sf.ehcache.hibernate.SingletonEhCacheProvider</property> -->

        <property name="cache.use_minimal_puts">true</property>
        <property name="hibernate.generate_statistics">true</property>
        <property name="hibernate.cache.use_structured_entries">true</property>

        <!-- Mapping files -->
        ...

       </session-factory>

</hibernate-configuration>