org.hibernate.exception.GenericJDBCException 的发生:无法为长时间运行的进程打开连接

Posted

技术标签:

【中文标题】org.hibernate.exception.GenericJDBCException 的发生:无法为长时间运行的进程打开连接【英文标题】:Occurrence of org.hibernate.exception.GenericJDBCException: Could not open connection for long running process 【发布时间】:2018-04-22 12:49:30 【问题描述】:

在我的应用程序中,我使用 20 个线程来完成一项工作。每个线程调用一个远程 Web 服务并更新 Oracle DB。在 Jboss restart 之后,该作业无法更新数据库,但我在日志中发现的异常为:

2017-11-08 23:36:20,706 ERROR [org.hibernate.engine.jdbc.spi.SqlExceptionHelper] (EJB default - 21) javax.resource.ResourceException: IJ000460: Error checking for a transaction
2017-11-08 23:36:20,706 ERROR [org.jboss.as.ejb3] (EJB default - 16) javax.ejb.EJBTransactionRolledbackException: org.hibernate.exception.GenericJDBCException: Could not open connection
2017-11-08 23:36:20,707 ERROR [org.jboss.as.ejb3] (EJB default - 20) javax.ejb.EJBTransactionRolledbackException: org.hibernate.exception.GenericJDBCException: Could not open connection
2017-11-08 23:36:20,707 ERROR [org.jboss.as.ejb3] (EJB default - 19) javax.ejb.EJBTransactionRolledbackException: org.hibernate.exception.GenericJDBCException: Could not open connection
2017-11-08 23:36:20,710 ERROR [org.jboss.as.ejb3] (EJB default - 21) javax.ejb.EJBTransactionRolledbackException: org.hibernate.exception.GenericJDBCException: Could not open connection
2017-11-08 23:36:20,711 ERROR [org.jboss.as.ejb3.invocation] (EJB default - 16) JBAS014134: EJB Invocation failed on component PersonServiceDaoImpl for method public abstract long com.kaviletta.re.caltek.db.dao.PersonServiceDao.getNotUpdatedAddressCountBySubmissionId(int): javax.ejb.EJBTransactionRolledbackException: org.hibernate.exception.GenericJDBCException: Could not open connection
    at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleInCallerTx(CMTTxInterceptor.java:162) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInCallerTx(CMTTxInterceptor.java:252) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:341) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:238) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at com.kaviletta.re.caltek.db.dao.PersonServiceDao$$$view6.getNotUpdatedAddressCountBySubmissionId(Unknown Source) [classes:]
    at com.kaviletta.re.caltek.rest.service.impl.AsynchronousServiceImpl.asyncAddressDetails(AsynchronousServiceImpl.java:61) [classes:]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.7.0_55]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) [rt.jar:1.7.0_55]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.7.0_55]
    at java.lang.reflect.Method.invoke(Method.java:606) [rt.jar:1.7.0_55]
    at org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:58) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) [jboss-as-jpa-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:272) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:339) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:238) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:55) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:182) [jboss-as-ee-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288) [jboss-invocation-1.1.2.Final-redhat-1.jar:1.1.2.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.AsyncFutureInterceptorFactory$1$1.runInvocation(AsyncFutureInterceptorFactory.java:89) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at org.jboss.as.ejb3.component.interceptors.AsyncInvocationTask.run(AsyncInvocationTask.java:73) [jboss-as-ejb3-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_55]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_55]
    at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_55]
    at org.jboss.threads.JBossThread.run(JBossThread.java:122)
Caused by: javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: Could not open connection
    at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1387) [hibernate-entitymanager-4.2.7.SP5-redhat-1.jar:4.2.7.SP5-redhat-1]
    at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1310) [hibernate-entitymanager-4.2.7.SP5-redhat-1.jar:4.2.7.SP5-redhat-1]
    at org.hibernate.ejb.QueryImpl.getSingleResult(QueryImpl.java:316) [hibernate-entitymanager-4.2.7.SP5-redhat-1.jar:4.2.7.SP5-redhat-1]
    at org.jboss.as.jpa.container.QueryNonTxInvocationDetacher.getSingleResult(QueryNonTxInvocationDetacher.java:69) [jboss-as-jpa-7.3.4.Final-redhat-1.jar:7.3.4.Final-redhat-1]

线程用于数据库操作的代码:

    @PersistenceContext
EntityManager entityManager;
protected EntityManager getEntityManager() 
    return entityManager;


protected void setEntityManager(EntityManager manager) 
    this.entityManager = manager;

 public long getNotUpdatedAddressCountBySubmissionId(int submissionId) 
            Query q = getEntityManager().createQuery(
                    "select count(address.addressId.submissionId) from AddressStruc address where processState!=1  and address.addressId.submissionId = :submissionId");
            q.setParameter("submissionId", submissionId);
            return (long) q.getSingleResult();

        

        public void updateStudentdAddress(List<AddressStruc> studnetsAddressDetails) 
            try 
                for (AddressStruc studentdAddress : studnetsAddressDetails) 
                    AddressStruc addressDetails = entityManager.find(AddressStruc.class, new AddressId(
                            studentAddress.getAddressId().getSubmissionId(), studentAddress.getAddressId().getRowId()));
                    if (addressDetails != null) 
                        entityManager.merge(updateStudentDetails(addressDetails, studentAddress));
                        entityManager.flush();
                    

                
             catch (Exception e) 
                log.error("Exception occured while updating Address(method-->updateStudentAddress))-->" + e);
            
        

Asyn 方法调用 updateStudentAddress 方法

public AsyncResult<StudentStatusModel> asyncAddressDetails(int startRow, int noOfRecord, int submissionId) 
    // method for fetching record based on the start row provided
    Collection<AddressStruc> addressList = studentEjb.getAddressBySubmissionId(startRow, noOfRecord, submissionId);
    StudentStatusModel status = new StudentStatusModel();
    status.setStartRow(startRow);
    status.setRetry(false);
    status.setMessage(null);
    // invoke the webservice and generate student for the given address
    if (StudentUtil.webServiceFailed <= 5) 
        OutputModel outputModel = invokeWebservice(addressList, status);

        if ((outputModel != null) && (outputModel.getStudentModels().size() > 0)) 
        // update the address details based on the submissionId+rowId
        studentEjb.updateStudentAddress(new ArrayList<>(convertModelToEntity(outputModel).values()));
    

return new AsyncResult<StudentStatusModel>(status);

我正在使用容器托管实体管理器进行数据库转换。

每个线程调用updateStudentAddress方法,更新100条记录。

由于这项工作是长期运行,一段时间后 jboss 无法与 oracle 建立进一步的连接。

有没有办法解决这个问题?

【问题讨论】:

乍一看,我怀疑您的线程如何将连接释放回 DataSource 的连接池。 Oracle 有一个允许连接的限制(我记得默认是 1000)。因此,如果未释放连接,则下一个线程会从池中选择一个新线程,当达到最大值时 - 它会失败。您能否调查获得该异常需要多长时间?通过了多少次成功调用,以及您的 Oracle 的限制是什么。另外,请求完成时您的线程如何处理?至少我会先这样…… 此异常发生在午夜重新启动 jboss 之后,因此当作业运行时,可能会发生 jboss 重新启动,这会停止当前正在运行的线程,重新启动后它会使用新线程恢复。 你好,你有什么更新吗?原因是什么?请问你是怎么解决的? 【参考方案1】:

正如另一个答案指出的那样,您可能永远不会使用 Hibernate 捕获和忽略持久性异常,文档中明确禁止它。但是,我认为您的问题与每晚重新启动有关。如果没有完整的日志,很难确定,但是当 JBoss 启动时,某些部分会先于其他部分启动。在一切初始化之前收到入站请求是很常见的,然后出现错误。如果你能阻止重启一晚,你可以看看你是否仍然收到错误。

修复(除了不重新启动)是确保组件以正确的顺序启动或防止传入调用(如果它们通过负载平衡器很容易)直到重新启动完成。

要控制启动顺序,请编辑application.xml 并确保initialize-in-order 为真。然后按所需的启动顺序列出模块。 Web/作业层应该最后开始。

一个非常肮脏的修复方法是在代码中通过捕获错误、休眠几秒钟然后重试来处理它。我知道有些人这样做了,尽管我不推荐这样做。

【讨论】:

如果我阻止重启,错误将无法重现。【参考方案2】:

org.hibernate.exception.GenericJDBCException:无法打开连接。 你能显示代码吗?这可能是同步线程的问题。或实施不当。

当您尝试对 DB 进行写入时,您需要打开事务。一个例子

employee = em.find(Employee.class, 1);

  em.getTransaction().begin();
  employee.setNickname("Joe the Plumber");
  entityManager.merge(employee);
  em.getTransaction().commit();

【讨论】:

好的,谢谢您正在准备对数据库进行交易,但您忘记了打开的交易。这可能是问题所在。 我正在使用容器管理实体管理器,所以我不需要开始交易。【参考方案3】:

我认为问题可能是由于里面的异常处理引起的:asyncAddressDetails

我们在 jboss7 中发生了这个错误,在 RuntimeException 或 ApplicationException 导致事务回滚后,当这个异常被捕获并且应用程序代码在当前事务上下文完成之前对 EntityManager 进行了进一步的操作。

示例:

void methodInNewTransaction() 
  try 
    dao.inRequiredTransaction();
   catch (RuntimeException ex) 
    ; // ignore it
  
  dao.secondCallInRequiredTransaction();

在忽略 RuntimeException 时,secondCallInRequiredTransaction 可能会导致错误。

如何解决这个问题,我看到以下可能性:

防止异常发生 使用@ApplicationException-Annotation 来控制发生异常的事务影响 重新抛出异常,以便不再使用外部事务上下文

【讨论】:

您好,我遇到了同样的问题。我同意你的观点,因为在我的日志中还提到了Caused by: org.jboss.util.NestedSQLException: Transaction is not activeCaused by: javax.resource.ResourceException: Transaction is not active,你是如何找到导致问题的根异常的?您认为数据库中的ORA-00060: deadlock detected while waiting for resource 可能会导致问题吗?谢谢 当然,ORA-00060 将导致当前事务的回滚,至少是所选事务的回滚。因此,之后期望事务的 JPA 操作或休眠操作将导致您在此处看到的内容。 你有调试ORA-00060的经验吗,请分享一下? DBMS 本身通常会记录导致死锁的语句。查看这些语句,您应该能够找出应用程序中的哪些代码是罪魁祸首。请您的 db-admins 为您提供死锁相关信息。

以上是关于org.hibernate.exception.GenericJDBCException 的发生:无法为长时间运行的进程打开连接的主要内容,如果未能解决你的问题,请参考以下文章