故障•监听TNS-12518TNS-00517和 Linux Error：32：Broken pipe

Posted 2022-03-11 小麦苗DB宝

​​【故障•监听】TNS-12518、TNS-00517和 Linux Error：32：Broken pipe​​

​【故障​|​监听】TNS-12518、TNS-00517和 Linux Error：32：Broken pipe​

1.1  ​BLOG文档结构图​

​​​ 

1.2  ​前言部分​

1.2.1  ​导读和注意事项​

各位技术爱好者，看完本文后，你可以掌握如下的技能，也可以学到一些其它你所不知道的知识，~O(∩_∩)O~：

① Linux Error: 32: Broken pipe的可能原因（重点）

② TNS-12518: TNS:listener could not hand off client connection的一般解决过程

③ SQL*Plus登录报错：ORA-12537: TNS:connection closed

​Tips：​

① 本文在itpub（http://blog.itpub.net/26736162）和微信公众号（xiaomaimiaolhr）上有同步更新。

② 文章中用到的所有代码、相关软件、相关资料及本文的pdf版本都请前往小麦苗的云盘下载，小麦苗的云盘地址见：http://blog.itpub.net/26736162/viewspace-1624453/。

③ 若网页文章代码格式有错乱，请下载pdf格式的文档来阅读。

④ 在本篇BLOG中，代码输出部分一般放在一行一列的表格中。

​本文如有错误或不完善的地方请大家多多指正，ITPUB留言或QQ皆可，您的批评指正是我写作的最大动力。​

1.3  ​故障分析及解决过程​

1.3.1  ​故障环境介绍​

项目	source db
db 类型	单机
db version	11.2.0.3.0
db 存储	ASM
OS版本及kernel版本	RHEL 6.5

1.3.2  ​故障发生现象及报错信息​

登陆报错：

[oracle@orcltest ~]$ sqlplus sys/"l@h\\r/0"@LHRDB as sysdba SQL*Plus: Release 11.2.0.3.0 Production on Thu Mar 16 12:06:36 2017 Copyright (c) 1982, 2011, Oracle.  All rights reserved. ERROR: ORA-12537: TNS:connection closed

查监听日志（/u01/app/11.2.0/grid/network/admin/listener.ora），报错如下：

16-MAR-2017 12:06:36 * (CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=lhrdb)(CID=(PROGRAM=sqlplus)(HOST=orcltest)(USER=oracle))) * (ADDRESS=(PROTOCOL=tcp)(HOST=192.168.59.129)(PORT=12333)) * establish * lhrdb * 12518 TNS-12518: TNS:listener could not hand off client connection TNS-12547: TNS:lost contact   TNS-12560: TNS:protocol adapter error    TNS-00517: Lost contact     Linux Error: 32: Broken pipe

1.3.3  ​故障分析

我的解决参考：http://www.itpub.net/thread-1870217-1-1.html

根据MOS：Troubleshooting Guide for TNS-12518 TNS listener could not hand off client connection (文档 ID 550859.1)中，

Error: 32: Broken pipe Error stack in listener log: TNS-12518: TNS:listener could not hand off client connection TNS-12547: TNS:lost contact    TNS-12560: TNS:protocol adapter error      TNS-00517: Lost contact        IBM/AIX RISC System/6000 Error: 32: Broken pipe Cause: The error 32 indicates the communication has been broken while the listener is trying to hand off the client connection to the server process or dispatcher process. Action: 1.  One of reason would be processes parameter being low, and can be verified by the v$resource_limit view. 2. In Shared Server mode, check the lsnrctl services output and see if the dispatcher has refused any connections, if so, then consider increasing the number of dispatchers. 3. Check the alert log for any possible errors. 4. Memory resource is also another cause for this issue. Check the swap, memory usage of the OS. 5. If RAC/SCAN or listener is running in separate home, check the following note: Note: 1069517.1  ORA-12537 or TNS-12518 if Listener (including SCAN Listener) and Database are Owned by Different OS User

根据第5条，因为我的grid是11.2.0.1，而Oracle是11.2.0.3的，查询MOS:ORA-12537 / ORA-12547 or TNS-12518 if Listener (including SCAN Listener) and Database are Owned by Different OS User (文档 ID 1069517.1)，里边明确指出是由于$RDBMS_HOME/bin/oracle文件的权限问题导致。

1.3.4  ​故障解决​

$ORACLE_HOME/bin/oracle文件的权限问题，应该为-rwsr-s--x，属主应该是oracle:asmadmin。

解决办法：

方法：

1、修改oracle的权限为6751

[root@orcltest bin]# ll /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle -rwxr-s--x. 1 oracle asmadmin 210823844 May 18  2014 /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle [root@orcltest ~]#  chmod 6751 /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle [root@orcltest bin]# ll /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle -rwsr-s--x. 1 oracle asmadmin 210823844 May 18  2014 /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle [root@orcltest bin]#

2、重新用setasmgidwrap设置oracle

[root@orcltest bin]# ll /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle -rwxr-s--x. 1 oracle asmadmin 210823844 May 18  2014 /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle [root@orcltest ~]# /u01/app/11.2.0/grid/bin/setasmgidwrap -o /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle [root@orcltest bin]# ll /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle -rwsr-s--x. 1 oracle asmadmin 210823844 May 18  2014 /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle [root@orcltest bin]# [root@orcltest bin]# stat /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle   File: `/u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle   Size: 210823844       Blocks: 411776     IO Block: 4096   regular file Device: 802h/2050d      Inode: 1717737     Links: 1 Access: (6751/-rwsr-s--x)  Uid: (  501/  oracle)   Gid: (  504/asmadmin) Access: 2017-03-16 11:05:44.809363974 +0800 Modify: 2014-05-18 17:09:50.508549983 +0800 Change: 2017-03-16 12:33:15.733816820 +0800 [root@orcltest bin]# [root@orcltest bin]#

重新连接：

[oracle@orcltest ~]$ sqlplus sys/"l@h\\r/0"@LHRDB as sysdba SQL*Plus: Release 11.2.0.3.0 Production on Thu Mar 16 13:32:48 2017 Copyright (c) 1982, 2011, Oracle.  All rights reserved. Connected to: Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production With the Partitioning, Automatic Storage Management, OLAP, Data Mining and Real Application Testing options SYS@LHRDB>

1.4  ​故障处理总结​

如果可执行文件$ORACLE_HOME/bin/oracle的属主或权限设定出了问题，那么可能会造成很多问题。例如：无法登陆到数据库、ora-600错误、“TNS-12518: TNS:listener could not hand off client connection”、“Linux Error: 32: Broken pipe”、“ORA-12537: TNS:connection closed”、访问ASM磁盘出错等。解决办法很简单，可以在grid用户下运行setasmgidwrap命令重新配置$ORACLE_HOME/bin/oracle可执行文件的权限和属主或者直接将oracle文件的权限修改为6751。$ORACLE_HOME/bin/oracle可执行文件正确属主应该是oracle:asmadmin，并且权限必须有s共享才可以，如下所示：

[root@orcltest ~]$ which setasmgidwrap /u01/app/11.2.0/grid/bin/setasmgidwrap [root@orcltest ~]$ setasmgidwrap -o /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle [root@orcltest ~]$ ll /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle -rwsr-s--x 1 oracle asmadmin 232399083 Apr 21  2015 /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle [root@orcltest ~]# ll /u01/app/11.2.0/grid/bin/oracle -rwsr-s--x. 1 grid oinstall 203972117 Jan  5  2015 /u01/app/11.2.0/grid/bin/oracle [root@orcltest ~]# chmod 6751 /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle [root@orcltest ~]# ll /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle        -rwsr-s--x 1 oracle asmadmin 232399083 Apr 21  2015 /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle [root@orcltest bin]# cd /u01/app/oracle/product/11.2.0/dbhome_1/bin/ [root@orcltest bin]# which stat /usr/bin/stat [root@orcltest bin]# stat oracle   File: `oracle   Size: 210823844       Blocks: 411776     IO Block: 4096   regular file Device: 802h/2050d      Inode: 1717737     Links: 1 Access: (6751/-rwsr-s--x)  Uid: (  501/  oracle)   Gid: (  504/asmadmin) Access: 2017-03-16 12:33:44.809363974 +0800 Modify: 2014-05-18 17:09:50.508549983 +0800 Change: 2017-03-16 11:05:15.733816820 +0800

问：在UNIX/LINUX环境中，oracle数据库启动后存在许多后台进程和前台进程，虽然相关进程产生一些trace文件也是常有的事情，但是真正是什么决定了oracle相关进程的属性呢？

答：通常来说，oracle的后台进程的调用是依赖于$ORACLE_HOME/bin/oracle这个二进制文件，但它从远端连入而分配的服务器进程（server process）相关属主的属性则是继承自listener进程，而listener进程的属主属性同样是进程自其启动的用户（分oracle用户和grid用户）$ORACLE_HOME/bin/oracle的属主属性。

其他原因：

启动ORACLE监听的时候 报错 Linux Error: 32: Broken pipe

原因：原来/home/oracle/product/9.2.0.1.0/network/log/listener.log 文件超过2G，监听就会断掉。

解决办法：清空日志文件

              [oracle@localhost oracle]$ cd $ORACLE_HOME/network/log

              [oracle@chicago log]$ cat listener.log

至此问题解决

有个命令可以对这个写日志进行关闭

LSNRCTL> set log_status off

要继续写日志，只需要

LSNRCTL> set log_status on

也可以在 listener.ora 文件里面添加一项

LOGGING_LISTENER = OFF

添加了这个之后无论你怎么重启监听，都不会写日志了。

1.5  ​用到的SQL集合​

setasmgidwrap -o /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle chmod 6751 /u01/app/oracle/product/11.2.0/dbhome_1/bin/oracle stat oracle

1.6  ​参考文章​

1.6.1  ​MOS​

---

---

1.6.1.1  ​Troubleshooting Guide for TNS-12518 TNS listener could not hand off client connection (文档 ID 550859.1)

​In this Document

​​Purpose​​

​​Troubleshooting Steps​​

​​Section I: Steps to approach ORA-12518/TNS-12518 Error:​​

​​Section II: Commonly Known Errors:​​

​​Error: 32: Broken pipe​​

​​Error: 11: Resource temporarily unavailable​​

​​Error: Connection Pooling limit reached​​

​​Error: 24: Too many open files​​

​​Section III: Errors Specific to Windows​​

​​Error: 2: No such file or directory ​​

​​Error: 233: Unknown error​​

​​Error: 54: Unknown error​​

​​Error: 10022: Unknown error​​

​​References​​

---

APPLIES TO:

Oracle Net Services - Version 10.1.0.5 to 12.1.0.2 [Release 10.1 to 12.1]

Information in this document applies to any platform.

PURPOSE

This article discusses  how to troubleshoot the ORA-12518/TNS-12518 listener errors.

TROUBLESHOOTING STEPS

Section I: Steps to approach ORA-12518/TNS-12518 Error:

ORA-12518/TNS-12518 indicates a problem while listener hands off the client connection to the server process or dispatcher process.TNS-12518 is logged in the listener log. The client might receive ORA-12518 or some other disconnection errors like ORA-12537. Once TNS-12518 is noted in the listener log, follow the below steps to approach and resolve this error. 

Let us have a small discussion about how actually database connections are made:

In Dedicated mode, database client contacts listener and supplies the SERVICE NAME of the database. Then listener spawns a dedicated server process and hands off the client connection to this dedicated server process. TNS-12518 indicates a problem while handing off the client connection to the server process.

In Shared Server mode, database client contacts listener and supplies the SERVICE NAME of the database. Then listener hands off the client connection to one of the dispatcher configured for that service. TNS-12518 indicates a problem while handing off the client connection to the dispatcher server process.

Though this error is logged in the listener log, the listener is just the messenger, ORA-12518/TNS-12518 is mostly related to RDBMS and OS resources. 

​
Step 1. Is listener version compatible to the database
​

Check if you are using compatible listener version for your database version. 

If the database is 8i then use 8i or 9i listeners only. 10g listeners are not compatible to work with 8i databases. 

For 9i databases, 9i or 10g listeners can be used. 

For 10g databases, only 10g listeners can be used. 

The general rule is that use the higher version of the listener when there is a version mismatch between database and the listener. 

​Step 2. Gather more information from listener log
​

The first place you would look for the TNS-12518 error is the listener log. Usually the listener log would be located under $ORACLE_HOME/network/log directory. You can use lsnrctl status command output to look for the location of the listener log file.

$lsnrctl status 

- - - 

- - - 

Listener Parameter File /ora10g/home_ora10g/network/admin/listener.ora 

Listener Log File /ora10g/home_ora10g/network/log/listener.log 

- - - 

- - -

listener log gives the complete error stack and the database service name to which the client tried to connect to.

19-SEP-2007 13:55:34 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=__jdbc__)(USER=))(SERVICE_NAME=test.oracle.com)) * (ADDRESS=(PROTOCOL=tcp)(HOST=10.10.10.3)(PORT=36030)) * establish * test.oracle.com * 12518 

TNS-12518: TNS:listener could not hand off client connection 

 TNS-12547: TNS:lost contact 

  TNS-12560: TNS:protocol adapter error 

   TNS-00517: Lost contact 

    Linux Error: 32: Broken pipe

In the above example, listener log shows the complete error stack, the bottom error being 32 is the OS error. It also shows that the jdbc client from IP 10.10.10.3 has tried to connect to the database service ​test.oracle.com ​and failed with the error 12518.

Look for the lowest error in the stack. That is the error we have to concentrate on and try to resolve it. In the above example, the lower error is ​Linux Error: 32: Broken pipe.​

​
Step 3​.​ Are service​ ​handlers in blocked state
​

Check if the handlers are in blocked state. Check the output of the ​lsnrctl services​. Examine the status information under the database service name. From the listener log you would know which database service was affected by the 12518 error, now with the output of the lsnrctl services under that service name gather more information.

Service "test.oracle.com" has 1 instance(s). 

  Instance "db10g", status READY, has 2 handler(s) for this service... 

    Handler(s): 

      "DEDICATED" established:9 

​refused​:0 

​state​:ready 

         LOCAL SERVER 

      "D000" established:10 

​refused​:0 current:0 max:972 

​state​:ready 

         DISPATCHER  

         (ADDRESS=(PROTOCOL=tcp)(HOST=oid.mohan.com)(PORT=33487))

The highlighted state should be in​ ready​ state for the connection to be successful. If the state is in ​blocked​ then the connection are not possible. The state of a handler could be in blocked state in the following scenario: 

i. The database parameter processes reached its value. 

ii. The database is in the process of startup or shutting down. 

In shared server mode, the number of dispatchers should be set according to the load that you expect. ​lsnrctl services​ output shows the maximum number(max:997) of connections that the dispatcher would accept and the number connections refused (refused:0) by this dispatcher. If any connections refused by the dispatcher, then consider increasing the number of dispatchers. 

If you are using PFILE edit init.ora and increase the dispatchers parameter. If you are using SPFILE you can dynamically increase the dispatchers parameter by the​alter system set​ command. 

​
Step 4​. ​Is a local BEQ connection successful​

Check if local BEQ connection to the database works fine. It also verifies if the database is up and in good condition to accept the connection. If the database is down or in a hung state then a connection request to the database by the listener will not be possible. 

Connect to the database server via telnet or ssh and check if a local bequeath SQL*Plus connection works. In other words, issue:

sqlplus username/password [Enter]

This connection bypasses the listener and directly connects to the database via the BEQ (bequeath) protocol. If this fails, then the TNS-12518 listener error is simply a result of the database issue. 

One such error is:

ORA-12560: TNS:protocol adapter error

A possible cause for this error on Microsoft Windows servers, is that the Windows Database Service has not yet been created (common when creating a "standby" instance).

Resolution for this would be to create the Windows Service first by using the "oradim" command (see the Database Admin guide for details on oradim and service creation).

​
Step 5​. ​Has number of processes reached its limit?​

If local BEQ is successful, check the below query

SQL> select * from v$resource_limit; 

RESOURCE_NAME CURRENT_UTILIZATION MAX_UTILIZATION INITIAL_AL LIMIT_VALU 

-------------------- ------------------- --------------- ---------- ---------- 

processes 249 250 250 250 

sessions 54 82 280 280 

- - - - - -

- - - - - -

Verify if the ​processes​ or ​sessions​ reached its limit value. If these database parameters reached its limit value, then consider increasing it accordingly. 

In the above example, the ​processes​ parameter has been set to 250. Its ​MAX_UTILIZATION​ has reached the limit value of 250, so the processes parameter should be increased further to accomodate the number of incoming connections. 

Edit the init.ora and set the ​processes​ parameter to a higher value. By default, if you just increase the processes it is enough, the ​sessions​ value would automatically be increased.

Check the alert.log for a corresponding error such as "ksvcreate" process failed, etc.  Check timestamp against listener.log timestamps for causal relationship.

​Step 6​. ​Are OS kernel parameters configured for optimum?

Database is operated by a single user, normally it would be ​oracle​ user. At the Operating System level, there is a limit for the number of process spawned by a user. And also there is a limit for the total number of process running on the entire OS. 

The Oracle Database and the newly spawned processes would be owned by the ​oracle​ user. And so make sure that these values are set accordingly.On Unix these values are configured through the configurable OS kernel parameters and is specific to Operating Systems. You will have check your corresponding OS documentation for your OS. 

For example for HP-UX the configurable kernel parameters are, 

maxuprc  Maximum number of processes for each non-root user 

nproc      Maximum number of processes on the system 

​Step 7​. ​Does alert log have any errors?​

Look in alert log and look for any errors related to memory or process during the time the error TNS-12518 occurred in the listener log. If the alert log has any memory related errors, there there is a potential memory resource issue at the OS level. 

OS memory issues can be addressed by the below: 

i.   Make sure that the OS has been configured with the enough Swap memory. In case of Windows it is called as Virtual Memory. 

ii.  Reduce the size of SGA, thus the newly spawned server process will have some more system memory available. 

iii. Reduce the PGA size, so that the newly spawned server process would occupy less memory. 

iv. If you are in DEDICATED mode, try switching to MTS mode. 

To address memory issue for 32-bit Windows: Refer ​​Note 371983.1​​

If there are any memory or process related error in the alert log during the time the TNS-12518 is logged in the listener log, then those errors in the alert log should be focused on and should be solved at first. Because, the errors in the alert log is the base error for the TNS-12518 in the listener log. 

However, the errors in the alert logs are not being discussed in this article, they are out of the scope of this article.

​Step 8. If using a statically defined SID_DESC in the listener.ora file for your sid, ensure that it is configured properly.​ 

A common mistake is to include a (PROGRAM=EXTPROC) parameter:

  (SID_LIST= 

    (SID_DESC = 

      (GLOBAL_DBNAME = ORCL.oracle.com) 

      (SID_NAME = ORCL) 

      (PROGRAM=extproc) 

      (ORACLE_HOME = D:\\oracle\\product\\10.2.0\\db_1)))

This misconfiguration occurs when the PLSExtproc SID_DESC is copied, pasted and edited in the listener.ora file.  The inclusion of the PROGRAM line will cause an ORA-12518 to be returned to the client.  Heres the corrected SID_DESC for our example SID:

  (SID_LIST= 

     (SID_DESC = 

     (GLOBAL_DBNAME = ORCL.oracle.com) 

     (SID_NAME = ORCL) 

     (ORACLE_HOME = D:\\oracle\\product\\10.2.0\\db_1)))

Section II: Commonly Known Errors:

This section lists some of the known and reported errors. It is also recommended that you refer the Section I above for a generic troubleshooting approach to the error TNS-12518.

Below shows example error stack that can be found in the listener log. The last line in the error stack shows the actual operating system name. Depending on the OS, only the operating system name would be different. For example, if you encounter error 32: Broken pipe, according to the OS, the last line in the error stack would be different only by the OS name, as shown below.

Solaris Error: 32: Broken pipe 

HPUX Error: 32: Broken pipe 

Linux Error: 32: Broken pipe

_______________________________________________________________________________________________________________________________________ 

Error: 32: Broken pipe

​
Error stack in listener log: 

TNS-12518: TNS:listener could not hand off client connection 

 TNS-12547: TNS:lost contact 

   TNS-12560: TNS:protocol adapter error 

     TNS-00517: Lost contact 

       IBM/AIX RISC System/6000 Error: 32: Broken pipe 

​Cause: 

The error 32 indicates the communication has been broken while the listener is trying to hand off the client connection to the server process or dispatcher process.

​Action: 

1.  One of reason would be processes parameter being low, and can be verified by the v$resource_limit view. 

2. In Shared Server mode, check the lsnrctl services output and see if the dispatcher has refused any connections, if so, then consider increasing the number of dispatchers. 

3. Check the alert log for any possible errors. 

4. Memory resource is also another cause for this issue. Check the swap, memory usage of the OS.

5. If RAC/SCAN or listener is running in separate home, check the following note:

​​​Note: 1069517.1​​  ORA-12537 or TNS-12518 if Listener (including SCAN Listener) and Database are Owned by Different OS User ​

______________________________________________________________________________________________________________________________________

Error: 11: Resource temporarily unavailable

​Error stack in listener log: 

TNS-12518: TNS:listener could not hand off client connection 

 TNS-12549: TNS:operating system resource quota exceeded 

  TNS-12560: TNS:protocol adapter error 

   TNS-00519: Operating system resource quota exceeded 

    IBM/AIX RISC System/6000 Error: 11: Resource temporarily unavailable 

​Cause: 

As the error indicates operating system resource has exceeded. 

​Action: 

1. Increase the appropriate OS kernel parameters for ​maximum number of processes allowed per user​. 

For example for HP-UX the parameters are ​maxuprc​ and ​nproc​. 

______________________________________________________________________________________________________________________________________

Error: 12: Not enough space

 ​Error stack in listener log: 
​ TNS-12518: TNS:listener could not hand off client connection 

   TNS-12549: TNS:operating system resource quota exceeded 

     TNS-12560: TNS:protocol adapter error 

       TNS-00519: Operating system resource quota exceeded 

         IBM/AIX RISC System/6000 Error: 12: Not enough space 

​Cause: 

This is a memory related issue. The error indicates that there is not enough memory available to spawn and hand off the client connections. 

Typical problems are: 

- Out of system memory / swap 

- Out of process slots in the process table 

- Streams resources depleted 

- Out of File Handles 

- sga memory usage 

​Action: 

1. Check in the alert log for any possible memory related error. 

2. Increase swap/Virtual memory if possible the available memory. 

3. SGA and PGA can be reduced to address the memory consumption. 

4. MTS mode can be used to reduce the amount of process and memory consumption. 

________________________________________________________________________________________________________________________________________

Error: Connection Pooling limit reached

 ​Error stack in listener log: 

 TNS-12518 TNS:listener could not hand off client connection 

  TNS-12564 TNS:connection refused 

   TNS-12602 TNS: Connection Pooling limit reached 

​Action: 

1. Try increasing initial number of dispatchers. 

_______________________________________________________________________________________________________________________________________ 

Error: 24: Too many open files

​Error stack in listener log:​

    TNS-12518: TNS:listener could not hand off client connection

      TNS-12560: TNS:protocol adapter error

       TNS-00530: Protocol adapter error

        Linux Error: 24: Too many open files

or    

 ​   ​TNS-12518: TNS:listener could not hand off client connection​
   ​   TNS-12560: TNS:protocol adapter error

       TNS-00530: Protocol adapter error

        Solaris Error: 24: Too many open files

​Error Description:
 ​Out of file descriptors

​Cause:
​Can be caused by the following unpublished defect:

BUG 13078786 - LISTENER GOES DOWN SUDDENLY W/ LINUX ERROR: 32: BROKEN PIPE

​Action:

​See ​​Note 1527483.1​​  11.2 : ORA-12518 Listener Hangs and Reports "Too Many Open Files" 

Use prescribed workaround OR apply one-off patch to your environment if available.

_______________________________________________________________________________________________________________________________________

Error: 1: Not Owner

​Error Description:
​The listener doesnt own or have permissions to write a necessary file

​Cause:

​The listener doesnt have adequate permission on socket files​

Error stack in listener log:​

TNS-12518: TNS:listener could not hand off client connection 

   TNS-12560: TNS:protocol adapter error 

     TNS-00530: Protocol adapter error 

        HPUX Error: 1: Not owner

​Action:  ​Clear /var/tmp/.oracle/ directory

​IMPORTANT NOTE:​  In RAC environment, please use caution when removing existing socket files.  See ​​Note 2099377.1​​  How to remove Network socket files in a RAC Environment for Cluster/Resource startup issues

========================================================================================================================= 

Section III: Errors Specific to Windows

It is also recommended that you refer the Section I above for a generic troubleshooting approach to the error TNS-12518.

This section briefly describes about the errors that are encountered on Windows Operating System. TNS-12518 most commonly occurs on 32-bit OS due to its memory constraint, however TNS-12518 can occur on 64-bit OS as well. See Note 873752.1 for more information on Windows memory addressing and the 3GB switch.

_______________________________________________________________________________________________________________________________________  

Error: 2: No such file or directory 

​Error stack in listener log:​  

    TNS-12518: TNS:listener could not hand off client connection   

      TNS-12560: TNS:proto adapter error    

        TNS-00530: Proto adapter error     

        32-bit Windows Error: 2: No such file or directory 

​Error Description: ​ERROR_FILE_NOT_FOUND 2 The system cannot find the file specified. 

​Cause:​ This indicates the database service is not actually available 

​Action:​ 1.Verify if the intended database really up and accepting local BEQ connections. 

_______________________________________________________________________________________________________________________________________  

Error: 233: Unknown error

​Error stack in listener log: 

 TNS-12518: TNS:listener could not hand off client connection 

  TNS-12560: TNS:protocol adapter error 

   TNS-00530: Protocol adapter error 

    32-bit Windows Error: 233: Unknown error 

​Error Description: 
​ERROR_PIPE_NOT_CONNECTED 

233 

No process is on the other end of the pipe. 

​Cause: 
​The communication has been broken while the listener is trying to hand off the client connection to the server process or dispatcher process. 

​Action: 
​Refer ​​Note 371983.1​​

_______________________________________________________________________________________________________________________________________

Error: 54: Unknown error

​Error stack in listener log: 

 TNS-12518: TNS:listener could not hand off client connection 

  TNS-12571: TNS:packet writer failure 

   TNS-12560: TNS:protocol adapter error 

    TNS-00530: Protocol adapter error 

      32-bit Windows Error: 54: Unknown error 

​Error Description: 

ERROR_NETWORK_BUSY 

54 

0x36 

The network is busy. 

​Cause: 

This indicates a bottleneck at the network layer(TCP/IP). 

​Action: 

1.Try increasing dispatchers and shared servers.

Error: 10022: Unknown error

​Error stack in listener log:​   

12518 TNS-12518: TNS:listener could not hand off client connection 

     TNS-12560: TNS:protocol adapter error TNS-00534: Failed to grant connection ownership to child           

         64-bit Windows Error: 10022: Unknown error 

​Error Description:
​Error: 10022: Invalid Argument

​Cause:
​An invalid argument was supplied.

​Action: 

Check the Control Panel Services Properties and confirm both listener and instance are started with same "Log on As" account.

Note that in 12c, the account would be the Oracle Home User. 

Note that a JDBC Thin connection from a Windows client might also yield the 12518:

MacromediaOracle JDBC Driver][Oracle]Connection refused, (DESCRIPTION=(TMP=)(VSNNUM=186646784)(ERR=12518)(ERROR_STACK= (ERROR=(CODE=12518)(EMFI=4))(ERROR=(CODE=12560)(EMFI=4))(ERROR=(CODE=530)(EMFI=4))(ERROR=(BUF=64-bit Windows Error:203: Unknown error))))

---

REFERENCES

​​NOTE:873752.1​​ - Windows Memory Configuration: 32-bit and 64-bit

​​NOTE:1069517.1​​ - ORA-12537 / ORA-12547 or TNS-12518 if Listener (including SCAN Listener) and Database are Owned by Different OS User

​​NOTE:371983.1​​ - 10gR2 Dedicated Connections Intermittently Fail with TNS-12518

​​NOTE:1527483.1​​ - 11.2 : ORA-12518 Listener Hangs and Reports "Too Many Open Files" 

---

---

1.6.1.2  ​ORA-12537 / ORA-12547 or TNS-12518 if Listener (including SCAN Listener) and Database are Owned by Different OS User (文档 ID 1069517.1)

​In this Document

​​Symptoms​​

​​Cause​​

​​Solution​​

​​References​​

---

APPLIES TO:

Oracle Database - Enterprise Edition - Version 11.2.0.3 and later

Oracle Database Configuration Assistant - Version 11.1.0.7 and later

Oracle Net Services - Version 11.2.0.2 and later

Information in this document applies to any platform.

SYMPTOMS

In environment where listener home (including SCAN listener which resides in GRID Infrastructure/ASM home) and database home are owned by different OS user, ORA-12537 could happen when connecting through listener, when creating database through DBCA, or when installing database software and creating a database in runInstaller. Job Role Separation is a typical example as SCAN and local grid home listener is owned differently than database.

• Error detail

12537, 00000, "TNS:connection closed"

// *Cause: "End of file" condition has been reached; partner has disconnected.

// *Action: None needed; this is an information message.

Before the error is reported, connection could hang for a while, and a core file may also be generated.

• Screen output:

SQL> conn system/xxx@OSES

ERROR:

ORA-12537: TNS:connection closed

​OR​

PRCR-1079 : Failed to start resource ora.db11.db

CRS-5011: Check of resource db11 failed: details at ..

CRS-2674: Start of ora.db11.db on racnode1 failed

ORA-12546: TNS:permission denied

• listener_scan​n​.log or listener.log

TNS-12518: TNS:listener could not hand off client connection

TNS-12546: TNS:permission denied

 TNS-12560: TNS:protocol adapter error

  TNS-00516: Permission denied

   Linux Error: 13: Permission denied

​OR​

TNS-12518: TNS:listener could not hand off client connection

TNS-12547: TNS:lost contact

 TNS-12560: TNS:protocol adapter error

  TNS-00517: Lost contact

   Linux Error: 32: Broken pipe

• runInstaller error if installing

INFO: Starting Output Reader Threads for process /ocw/grid/bin/kfod

INFO: Parsing KFOD-00300: OCI error [-1] [OCI error] [ORA-12547: TNS:lost contact

INFO: Parsing ] [12547]

INFO: Parsing

INFO: The process /ocw/grid/bin/kfod exited with code 1

..

SEVERE: [FATAL] [INS-30502] No ASM disk group found.

   CAUSE: There were no disk groups managed by the ASM instance +ASM1.

CAUSE

1. In environments where the listener is not run in the same ORACLE_HOME where the database resides, the listener owner (including SCAN listener) may not be able to access the oracle binary in the database home.  This is common in RAC or whenever a GRID_HOME and a database ORACLE_HOME are installed.

As listener owner:

​$ ls -l $RDBMS_HOME/bin/oracle​

ls: /home/oracle/app/oracle/product/11.2/db/bin/oracle: 

​Permission denied

2. Oracle binary in database home has wrong permission:

​ls -l $RDBMS_HOME/bin/oracle​

​-rwxr-x--x​ 1 oracle asmadmin 184286251 Aug  9 16:25 /home/oracle/app/oracle/product/11.2/db/bin/oracle

The permission "​-rwxr-x--x"​ is wrong as its missing suid bit, oracle binary should have permission of 6751: 

​-rwsr-s--x​ 1 oracle asmadmin 184286251 Aug  9 16:25 /home/oracle/app/oracle/product/11.2/db/bin/oracle

Note: If Job Role Separation is in place, the group will be , otherwise it will be

3. File System for database home does not support setuid/suid or has nosetuid/nosuid set:

​mount| grep <mount_point_of_oracle_home>​

/home/oracle on /dev/dsk/diskoracle read/write/nosuid..

4. RDBMS_HOME/lib has wrong ownership/permission:

As listener owner:

$ ls -l $RDBMS_HOME/lib

ls: /home/oracle/app/oracle/product/11.2/db/lib: 

​Permission denied

5. Another cause may be permissions on the RDBMS Directory structure or Home directory which needs to be accessed by the CRS user.

Check that the RDBMS $ORACLE_HOME is set to 755.

This can