oracle数据库报错ORA-27300 ORA-27301等错误

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了oracle数据库报错ORA-27300 ORA-27301等错误相关的知识,希望对你有一定的参考价值。

oracle数据库日志报错ORA-27300 ORA-27301 ORA-27302 ORA-27157处理记录:

1、事件的原因排查

应用连接数据库失败,先连接数据库服务器,启动数据库服务恢复应用业务,然后排查数据库crush原因:
1)查看messages日志是否有与Oracle用户相关的出错信息

# cat /etc/redhat-release 
CentOS Linux release 7.2.1511 (Core)
# cat /var/log/messages |grep failed

2)查看用户邮件日志是否有与Oracle用户相关的出错信息

# tail -n 200 /var/mail/oracle

3)检查跟踪日志文件

SQL> select value from v$diag_info where name =‘Diag Trace‘;
VALUE
--------------------------------
/app/oracle/diag/rdbms/crm/crm/trace

$ cat /app/oracle/diag/rdbms/crm/crm/trace/alert_crm.log |grep ORA-
$ cat /app/oracle/diag/rdbms/crm/crm/trace/alert_crm.log |grep err
$ cat /app/oracle/diag/rdbms/crm/crm/trace/alert_crm.log |grep fail 

SQL> select name,value from v$diag_info;
NAME              VALUE
--------------------------------
Diag Enabled    TRUE
ADR Base    /app/oracle/
ADR Home    /app/oracle/diag/rdbms/crm/crm
Diag Trace  /app/oracle/diag/rdbms/crm/crm/trace
Diag Alert  /app/oracle/diag/rdbms/crm/crm/alert
Diag Incident   /app/oracle/diag/rdbms/crm/crm/incident
Diag Cdump  /app/oracle/diag/rdbms/crm/crm/cdump
Health Monitor  /app/oracle/diag/rdbms/crm/crm/hm
Default Trace File  /app/oracle/diag/rdbms/crm/crm/trace/crm_ora_10974.trc
Active Problem Count    1
Active Incident Count   1

查看log.xml搜索问题出现时间点

$ less /app/oracle/diag/rdbms/crm/crm/alert/log.xml
Errors in file /app/oracle/diag/rdbms/crm/crm/trace/crm_dbw2_80900.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
Errors in file /app/oracle/diag/rdbms/crm/crm/trace/crm_dbw3_80902.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1

2、事件的解决处理

This could be due to some outside user or application removing the semaphores/shared memory. 
To monitor the semaphore/shared memory state we can use the following methods:

Setup a cronjob to run every 5-10min and dump the output of ‘ipcs‘ and ‘ps - ef‘ to a file with a timestamp. 
Rotate your logs every 4-7 days to build a history. 
Then if the problem re-occurs, we can at least try to make sure ‘ipcrm‘ wasn‘t the culprit and get some general information of the state of the IPC resources plus the processes running.

You can also consult with your sysadmin to check if there is any OS level auditing that can be turned on to audit the usage of commands like ‘ipcrm‘ which can remove shared memory segments /semaphore sets.

Note: This issue can happen on different platform, but in case you encounter the issue in RHEL7.2, then please also check below RHEL7.2 specific information.

In  RHEL7.2 operating system setting RemoveIPC=YES crashes the database.The default value for RemoveIPC in RHEL7.2 is YES.

         Workaround :

      1) Set RemoveIPC=no in /etc/systemd/logind.conf if it is not in that file

      2) Reboot the server or restart systemd-logind as follows:
       # systemctl daemon-reload
       # systemctl restart systemd-logind

Centos7.2有个特性,默认该配置是为yes

技术图片

# vim /etc/systemd/logind.conf
RemoveIPC=no
# systemctl daemon-reload
# systemctl restart systemd-logind

以上是关于oracle数据库报错ORA-27300 ORA-27301等错误的主要内容,如果未能解决你的问题,请参考以下文章

HP-UX平台Oracle启动实例遭遇:ORA-27154,ORA-27300,ORA-27301,ORA-27302

Ora-27300 OS system dependent operation:fork faile

ORA-27300: OS system dependent operation:sendmsg failed with status: 105 ORA-27301: OS failure messa

RHEL 7.2 ORA-27300 ORA-27301 Database Crash

ORA-00603ORA-27504ORA-27300ORA-27301:No buffer

12C RAC中的一个数据库实例自动crash并报ORA-27157ORA-27300等错误