一个在centos7.2上的11204库的实例crash了,检查alert发现如下:
Wed Sep 21 00:00:11 2016
Errors in file /apps/oracle/diag/rdbms/xberp/xberp/trace/xberp_ckpt_17537.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
Wed Sep 21 00:00:11 2016
Errors in file /apps/oracle/diag/rdbms/xberp/xberp/trace/xberp_dbrm_17523.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
Wed Sep 21 00:00:11 2016
Errors in file /apps/oracle/diag/rdbms/xberp/xberp/trace/xberp_mman_17527.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
Wed Sep 21 00:00:11 2016
Errors in file /apps/oracle/diag/rdbms/xberp/xberp/trace/xberp_j002_14570.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
CKPT (ospid: 17537): terminating the instance due to error 27157
Wed Sep 21 00:00:11 2016
Errors in file /apps/oracle/diag/rdbms/xberp/xberp/trace/xberp_j000_14566.trc:
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
Instance terminated by CKPT, pid = 17537
Errors in file /apps/oracle/diag/rdbms/xberp/xberp/trace/xberp_ckpt_17537.trc:
ORA-27300: OS system dependent operation:semctl failed with status: 22
ORA-27301: OS failure message: Invalid argument
ORA-27302: failure occurred at: sskgpwrm1
ORA-27157: OS post/wait facility removed
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
Wed Sep 21 10:20:34 2016
描述:
在centos7.2中,systemd-logind 服务引入了一个新特性,该新特性是:当一个user 完全退出os之后,remove掉所有的IPC objects。
该特性由/etc/systemd/logind.conf参数文件中RemoveIPC选项来控制。
在centos7.2中,RemoveIPC的默认值为yes
因此,当最后一个oracle 或者Grid用户退出时,操作系统会remove 掉这个user的shared memory segments and semaphores
由于Oracle ASM 和database 使用 shared memory segments ,remove shared memory segments将会crash掉Oracle ASM and database instances.
Oracle Linux 7.2 通过在/etc/systemd/logind.conf配置文件中明确设置RemoveIPC为no,Oracle Linux7.2 避免了该问题,
变通的解决方法:
1) Set RemoveIPC=no in /etc/systemd/logind.conf
2) Reboot the server or restart systemd-logind as follows:
# systemctl daemon-reload
# systemctl restart systemd-logind
这些错误跟os有关,在mos上有文档可能参考:
ALERT: Setting RemoveIPC=yes on Redhat 7.2 Crashes ASM and Database Instances as Well as Any Application That Uses a Shared Memory Segment (SHM) or Semaphores (SEM) (文档 ID 2081410.1)