Oracle 11gR2 RAC 数据库不能连接(ORA-12537: TNS:connection closed)的解决

Oracle 11gR2 RAC 数据库不能连接(ORA-12537: TNS:connection closed)的解决

[oracle@rac01 ~]$ sqlplus /nolog

SQL*Plus: Release 11.2.0.1.0 Production on Wed Aug 29 21:06:10 2012

Copyright (c) 1982, 2009, Oracle.  All rights reserved.

SQL> connect system/***@astt
ERROR:
ORA-12537: TNS:connection closed

[root@rac01 app]# ls -ltr
drwxr-xr-x 3 root   oinstall 4096 08-28 19:05 grid
drwxrwx--- 6 grid   oinstall 4096 08-29 00:03 oraInventory
drwxrwxr-x 7 oracle oinstall 4096 08-29 00:20 oracle
[root@rac01 app]# chmod 755 oracle
[root@rac01 app]# chmod 755 grid
[root@rac01 app]# ls -ltr
drwxr-xr-x 3 root   oinstall 4096 08-28 19:05 grid
drwxrwx--- 6 grid   oinstall 4096 08-29 00:03 oraInventory
drwxr-xr-x 7 oracle oinstall 4096 08-29 00:20 oracle

[oracle@rac01 ~]$ sqlplus /nolog

SQL*Plus: Release 11.2.0.1.0 Production on Wed Aug 29 21:28:23 2012

Copyright (c) 1982, 2009, Oracle.  All rights reserved.

SQL> connect system/***@astt;
????
SQL>

SQL> select instance_name from gv$instance;

INSTANCE_NAME
----------------
astt1
astt2

SQL> select instance_name from v$instance;

INSTANCE_NAME
----------------
astt1

SQL>

参考:

metalink  [ID 1069517.1]:

Symptoms
In environment where listener home (including SCAN listener which resides in Grid Infrastructure/ASM home) and database home are owned by different OS user, ORA-12537 could happen when connecting through listener, when creating database through DBCA, or when installing database software and creating a database in runInstaller. Job Role Separation is a typical example as SCAN and local grid home listener is owned differently than database.

Error detail

12537, 00000, "TNS:connection closed"
// *Cause: "End of file" condition has been reached; partner has disconnected.
// *Action: None needed; this is an information message.

Before the error is reported, connection could hang for a while, and a core file may also be generated.

Screen output:
SQL> conn system/xxx@OSES
ERROR:
ORA-12537: TNS:connection closed

OR

PRCR-1079 : Failed to start resource ora.db11.db
CRS-5011: Check of resource db11 failed: details at ..
CRS-2674: Start of ora.db11.db on racnode1 failed
ORA-12546: TNS:permission denied

listener_scann.log or listener.log
TNS-12518: TNS:listener could not hand off client connection
TNS-12546: TNS:permission denied
TNS-12560: TNS:protocol adapter error
  TNS-00516: Permission denied
   Linux Error: 13: Permission denied

OR

TNS-12518: TNS:listener could not hand off client connection
TNS-12547: TNS:lost contact
TNS-12560: TNS:protocol adapter error
  TNS-00517: Lost contact
   Linux Error: 32: Broken pipe

runInstaller error if installing

INFO: Starting Output Reader Threads for process /ocw/grid/bin/kfod
INFO: Parsing KFOD-00300: OCI error [-1] [OCI error] [ORA-12547: TNS:lost contact
INFO: Parsing ] [12547]
INFO: Parsing
INFO: The process /ocw/grid/bin/kfod exited with code 1
..
SEVERE: [FATAL] [INS-30502] No ASM disk group found.
   CAUSE: There were no disk groups managed by the ASM instance +ASM1.

Cause

1. Listener owner (including SCAN listener) can not access oracle binary in database home:

As listener owner:

$ ls -l $RDBMS_HOME/bin/oracle
ls: /home/oracle/app/oracle/product/11.2/db/bin/oracle: Permission denied

2. Oracle binary in database home has wrong permission:

ls -l $RDBMS_HOME/bin/oracle
-rwxr-x--x 1 oracle asmadmin 184286251 Aug  9 16:25 /home/oracle/app/oracle/product/11.2/db/bin/oracle

The permission "-rwxr-x--x" is wrong as it's missing suid bit, oracle binary should have permission of 6751:

-rwsr-s--x 1 oracle asmadmin 184286251 Aug  9 16:25 /home/oracle/app/oracle/product/11.2/db/bin/oracle

Note: If Job Role Separation is in place, the group will be , otherwise it will be

3. File System for database home does not support setuid/suid or has nosetuid/nosuid set:

mount| grep
/home/oracle on /dev/dsk/diskoracle read/write/nosuid..

Solution
Solution is to make sure file system for database home has setuid/suid set, database binary($RDBMS_HOME/bin/oracle) has correct ownership and permission, and listener owner is able to access database oracle binary (as listener owner, "ls -l $RDBMS_HOME/bin/oracle" will tell)

If its Job Role Separation environment, please refer to the following document:

Oracle® Grid Infrastructure
Installation Guide
11g Release 2 (11.2)

Oracle ASM Job Role Separation Option with SYSASM

官方解释如下：

Applies to:
Oracle Net Services - Version: 8.1.7.0 to 11.2.0.1 - Release: 8.1.7 to 11.2
Generic UNIX
Checked for relevance on 30-Apr-2010
Symptoms
Connection to the database server fails with ORA-12546 or ORA-12537

Local connections working fine but the connection via listener fails with ORA-12537 or ORA-12547

[oracle@myhost]$ sqlplusmyuser/mypass@mytns

SQL*Plus: Release 11.1.0.7.0 - Production on Fri Apr 30 21:32:18 2010

Copyright (c) 1982, 2008, Oracle. All rights reserved.

ERROR:
ORA-12537: TNS:connection closed

The listener log for the failing connection shows following error message -

TNS-12518: TNS:listener could not hand off client connection
TNS-12546: TNS:permission denied
TNS-12560: TNS:protocol adapter error
TNS-00516: Permission denied
Linux Error: 13: Permission denied

Or

TNS-12518: TNS:listener could not hand off client connection
TNS-12547: TNS:lost contact
TNS-12560: TNS:protocol adapter error
TNS-00517: Lost contact
IBM/AIX RISC System/6000 Error: 32: Broken pipe

Cause
Permissions on oracle binary ( $ORACLE_HOME/bin/oracle ) are not set correctly.

Solution
The OS level error message in the listener log "Linux Error: 13: Permission denied" indicates that the listener was not able to hand off the connection to oracle processes or not able to spawn a new dedicated user process because of lack of permission at the OS level. To resolve the error -

1. Check and correct the permission on /var/tmp/.oracle directory. Permissions on this directory should be 777 with sticky bit set.

# chmod 01777 /var/tmp/.oracle

2. Check and correct permissions on $ORACLE_HOME/bin/oracle binary and make sure that it is set to 6751.

$ chmod 6751 $ORACLE_HOME/bin/oracle

检查自己环境，确实发现不是正确的权限

[root@ract1 bin]# pwd
/oracle/product/11.2.0/bin
[root@ract1 bin]# ls -la oracle
-rwxr-s--x 1 oracle asmadmin 228886426 Jan 7 15:46 oracle
[root@ract1 bin]# chmod 6751 oracle- -注意使用root用户给权限
[root@ract1 bin]# ls -la oracle
-rwsr-s--x 1 oracle asmadmin 228886426 Jan 7 15:46 oracle

修改后登陆正常：

[oracle@ract2 ~]$ sqlplussys/oracle@10.20.31.36:1521/racdbas sysdba

SQL*Plus: Release 11.2.0.2.0 Production on Thu Jan 17 14:48:25 2013

Copyright (c) 1982, 2010, Oracle. All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Data Mining and Real Application Testing options

SQL>