Oracle 12c和18c中的MGMTDB(下)

Oracle 12c和18c中的MGMTDB(下)

上一篇参考： http://blog.itpub.net/26736162/viewspace-2132763/

12c Grid Infrastructure 管理资料库（GIMR） 问答 (文档 ID 2047608.1)

文档内容

用途

问题和答案

什么是管理资料库？

管理资料库是用来干什么的？

管理资料库的数据文件放在哪里？

在安装升级的时候，如果不配置管理资料库会有什么样的问题？

有哪些和管理资料库相关的资源？

如果管理资料库停掉，怎样手工启动？

怎样“cd”到管理资料库的子目录去查看跟踪文件？

有没有必要去手动备份或优化资料库？

在Oracle Restart环境下，GIMR会被默认创建吗？

需要分配多少磁盘空间给资料库？

参考

---

适用于:

Oracle Database - Enterprise Edition - 版本 12.1.0.1 和更高版本
Oracle Database Cloud Schema Service - 版本 N/A 和更高版本
Oracle Database Exadata Express Cloud Service - 版本 N/A 和更高版本
Oracle Database Exadata Cloud Machine - 版本 N/A 和更高版本
Oracle Cloud Infrastructure - Database Service - 版本 N/A 和更高版本
本文档所含信息适用于所有平台

用途

此文档是一个对 12c Grid Infrastructure 的新功能：管理资料库的 FAQ 文档。

问题和答案

 

什么是管理资料库？

管理资料库是一个由 12c 集群软件管理的单实例数据库。因为此数据库为单实例数据库，所以它只在集群的一个节点运行。也正是因为这个数据库是被集群管理的，所以如果数据库运行节点 down 掉，数据库也会被自动切换到另外的节点。

 

管理资料库是用来干什么的？

在 12c，管理资料库是一个用来集中存放 Cluster Health Monitor 数据（又称：CHM/OS，ora.crf）以及其他的数据的资料库。

 

管理资料库的数据文件放在哪里？

在12.1版本，管理资料库默认和 OCR、Voting disk 使用一样的共享存储。在12.2，在安装时可以指定使用单独的diskgroup。

 

在安装升级的时候，如果不配置管理资料库会有什么样的问题？

管理资料库在 12.1是非强制的，如果在使用 OUI 安装升级的时候未选择该选项，所有的依赖于管理资料库的功能都会被禁用（比如 CHM/OS 等等）。

在 12.1.0.2 中，管理资料库变为强制安装(除了 Exadata环境)。

   

有哪些和管理资料库相关的资源？

从“crsctl stat res –t”的输出结果来看，以下是和管理资料库有关的资源：

ora.mgmtdb
1 ONLINE ONLINE racnode1f Open,STABLE

ora.MGMTLSNR
1 ONLINE ONLINE racnode1f 169.254.118.96 10.3.0.70,STABLE

在 OS 层面，可以看到数据库“-MGMTDB”和监听器“MGMTLSNR”：

ps -ef| grep pmon_-MGMTDB

grid 3500 1 0 Jun04 ? 00:06:18 mdb_pmon_-MGMTDB

ps -ef| grep MGMTLSNR

grid 3308 1 0 Jun04 ? 00:00:55 /ocw/c101/bin/tnslsnr MGMTLSNR -no_crs_notify -inherit

 

如果管理资料库停掉，怎样手工启动？

管理资料库被集群管理而且应该一直运行。如果资料库因为一些原因停掉，下面的 srvctl 命令可以用来启动资料库：

Usage: srvctl start mgmtdb [-startoption <start_option>] [-node <node_name>]
Usage: srvctl start mgmtlsnr [-node <node_name>]

 

怎样“cd”到管理资料库的子目录去查看跟踪文件？

通常来讲，一般没有必要去查看管理资料库的跟踪文件。如果需要查看则必须在数据库名前加上“./”以避免以下的错误，因为资料库数据库名前面有个“-”：

cd -MGMTDB
-bash: cd: -M: invalid option
cd: usage: cd [-L|-P] [dir]

cd  ./ -MGMTDB        ==>> this will work as "./" is specified

more -MGMTDB_m000_9912.trc
more: unknown option "-M"
usage: more [-dflpcsu] [+linenum | +/pattern] name1 name2 ...

 

more  ./ -MGMTDB_m000_9912.trc
Trace file /home/grid/app/grid/diag/rdbms/_mgmtdb/-MGMTDB/trace/-MGMTDB_m000_9912.trc
Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Partitioning, Automatic Storage Management, Advanced Analytics and Management Database options

 

有没有必要去手动备份或优化资料库？

目前没有必要。

在Oracle Restart环境下，GIMR会被默认创建吗？

不。在Oracle Restart环境下GIMR默认不会被创建。

 

需要分配多少磁盘空间给资料库？

OCR 和管理资料库在外部冗余的情况下：

至少 5.2GB 的空间的 OCR 用来存储资料库（4.5GB+300MB Voting files+400MB OCR），如果节点数超过 4，则每个多出的节点多分配 500MB 的空间。 如：一个六节点的集群应该使用 6.2GB 空间。

Oracle 的文档说明：  http://docs.oracle.com/database/121/CWLIN/storage.htm#CHDDCAHD

 

 

参考

NOTE:1589394.1  - How to Move/Recreate GI Management Repository (GIMR / MGMTDB) to Different Shared Storage (Diskgroup, CFS or NFS etc)
NOTE:2065175.1  - MDBUtil: GI Management Repository configuration tool

12.2：如何创建 GI Management Repository (文档 ID 2364214.1)

文档内容

目标

解决方案

参考

---

适用于:

Oracle Database - Enterprise Edition - 版本 12.2.0.1 和更高版本
本文档所含信息适用于所有平台

目标

本文概括了创建 12.2 GIMR (GridInfrastructure Management Repository) Database (MGMTDB) 的步骤。

解决方案

 
1. 发出 DBCA 命令来创建 MGMTDB 的 container。

以 grid 用户，

$ <GI_HOME>/bin/dbca -silent -createDatabase -createAsContainerDatabase true -templateName MGMTSeed_Database.dbc
                                                                             -sid -MGMTDB
                                                                             -gdbName _mgmtdb
                                                                             -storageType ASM
                                                                             -diskGroupName  <DG target>
                                                                             -datafileJarLocation <GI_HOME>/assistants/dbca/templates
                                                                             -characterset AL32UTF8
                                                                             -autoGeneratePasswords
                                                                             -skipUserTemplateCheck

注意，根据需要修改<DG target> （比如 +DATA, +REDO, ...）

2. 创建 pluggable GIMR 库。

以 grid 用户，

$ <GI_HOME>/bin/mgmtca -local

 

注意，为了避免使用上述的手工步骤，您可以利用自动化工具 ： mdbutil.pl  ，请参阅  Note: 2065175.1

举例如下：
(oracle)$ ./mdbutil.pl --addmdb --target=+DATAC1
2017-04-04 14:47:48: I Starting To Configure MGMTDB at +DATAC1...
2017-04-04 14:47:57: I Container database creation in progress...
2017-04-04 14:55:20: I Plugable database creation in progress...
2017-04-04 14:57:03: I Executing "/tmp/mdbutil.pl --addchm" on xxxadm01 as root to configure CHM.
root@ xxxadm01 password:
2017-04-04 15:01:36: I MGMTDB & CHM configuration done!

参考

NOTE:2065175.1  - MDBUtil: GI Management Repository configuration tool

如何在不同的共享存储（磁盘组、CFS、NFS 等）上移动/重建 GIMR (MGMTDB) (文档 ID 2225754.1)

文档内容

目标

解决方案

参考

---

适用于:

Oracle Database - Enterprise Edition - 版本 12.1.0.1 到 12.2.0.1 [发行版 12.1 到 12.2]
本文档所含信息适用于所有平台

目标

本文目的给出转移 12c 的 GIMR（Grid Infrastructure Management Repository）到其它共享存储的概要步骤。

对于12.2 请参考 note 2065175.1  - MDBUtil: GI Management Repository configuration tool 来创建GIMR或者移动数据文件到不同的磁盘组上。这篇文档的内容同样适用于在12.2 上删除已经存在的mgmtdb和创建新的mgmtdb

这个过程执行时GIMR 管理数据库中原有的数据无法保留。请通过 grid 用户，在任何节点执行下面命令，以手动方式备份“管理数据库”：

$ <GI_HOME>/bin/oclumon dumpnodeview -allnodes -v > <backup-file>

 

解决方案

 

1. 停止并禁用 ora.crf 资源

需要使用 root 用户，在每一个节点执行命令：

# <GI_HOME>/bin/crsctl stop res ora.crf -init
# <GI_HOME>/bin/crsctl modify res ora.crf -attr ENABLED=0 -init

不要停止 ora.mgmtlsnr 资源 ora.mgmtdb，否则第二步会遇到下面错误：

Oracle Grid Management database is running on node "". Run dbca on node "" to delete the database.

2. 执行 DBCA 命令删除管理数据库

使用 grid 用户执行下面命令，来确定 MGMTDB 运行的节点：

$ <GI_HOME>/bin/srvctl status mgmtdb

在MGMTDB运行的节点上，执行以下命令：

$  <GI_HOME>/bin/dbca -silent -deleteDatabase -sourceDB -MGMTDB
Connecting to database
4% complete
9% complete
14% complete
19% complete
23% complete
28% complete
47% complete
Updating network configuration files
48% complete
52% complete
Deleting instance and datafiles
76% complete
100% complete
Look at the log file "/home/grid/app/grid/cfgtoollogs/dbca/_mgmtdb.log" for further details.

 

注意：

如果管理MGMTDB库没有运行，DBCA 会遇到下面错误：

Oracle Grid Management database is running on node "". Run dbca on node "" to delete the database.

解决该问题的办法是通过下面方式手动删除：

- 在文件系统层面，删除管理数据库的相关文件。并删除 oratab 的相关条目

- 使用 srvctl 删除 mgmtdb 资源：srvctl remove mgmtdb

 

3. 重建管理数据库（MGMTDB）

3A. 针对 12.1.0.1 版本

在任一节点使用 grid 用户执行 DBCA 命令：

$  <GI_HOME>/bin/dbca -silent -createDatabase -templateName MGMTSeed_Database.dbc -sid -MGMTDB -gdbName _mgmtdb -storageType ASM -diskGroupName <+NEW_DG> -datafileJarLocation <GI_HOME>/assistants/dbca/templates -characterset AL32UTF8 -autoGeneratePasswords -oui_internal
Copying database files
1% complete
3% complete
10% complete
17% complete
24% complete
31% complete
35% complete
Creating and starting Oracle instance
37% complete
42% complete
47% complete
52% complete
53% complete
56% complete
58% complete
Registering database with Oracle Grid Infrastructure
64% complete
Completing Database Creation
68% complete
78% complete
89% complete
100% complete
Look at the log file "/home/grid/app/grid/cfgtoollogs/dbca/_mgmtdb/_mgmtdb1.log" for further details.

注意：无论是新创建还是修改磁盘组的过程，我们都建议把磁盘组的参数 compatible.asm 和 compatible.rdbms 设置成 12.1。

以上例子是创建了管理数据库到一个新的磁盘组，如果是使用共享的 NFS/CFS， 请参考以下例子：

<GI_HOME>/bin/dbca -silent -createDatabase -templateName MGMTSeed_Database.dbc -sid -MGMTDB -gdbName _mgmtdb -storageType FS -datafileDestination <NFS_Location> -datafileJarLocation <GI_HOME>/assistants/dbca/templates -characterset AL32UTF8 -autoGeneratePasswords -oui_internal

 

3B. 针对 12.1.0.2 版本

a. 在任一节点使用 grid 用户执行 DBCA 命令，并指定好期望使用的磁盘组名：

$ <GI_HOME>/bin/dbca -silent -createDatabase -sid -MGMTDB -createAsContainerDatabase true -templateName MGMTSeed_Database.dbc -gdbName _mgmtdb -storageType ASM -diskGroupName <+NEW_DG> -datafileJarLocation $GI_HOME/assistants/dbca/templates -characterset AL32UTF8 -autoGeneratePasswords -skipUserTemplateCheck

Copying database files
1% complete
3% complete
10% complete
17% complete
24% complete
31% complete
35% complete
Creating and starting Oracle instance
37% complete
42% complete
47% complete
52% complete
53% complete
56% complete
58% complete
Registering database with Oracle Grid Infrastructure
64% complete
Completing Database Creation
68% complete
78% complete
89% complete
100% complete
Look at the log file "<GI_HOME>/cfgtoollogs/dbca/_mgmtdb/_mgmtdb1.log" for further details.

注意：无论是新创建还是修改磁盘组的过程，我们都建议把磁盘组的参数 compatible.asm 和 compatible.rdbms 设置成12.1。

Ex: CREATE DISKGROUP … ATTRIBUTE 'compatible.rdbms' = '12.1', 'compatible.asm' = '12.1';

管理数据库可以通过下面命令，从 NFS/CFS 迁移到 ASM 磁盘组：

$ <GI_HOME>/bin/dbca -silent -createDatabase -sid -MGMTDB -createAsContainerDatabase true -templateName MGMTSeed_Database.dbc -gdbName _mgmtdb -storageType FS -datafileDestination <NFS_Location> -datafileJarLocation $GI_HOME/assistants/dbca/templates -characterset AL32UTF8 -autoGeneratePasswords -skipUserTemplateCheck

b. 使用 DBCA 再创建一个 PDB：

使用 grid 用户执行下面 DBCA 命令：

注意：命令中的 CLUSTER_NAME 中不可以有中横线(“-“)，需要使用下划线替代(“_”)。

$ <GI_HOME>/bin/dbca -silent -createPluggableDatabase -sourceDB -MGMTDB -pdbName <CLUSTER_NAME> -createPDBFrom RMANBACKUP -PDBBackUpfile <GI_HOME>/assistants/dbca/templates/mgmtseed_pdb.dfb -PDBMetadataFile <GI_HOME>/assistants/dbca/templates/mgmtseed_pdb.xml -createAsClone true 
Creating Pluggable Database
4% complete
12% complete
21% complete
38% complete
55% complete
85% complete
Completing Pluggable Database Creation
100% complete
Look at the log file "<GI_HOME>/cfgtoollogs/dbca/_mgmtdb/<pdbName>/_mgmtdb2.log" for further details.

 

4. 确保管理数据库创建成功

使用grid 用户，检查管理数据库运行在哪个节点：

$ <GI_HOME>/bin/srvctl status MGMTDB
Database is enabled
Instance -MGMTDB is running on node <NODE_NAME>

On <NODE_NAME>:

$ <GI_HOME>/bin/mgmtca

 

5. 启用并启动 ora.crf 资源

使用root 用户在每一个节点执行：

# <GI_HOME>/bin/crsctl modify res ora.crf -attr ENABLED=1 -init
# <GI_HOME>/bin/crsctl start res ora.crf -init

  

参考

NOTE:2168379.1  - Mgmtca MGTCA-1101 : Failure in Oracle Grid Infrastructure Management Repository Operation
NOTE:2246123.1  - 12.2: How to Create GI Management Repository
BUG:19530197  - MGTCA-1101: FAILURE IN ORACLE GRID INFRASTRUCTURE MANAGEMENT REPOSITORY OPERATI

12.1.0.2 GI GIMR/MGMTDB SYSMGMTDATA Tablespace Increasing Rapidly (文档 ID 2177879.1)

In this Document

Symptoms

Cause

Solution

References

---

APPLIES TO:

Oracle Database - Enterprise Edition - Version 12.1.0.2 and later
Information in this document applies to any platform.

SYMPTOMS

12.1.0.2 GIMR database (-MGMTDB) SYSMGMTDATA tablespace keeps increasing with the same retention size:

$ oclumon manage -get reppath

CHM Repository Path = +CRS/_MGMTDB/FD9B43BF6A646F8CE043B6A9E80A2815/DATAFILE/sysmgmtdata. 269.917710623

Tablespace Name File Name Size (M) Used (M) Free (M) % Used
-------------------- -------------------------------------------------------------------------------- ---------- ---------- ---------- ----------
SYSMGMTDATA +CRS/_MGMTDB/FD9B43BF6A646F8CE043B6A9E80A2815/DATAFILE/sysmgmtdata. 269.917710623  16894  16032  521 95

 

Expired CHM data are not purged: 

SQL> select trunc(BEGINTIME), count(*) from chm.CHMOS_PROCESS_INT_TBL group by trunc(BEGINTIME) order by trunc(BEGINTIME);

TRUNC(BEG COUNT(*)
--------- ----------
25-JUL-16 1824587
26-JUL-16 3019039
27-JUL-16 3236612
28-JUL-16 3559627
29-JUL-16 3687053
....
09-AUG-16 3626059
10-AUG-16 3638210
11-AUG-16 3665800
12-AUG-16 3565969
13-AUG-16 3650727
14-AUG-16 3638103
15-AUG-16 3622208
16-AUG-16 3680251
17-AUG-16 2321889

 

CAUSE

Issue was investigated in:

Bug 23563549  - -MGMTDB SYSMGMTDATA IS FILLING UP WHICH CAUSES GRID1 DISK TO REACH 100%.

Bug 24331121  - OCR_VOTE DISK GROUP IS 99% FULL, HIGH UNDO USED BY MGMTDB

 

The cause is due to autoextend for the tablespace being set to on.

 

SOLUTION

Solution 1:

Recreate GIMR which will drop the existing database and set autoextend back to off.

note 1589394.1  - How to Move/Recreate GI Management Repository to Different Shared Storage (Diskgroup, CFS or NFS etc)

 

Or

Solution 2: 

Manually perform the following steps to shrink:

 

1. Stop and disable ora.crf resource.

On each node, as root user:

# <GI_HOME>/bin/crsctl stop res ora.crf -init
# <GI_HOME>/bin/crsctl modify res ora.crf -attr ENABLED=0 -init

2. Identify huge objects from dba_segments and truncate them:

Connect to MGMTDB:

export ORACLE_HOME=<GRID_HOME>
export ORACLE_SID=-MGMTDB
$ORACLE_HOME/bin/sqlplus / as sysdba
SQL> select pdb_name from dba_pdbs where pdb_name!='PDB$SEED';


PDB_NAME
--------------------------------------------------------------------------------------------------------------------------------
CEHAOVMSP1CLU68 ====>>>> this is your PDB name, use it in next command

SQL> alter session set container=CEHAOVMSP1CLU68;

Session altered.

SQL> col obj format a50
SQL> select owner||'.'||SEGMENT_NAME obj, BYTES from dba_segments where owner='CHM' order by 2 asc

Likely you are going to see the following as the biggest table:

CHM.CHMOS_PROCESS_INT_TBL
CHM.CHMOS_DEVICE_INT_TBL

Truncate the table, i.e.:

SQL> truncate table CHM.CHMOS_PROCESS_INT_TBL;
SQL> truncate table CHM.CHMOS_DEVICE_INT_TBL;
..

3. Shrink the tablespace:

https://docs.oracle.com/database/121/ADMIN/schema.htm#ADMIN11600

4. Enable and start ora.crf resource. 

On each node, as root user:

# <GI_HOME>/bin/crsctl modify res ora.crf -attr ENABLED=1 -init
# <GI_HOME>/bin/crsctl start res ora.crf -init

 

REFERENCES

NOTE:1589394.1  - How to Move/Recreate GI Management Repository to Different Shared Storage (Diskgroup, CFS or NFS etc)
BUG:23563549  - -MGMTDB SYSMGMTDATA IS FILLING UP WHICH CAUSES GRID1 DISK TO REACH 100%.

MDBUtil: GI Management Repository configuration tool (文档 ID 2065175.1)

In this Document

Abstract

History

Details

Grid Infrastructure Management Repository (GIMR) database MGMTDB

MDBUtil: GI Management Repository configuration tool

MDBUtil usage

Create MGMTDB/CHM and reconfigure related functions

Migrate MGMTDB database files to another disk group

Check the CHM & MGMTDB status

MDBUtil: troubleshooting

References

---

APPLIES TO:

Oracle Database - Enterprise Edition - Version 12.1.0.1 to 12.2.0.1 [Release 12.1 to 12.2]
Oracle Database Cloud Schema Service - Version N/A and later
Oracle Database Exadata Express Cloud Service - Version N/A and later
Oracle Database Exadata Cloud Machine - Version N/A and later
Oracle Cloud Infrastructure - Database Service - Version N/A and later
Linux x86-64

ABSTRACT

This document explains how to use " mdbutil ", a tool used to create the Grid Infrastructure Management Repository (GIMR) database MGMTDB or move its database files to a different disk group in one simple command

HISTORY

Author: Ruggero Citton (ruggero.citton@oracle.com) - RACPack Team
Create Date 2014
Update Date June-2018
Latest " mdbutil " Version: 20180614 $Revision: 1.95 $ 

DETAILS

Grid Infrastructure Management Repository (GIMR) database MGMTDB

The Grid Infrastructure Management Repository enables such features as Cluster Health Monitor, Oracle Database QoS Management, and Rapid Home Provisioning, and provides a historical metric repository that simplifies viewing of past performance and diagnosis of issues.
During the installation of Oracle Grid Infrastructure 12.1.0.1 you've had the option to install the Grid Infrastructure Management Repository (GIMR) database MGMTDB. Starting with Oracle Grid Infrastructure 12.1.0.2 the GIMR database has become mandatory and is automatically installed.

MDBUtil: GI Management Repository configuration tool

If you need to create or re-create the Grid Infrastructure Management Repository (GIMR) database MGMTDB after the installation, specific knowledge and many manual steps are required. (see  Note 1589394.1  for details). You also may desire to move the MGMTDB database files to a different disk group from your Clusterware files.  MDBUtil  has been created to provide this functionality in one simple command.

Note:
 This script is provided for educational purposes only. 
 It is NOT supported by Oracle World Wide Technical Support. 
 The script has been tested and appears to work as intended. 
 You should always run new scripts on a test instance initially.

MDBUtil usage

# ./mdbutil.pl -h
Usage:
     Create/Enable MGMTDB & CHM
       mdbutil.pl --addmdb --target=<MGMTDB destination>
     Move MGMTDB to another location
       mdbutil.pl --mvmgmtdb --target=<new MGMTDB destination>   
     Check MGMTDB status
       mdbutil.pl --status
   
     mdbutil.pl OPTIONS
       --addmdb            Create MGMTDB/CHM and reconfigure related functions
       --mvmgmtdb          Migrate MGMTDB to another location   
       --target='+DATA'    MGMTDB Disk Group location
       --status            Check the CHM & MGMTDB status
       --help              Display this help and exit
       --debug             Verbose commands output/trace
   
     Example:
       Create/Enable MGMTDB:
         mdbutil.pl --addmdb --target=+DATA
       Move MGMTDB to another location:
         mdbutil.pl --mvmgmtdb --target=+REDO             
       Check CHM:
         mdbutil.pl --status

Create MGMTDB/CHM and reconfigure related functions

You can create MGMTDB and configure the related functions such mgmtlsnr, chm issuing (as 'grid' user). PLEASE NOTE: You will also need root user credentials if the CHM resource (ora.crf) has not been previously created.

mdbutil --addmdb --target=<MGMTDB destination>

example:

[grid@rwsdb1 tmp]$ ./mdbutil.pl --addmdb --target=+DATA       
2015-10-12 09:11:17:  I  Starting To Configure MGMTDB at +DATA...
2015-10-12 09:11:19:  I  Container database creation in progress...
2015-10-12 09:17:35:  I  Plugable database creation in progress...
2015-10-12 09:19:00:  I  Executing "/tmp/mdbutil.pl --addchm" on rwsdb1 as root to configure CHM.
root@rwsdb1's password: 
2015-10-12 09:20:17:  I  Executing "/tmp/mdbutil.pl --addchm" on rwsdb2 as root to configure CHM.
root@rwsdb2's password: 
2015-10-12 09:20:22:  I  MGMTDB & CHM configuration done!

Migrate MGMTDB database files to another disk group

You can move MGMTDB database files to another ASM disk group issuing the following command as GI software owner (ex: ‘grid') user:

mdbutil.pl --mvmgmtdb --target=<new MGMTDB destination>  

example:
[grid@rwsdb1 tmp]$ /tmp/mdbutil.pl --mvmgmtdb --target=+REDO
Moving MGMTDB, it will be stopped, are you sure (Y/N)? y
2015-10-12 09:24:53:  I  Checking for the required paths under +REDO
2015-10-12 09:24:54:  I  Creating new path +REDO/_MGMTDB/PARAMETERFILE
2015-10-12 09:24:56:  I  Creating new path +REDO/_MGMTDB/CONTROLFILE
2015-10-12 09:24:59:  I  Creating new path +REDO/_MGMTDB/ONLINELOG
2015-10-12 09:25:01:  I  Creating new path +REDO/_MGMTDB/DATAFILES
2015-10-12 09:25:04:  I  Creating new path +REDO/_MGMTDB/TEMPFILE
2015-10-12 09:25:06:  I  Creating new path +REDO/_MGMTDB/DATAFILES/rwsdb_c
2015-10-12 09:25:08:  I  Creating new path +REDO/_MGMTDB/TEMPFILE/rwsdb_c
2015-10-12 09:25:08:  I  Getting MGMTDB Database files location
2015-10-12 09:25:09:  I  Getting MGMTDB Temp files location
2015-10-12 09:25:09:  I  Getting MGMTDB PDB rwsdb_c files location
2015-10-12 09:25:09:  I  Getting MGMTDB PDB rwsdb_c Temp files location
2015-10-12 09:25:10:  I  Creating temporary PFILE
2015-10-12 09:25:10:  I  Creating target SPFILE
2015-10-12 09:25:16:  I  Stopping mgmtdb
2015-10-12 09:25:36:  I  Copying MGMTDB DBFiles to +REDO
2015-10-12 09:25:52:  I  Copying MGMTDB rwsdb_c PDB DBFiles to +REDO
2015-10-12 09:26:33:  I  Creating the CTRL File
2015-10-12 09:26:59:  I  The CTRL File has been created and MGMTDB is now running from +REDO
2015-10-12 09:26:59:  I  Setting MGMTDB SPFile location
2015-10-12 09:27:00:  I  Modifing the init parameter
2015-10-12 09:27:00:  I  Removing old MGMTDB
2015-10-12 09:27:02:  I  Restarting MGMTDB using target SPFile
2015-10-12 09:27:47:  I  MGMTDB Successfully moved to +REDO!

Check the CHM & MGMTDB status

If for any reason, MDBUtil is failing, you can get diagnostics information running mdbutil again using the “-debug” option, example:

mdbutil.pl --status

example:
[grid@rwsdb1 tmp]$ ./mdbutil.pl --status
2015-10-12 09:30:37:  I  Checking CHM status...
2015-10-12 09:30:38:  I  Listener MGMTLSNR is configured and running on rwsdb1
2015-10-12 09:30:39:  I  Database MGMTDB is configured and running on rwsdb1
2015-10-12 09:30:39:  I  Cluster Health Monitor (CHM) is configured and running
--------------------------------------------------------------------------------
CHM Repository Path = +RECO/_MGMTDB/DATAFILES/rwsdb_c/sysmgmtdata.20151012092509.dbf
MGMTDB space used on DG +RECO = 504 Mb
--------------------------------------------------------------------------------

MDBUtil: troubleshooting

If for any reason, MDBUtil is failing, you could get diagnostics information running mdbutil again using " -debug " option, example:

[grid@rwsdb1 tmp]$ ./mdbutil.pl --addmdb --target=+REDO
2015-10-12 09:09:31:  I  Starting To Configure MGMTDB at +REDO...
2015-10-12 09:09:52:  I  Container database creation in progress...
2015-10-12 09:09:59:  E  Cannot create CDB MGMTDB on rwsdb1, exiting...


[grid@rwsdb1 tmp]$ ./mdbutil.pl --addmdb --target=+REDO  -debug
2015-10-12 09:10:18:  D  Executing: /u01/app/12.1.0.2/grid/bin/srvctl status diskgroup -g REDO
2015-10-12 09:10:19:  D  Exit code: 0
2015-10-12 09:10:19:  D  Output of last command execution: 
Disk Group REDO is running on rwsdb1,rwsdb1
2015-10-12 09:10:19:  I  Starting To Configure MGMTDB at +REDO...
2015-10-12 09:10:19:  D  Executing: /u01/app/12.1.0.2/grid/bin/srvctl status mgmtlsnr
2015-10-12 09:10:19:  D  Exit code: 0
2015-10-12 09:10:19:  D  Output of last command execution: 
Listener MGMTLSNR is enabled
2015-10-12 09:10:19:  D  Executing: /u01/app/12.1.0.2/grid/bin/srvctl status mgmtdb
2015-10-12 09:10:20:  D  Exit code: 1
2015-10-12 09:10:20:  D  Output of last command execution: 
PRCD-1120 : The resource for database _mgmtdb could not be found.
2015-10-12 09:10:20:  D  Executing: /u01/app/12.1.0.2/grid/bin/srvctl status mgmtdb
2015-10-12 09:10:20:  D  Exit code: 1
2015-10-12 09:10:20:  D  Output of last command execution: 
PRCD-1120 : The resource for database _mgmtdb could not be found.
2015-10-12 09:10:20:  D  Executing: /u01/app/12.1.0.2/grid/bin/srvctl stop mgmtlsnr
2015-10-12 09:10:21:  D  Exit code: 2
2015-10-12 09:10:21:  D  Output of last command execution: 
PRCC-1016 : MGMTLSNR was already stopped
2015-10-12 09:10:21:  D  Executing: /u01/app/12.1.0.2/grid/bin/srvctl enable oc4j
2015-10-12 09:10:21:  D  Exit code: 2
2015-10-12 09:10:21:  D  Output of last command execution: 
PRKO-2116 : OC4J is already enabled
2015-10-12 09:10:21:  D  Executing: /u01/app/12.1.0.2/grid/bin/srvctl start oc4j
2015-10-12 09:10:21:  D  Exit code: 2
2015-10-12 09:10:21:  D  Output of last command execution: 
PRCC-1014 : oc4j was already running
2015-10-12 09:10:21:  D  Executing: /u01/app/12.1.0.2/grid/bin/crsctl query crs activeversion
2015-10-12 09:10:21:  D  Exit code: 0
2015-10-12 09:10:21:  D  Output of last command execution: 
Oracle Clusterware active version on the cluster is [12.1.0.2.0]
2015-10-12 09:10:21:  I  Container database creation in progress...
2015-10-12 09:10:21:  D  Executing: /u01/app/12.1.0.2/grid/bin/dbca  -silent -createDatabase -createAsContainerDatabase true -templateName MGMTSeed_Database.dbc -sid -MGMTDB -gdbName _mgmtdb -storageType ASM -diskGroupName REDO -datafileJarLocation /u01/app/12.1.0.2/grid/assistants/dbca/templates -characterset AL32UTF8 -autoGeneratePasswords -skipUserTemplateCheck
2015-10-12 09:10:26:  D  Exit code: 1
2015-10-12 09:10:26:  D  Output of last command execution: 
Look at the log file "/u01/app/grid/cfgtoollogs/dbca/_mgmtdb1.log" for further details.
2015-10-12 09:10:26:  E  Cannot create CDB MGMTDB on rwsdb1, exiting...

--> 
[grid@rwsdb1 tmp]$ cat /u01/app/grid/cfgtoollogs/dbca/_mgmtdb1.log

+REDO does not have enough space. Required space is 1566 MB , available space is 1305 MB.

 

REFERENCES

NOTE:1589394.1  - How to Move/Recreate GI Management Repository (GIMR / MGMTDB) to Different Shared Storage (Diskgroup, CFS or NFS etc)
NOTE:1568402.1  - FAQ: 12c Grid Infrastructure Management Repository (GIMR)
NOTE:2062234.1  - How to Relocate Cluster Health Monitor (CHM) Repository and Increase Retention Time