Oracle Logminer的测试使用

Oracle Logminer的测试使用

前言

版本：11.2.0.4.0

几个月前生产帮忙业务追溯数据异常时候用了下，一直想出个总结拖到现在。

实验

对 logminer强需求的要求开启附加日志，不开启会有一个问题下边实验会讲。

alter database add supplemental log data;

1.安装logminer工具（默认是安装的）

1.$ORACLE_HOME/rdbms/admin/dbmslm.sql
2.$ORACLE_HOME/rdbms/admin/dbmslmd.sql

 这两个脚本必须均以SYS用户身份运行。其中第一个脚本用来创建DBMS_LOGMNR包，该包用来分析日志文件。第二个脚本用来创建DBMS_LOGMNR_D包，该包用来创建数据字典文件。

2..使用logminer功能一个重要的前提就是需要设置参数utl_file_dir，这个参数是需要重启DB才能生效的。

14:31:32 SYS@zkm(1)> alter system set utl_file_dir='/home/oracle/logminer/' scope=spfile;

System altered.

Elapsed: 00:00:00.00
14:33:33 SYS@zkm(1)> shutdown immediate;
Database closed.
Database dismounted.
ORACLE instance shut down.
14:33:43 SYS@zkm(1)> startup;
ORACLE instance started.

Total System Global Area  784998400 bytes
Fixed Size                  2257352 bytes
Variable Size             478154296 bytes
Database Buffers          301989888 bytes
Redo Buffers                2596864 bytes
Database mounted.
Database opened.

3.创建用于存放logminer表的表空间并指定

14:37:48 SYS@zkm(1)> create tablespace log_mnr_tbs datafile '/u01/app/oracle/oradata/zkm/log_mnr_tbs01.dbf' size 1g autoextend off;

Tablespace created.

Elapsed: 00:00:21.21

14:38:43 SYS@zkm(1)> EXECUTE DBMS_LOGMNR_D.SET_TABLESPACE('log_mnr_tbs');    

PL/SQL procedure successfully completed.

Elapsed: 00:00:03.61

默认logminer的表是放在system表空间的，从操作规范上需要专门为此创建一个新表空间。

4.提取数据字典信息（不使用数据字典会使得挖出来的语句中关于数据字典中的部分（如表名、列名等）和数值都将是16进制的形式，我们是无法直接理解的）

14:43:10 SYS@zkm(1)> EXECUTE dbms_logmnr_d.build(dictionary_filename => 'logmnrtest.ora',dictionary_location => '/home/oracle/logminer/');

PL/SQL procedure successfully completed.

Elapsed: 00:00:03.43

 这一步执行后会多出很多的表和其它的一些对象：

14:43:30 SYS@zkm(36)> col segment_name for a100
14:43:37 SYS@zkm(36)> set line 500
14:43:39 SYS@zkm(36)> select owner,segment_name from dba_segments where tablespace_name='LOG_MNR_TBS';

OWNER                     SEGMENT_NAME
------------------------- ----------------------------------------------------------------------------------------------------
SYSTEM                    LOGMNR_SPILL$
SYSTEM                    LOGMNR_SPILL$_PK
SYSTEM                    LOGMNR_SUBCOLTYPE$
SYSTEM                    LOGMNR_TAB$
SYSTEM                    LOGMNR_TABCOMPART$
SYSTEM                    LOGMNR_TABPART$
SYSTEM                    LOGMNR_TABSUBPART$
SYSTEM                    LOGMNR_TS$
SYSTEM                    LOGMNR_TYPE$
SYSTEM                    LOGMNR_UID$
SYSTEM                    LOGMNR_UID$_PK
SYSTEM                    LOGMNR_USER$
SYSTEM                    LOGMNRC_CONCOL_GG
SYSTEM                    LOGMNRC_CONCOL_GG_PK
SYSTEM                    LOGMNRC_CON_GG
SYSTEM                    LOGMNRC_CON_GG_PK
SYSTEM                    LOGMNRC_DBNAME_UID_MAP
SYSTEM                    LOGMNRC_DBNAME_UID_MAP_PK
SYSTEM                    LOGMNRC_GSBA
SYSTEM                    LOGMNRC_GSBA_PK
SYSTEM                    LOGMNRC_GSII
SYSTEM                    LOGMNRC_GSII_PK
SYSTEM                    LOGMNRC_GTCS
SYSTEM                    LOGMNRC_GTCS_PK
SYSTEM                    LOGMNRC_GTLO
SYSTEM                    LOGMNRC_GTLO_PK
SYSTEM                    LOGMNRC_I1CONGG
SYSTEM                    LOGMNRC_I1INDGG
SYSTEM                    LOGMNRC_I2CONGG
SYSTEM                    LOGMNRC_I2GTCS
SYSTEM                    LOGMNRC_I2GTLO
SYSTEM                    LOGMNRC_I2INDGG
SYSTEM                    LOGMNRC_I2SEQGG
SYSTEM                    LOGMNRC_I3GTLO
SYSTEM                    LOGMNRC_INDCOL_GG
SYSTEM                    LOGMNRC_INDCOL_GG_PK
SYSTEM                    LOGMNRC_IND_GG
SYSTEM                    LOGMNRC_IND_GG_PK
SYSTEM                    LOGMNRC_SEQ_GG
SYSTEM                    LOGMNRC_SEQ_GG_PK
SYSTEM                    LOGMNRP_CTAS_PART_MAP
SYSTEM                    LOGMNRP_CTAS_PART_MAP_I
SYSTEM                    LOGMNRP_CTAS_PART_MAP_PK
SYSTEM                    LOGMNR_AGE_SPILL$
SYSTEM                    LOGMNR_AGE_SPILL$_PK
SYSTEM                    LOGMNR_ATTRCOL$
SYSTEM                    LOGMNR_ATTRIBUTE$
SYSTEM                    LOGMNR_CCOL$
SYSTEM                    LOGMNR_CDEF$
SYSTEM                    LOGMNR_COL$
SYSTEM                    LOGMNR_COLTYPE$
SYSTEM                    LOGMNR_CON$
SYSTEM                    LOGMNR_DICTIONARY$
SYSTEM                    LOGMNR_DICTSTATE$
SYSTEM                    LOGMNR_ENC$
SYSTEM                    LOGMNR_ERROR$
SYSTEM                    LOGMNR_FILTER$
SYSTEM                    LOGMNR_GLOBAL$
SYSTEM                    LOGMNR_I1ATTRCOL$
SYSTEM                    LOGMNR_I1ATTRIBUTE$
SYSTEM                    LOGMNR_I1CCOL$
SYSTEM                    LOGMNR_I1CDEF$
SYSTEM                    LOGMNR_I1COL$
SYSTEM                    LOGMNR_I1COLTYPE$
SYSTEM                    LOGMNR_I1CON$
SYSTEM                    LOGMNR_I1DICTIONARY$
SYSTEM                    LOGMNR_I1ENC$
SYSTEM                    LOGMNR_I1ICOL$
SYSTEM                    LOGMNR_I1IND$
SYSTEM                    LOGMNR_I1INDCOMPART$
SYSTEM                    LOGMNR_I1INDPART$
SYSTEM                    LOGMNR_I1INDSUBPART$
SYSTEM                    LOGMNR_I1KOPM$
SYSTEM                    LOGMNR_I1LOB$
SYSTEM                    LOGMNR_I1LOBFRAG$
SYSTEM                    LOGMNR_I1LOGMNR_BUILDLOG
SYSTEM                    LOGMNR_I1NTAB$
SYSTEM                    LOGMNR_I1OBJ$
SYSTEM                    LOGMNR_I1OPQTYPE$
SYSTEM                    LOGMNR_I1PARTOBJ$
SYSTEM                    LOGMNR_I1PROPS$
SYSTEM                    LOGMNR_I1REFCON$
SYSTEM                    LOGMNR_I1SEED$
SYSTEM                    LOGMNR_I1SUBCOLTYPE$
SYSTEM                    LOGMNR_I1TAB$
SYSTEM                    LOGMNR_I1TABCOMPART$
SYSTEM                    LOGMNR_I1TABPART$
SYSTEM                    LOGMNR_I1TABSUBPART$
SYSTEM                    LOGMNR_I1TS$
SYSTEM                    LOGMNR_I1TYPE$
SYSTEM                    LOGMNR_I1USER$
SYSTEM                    LOGMNR_I2CDEF$
SYSTEM                    LOGMNR_I2COL$
SYSTEM                    LOGMNR_I2IND$
SYSTEM                    LOGMNR_I2INDPART$
SYSTEM                    LOGMNR_I2NTAB$
SYSTEM                    LOGMNR_I2OBJ$
SYSTEM                    LOGMNR_I2SEED$
SYSTEM                    LOGMNR_I2TAB$
SYSTEM                    LOGMNR_I2TABCOMPART$
SYSTEM                    LOGMNR_I2TABPART$
SYSTEM                    LOGMNR_I2TABSUBPART$
SYSTEM                    LOGMNR_I3COL$
SYSTEM                    LOGMNR_ICOL$
SYSTEM                    LOGMNR_IND$
SYSTEM                    LOGMNR_INDCOMPART$
SYSTEM                    LOGMNR_INDPART$
SYSTEM                    LOGMNR_INDSUBPART$
SYSTEM                    LOGMNR_INTEGRATED_SPILL$
SYSTEM                    LOGMNR_INTEG_SPILL$_PK
SYSTEM                    LOGMNR_KOPM$
SYSTEM                    LOGMNR_LOB$
SYSTEM                    LOGMNR_LOBFRAG$
SYSTEM                    LOGMNR_LOG$
SYSTEM                    LOGMNR_LOG$_FIRST_CHANGE#
SYSTEM                    LOGMNR_LOG$_FLAGS
SYSTEM                    LOGMNR_LOG$_PK
SYSTEM                    LOGMNR_LOG$_RECID
SYSTEM                    LOGMNR_LOGMNR_BUILDLOG
SYSTEM                    LOGMNR_NTAB$
SYSTEM                    LOGMNR_OBJ$
SYSTEM                    LOGMNR_OPQTYPE$
SYSTEM                    LOGMNR_PARTOBJ$
SYSTEM                    LOGMNR_PROCESSED_LOG$
SYSTEM                    LOGMNR_PROCESSED_LOG$_PK
SYSTEM                    LOGMNR_PROPS$
SYSTEM                    LOGMNR_REFCON$
SYSTEM                    LOGMNR_RESTART_CKPT$
SYSTEM                    LOGMNR_RESTART_CKPT$_PK
SYSTEM                    LOGMNR_RESTART_CKPT_TXINFO$
SYSTEM                    LOGMNR_RESTART_CKPT_TXINFO$_PK
SYSTEM                    LOGMNR_SEED$
SYSTEM                    LOGMNR_SESSION_ACTION$_PK
SYSTEM                    LOGMNR_SESSION_ACTIONS$
SYSTEM                    LOGMNR_SESSION_EVOLVE$
SYSTEM                    LOGMNR_SESSION_EVOLVE$_PK
SYSTEM                    SYS_IL0000001161C00010$$
SYSTEM                    SYS_IL0000001165C00008$$
SYSTEM                    SYS_IL0000001169C00011$$
SYSTEM                    SYS_IL0000001174C00009$$
SYSTEM                    SYS_IL0000001174C00012$$
SYSTEM                    SYS_IL0000001180C00009$$
SYSTEM                    SYS_LOB0000001161C00010$$
SYSTEM                    SYS_LOB0000001165C00008$$
SYSTEM                    SYS_LOB0000001169C00011$$
SYSTEM                    SYS_LOB0000001174C00009$$
SYSTEM                    SYS_LOB0000001174C00012$$
SYSTEM                    SYS_LOB0000001180C00009$$

148 rows selected.

Elapsed: 00:00:00.04
14:44:17 SYS@zkm(36)> select owner,table_name from dba_tables where tablespace_name='LOG_MNR_TBS';

OWNER                     TABLE_NAME
------------------------- ------------------------------
SYSTEM                    LOGMNR_SESSION_EVOLVE$
SYSTEM                    LOGMNR_GLOBAL$
SYSTEM                    LOGMNR_UID$
SYSTEM                    LOGMNRC_DBNAME_UID_MAP
SYSTEM                    LOGMNR_LOG$
SYSTEM                    LOGMNR_PROCESSED_LOG$
SYSTEM                    LOGMNR_SPILL$
SYSTEM                    LOGMNR_AGE_SPILL$
SYSTEM                    LOGMNR_RESTART_CKPT_TXINFO$
SYSTEM                    LOGMNR_ERROR$
SYSTEM                    LOGMNR_RESTART_CKPT$
SYSTEM                    LOGMNR_INTEGRATED_SPILL$
SYSTEM                    LOGMNR_FILTER$
SYSTEM                    LOGMNR_SESSION_ACTIONS$

14 rows selected.

Elapsed: 00:00:00.05

5.添加重做日志或者归档日志并开始挖掘

5.1手工添加日志

出现故障时，业务/开发会给出一个时间段或者时间点，通过以下SQL查找出包含这个时间点的归档日志。

SELECT THREAD#,SEQUENCE#,FIRST_TIME,NEXT_TIME,NAME FROM V$ARCHIVED_LOG WHERE TO_DATE('2020-12-30 09:17:04', 'YYYY-MM-DD HH24:MI:SS') BETWEEN FIRST_TIME AND NEXT_TIME;

如果此时归档日志已经被备份走不在归档路径里边，通过如下语句在RMAN中进行恢复。

RMAN> restore archivelog sequence 37268;
...
RMAN> restore archivelog sequence 32878;                                     
...
RMAN> restore archivelog sequence 32878 thread 2;
...

然后通过如下语句添加日志。

exec DBMS_LOGMNR.ADD_LOGFILE(LogFileName=>'+FRA_VG/xxxxxx/archivelog/2020_12_30/thread_1_seq_37268.427.1060525501', Options=>dbms_logmnr.new);
exec DBMS_LOGMNR.ADD_LOGFILE(LogFileName=>'+FRA_VG/xxxxxx/archivelog/2020_12_30/thread_2_seq_32878.410.1060526043', Options=>dbms_logmnr.addfile);

开始执行挖掘工作。

execute dbms_logmnr.start_logmnr(dictfilename=>'/ora/logminer/logmnr20201230.ora');

5.2开始挖掘并自动添加（我这次实验用的这种）

在Oracle数据库10g或以后的版本中，当针对生成重做日志文件的同一数据库使用LogMiner时，LogMiner会根据请求的时间或SCN范围扫描控制文件并确定所需的重做日志文件。

您不再需要将时间范围映射到一组明确的重做日志文件。为了进行此扫描，您需要使用continuous_mine选项并指定startscn或starttime。

BEGIN
    DBMS_LOGMNR.START_LOGMNR( STARTTIME => '2021-02-25 15:19:01',
        ENDTIME => '2021-02-25 15:21:00',
        OPTIONS => DBMS_LOGMNR.CONTINUOUS_MINE,
        dictfilename=>'/home/oracle/logminer/logmnrtest.ora');
END;
/

15:13:22 SYS@zkm(1)> BEGIN
15:13:23   2      DBMS_LOGMNR.START_LOGMNR( STARTTIME => '2021-02-25 14:31:32',
15:13:23   3          ENDTIME => '2021-02-25 14:44:17',
15:13:23   4            OPTIONS => DBMS_LOGMNR.CONTINUOUS_MINE,
15:13:23   5          dictfilename=>'/home/oracle/logminer/logmnrtest.ora');
15:13:23   6  END;
15:13:23   7  /

PL/SQL procedure successfully completed.

Elapsed: 00:00:01.29

6.分析数据

v$logmnr_contents 视图保存了从归档日志中挖掘出来的数据，包括 REDO SQL 和 UNDO SQL。
v$logmnr_contents 视图只对当前会话有效，如果数据量比较大的话建议重建成一张普通表，建立索引之后再查询，这里我们创建新的表 LOG_MNR_TAB 来保存挖掘出来的数据。

创建一张新表存放v$logmnr_contents的内容，就可以使用plsqldev工具查询比较美观。

15:13:24 SYS@zkm(1)> create table log_mnr_tab as select * from v$logmnr_contents;

Table created.

Elapsed: 00:00:00.54
15:13:49 SYS@zkm(1)> exec dbms_logmnr.end_logmnr;

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.03

可以看出列username，os_username等信息都是unknown。

这是因为没有开启附加日志，现在开启附加日志并且创建新表并插入记录在观察。

15:16:28 SYS@zkm(1)> drop table log_mnr_tab purge; 

Table dropped.

Elapsed: 00:00:00.43


15:19:01 SYS@zkm(1)> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;

Database altered.

Elapsed: 00:00:00.04
15:19:18 SYS@zkm(36)> conn zkm/oracle
Connected.
15:19:20 ZKM@zkm(36)> create table zkm (id int,name varchar2(20));

Table created.

Elapsed: 00:00:00.13
15:19:33 ZKM@zkm(36)> insert into zkm values(1,'a');

1 row created.

Elapsed: 00:00:00.07
15:19:46 ZKM@zkm(36)> commit;

Commit complete.

Elapsed: 00:00:00.00


注意：如果不重新提取数据字典，新的表也会是16进制的信息，这里演示
15:21:00 SYS@zkm(1)> BEGIN
15:21:08   2      DBMS_LOGMNR.START_LOGMNR( STARTTIME => '2021-02-25 15:19:01',
15:21:08   3          ENDTIME => '2021-02-25 15:21:00',
15:21:08   4            OPTIONS => DBMS_LOGMNR.CONTINUOUS_MINE,
15:21:08   5          dictfilename=>'/home/oracle/logminer/logmnrtest.ora');
15:21:08   6  END;
15:21:08   7  /

PL/SQL procedure successfully completed.

Elapsed: 00:00:01.09
15:21:10 SYS@zkm(1)> create table log_mnr_tab as select * from v$logmnr_contents;

Table created.

Elapsed: 00:00:00.31

plsqldev查看如下：

16:18:33 SYS@zkm(1)> select object_id from dba_objects where owner='ZKM' and object_name='ZKM';                                                   

 OBJECT_ID
----------
    100829

Elapsed: 00:00:00.00

 附加日志开了后，相关信息不再是unknown显示了。

16:21:07 SYS@zkm(1)> exec dbms_logmnr.end_logmnr;

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.03

至于16进制的问题，重新提取数据字典就行了。

16:21:07 SYS@zkm(1)> exec dbms_logmnr.end_logmnr;

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.03
16:21:31 SYS@zkm(1)> EXECUTE dbms_logmnr_d.build(dictionary_filename => 'logmnrtest.ora',dictionary_location => '/home/oracle/logminer/');

PL/SQL procedure successfully completed.

Elapsed: 00:00:02.63
16:21:53 SYS@zkm(1)> BEGIN
16:22:07   2      DBMS_LOGMNR.START_LOGMNR( STARTTIME => '2021-02-25 15:19:01',
16:22:07   3          ENDTIME => '2021-02-25 15:21:00',
16:22:07   4            OPTIONS => DBMS_LOGMNR.CONTINUOUS_MINE,
16:22:07   5          dictfilename=>'/home/oracle/logminer/logmnrtest.ora');
16:22:07   6  END;
16:22:07   7  /

PL/SQL procedure successfully completed.

Elapsed: 00:00:01.16
16:22:09 SYS@zkm(1)> drop table log_mnr_tab purge; 

Table dropped.

Elapsed: 00:00:00.13
16:22:28 SYS@zkm(1)> create table log_mnr_tab as select * from v$logmnr_contents;

Table created.

Elapsed: 00:00:00.41
16:22:32 SYS@zkm(1)> exec dbms_logmnr.end_logmnr;

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.01

参考

https://blog.csdn.net/yes_is_ok/article/details/79296614

https://www.cnblogs.com/youngerger/p/8858583.html

http://blog.itpub.net/31560527/viewspace-2662386/

How to Reclaim Space Used by LogMiner Tables (Doc ID 456814.1)

Oracle 10g New Features Of LogMiner (Doc ID 249001.1)