• Oracle Golden Gate 系列十二 GG 数据初始化装载二 基于SCN 的初始化 说明 与 示例


    一.初始化说明

           GG实施过程中,初始化是一个重要的工作,尤其是要初始化的数据较多,并且系统又是7*24的时。

           对于静态初始化,把业务停掉,DB 上的数据就不会有变化,这时候,我们可以用expdp/impdp 或者dblink 等方式把基数据同步过去, 在启动相关的GG 同步进程就可以了。

          但如果要求零停机,DB事务就会不间断进行,可以通过如下两种方法来保证初始化过程中事务的完整性和数据的准确性呢

         1. 利用 Keys + Handlecollisions

         2. 利用 commit SCN/CSN

    1.1 Keys+Handlecollisions 方法

    在我们系列九中的说明和测试使用的就是第一种方法:

    Oracle Golden Gate 系列九 --GG 数据初始化装载 说明 与 示例

    http://blog.csdn.net/tianlesoftware/article/details/6976551

           Handlecollisions参数依赖于表上的Key(Primarykey/Unique key)来对数据进行重复行和缺失行的处理,常在数据初始化过程中保证数据的一致性。但是这种办法在实际的工程实施中是有相当大的限制。

            首先,该初始化办法性能比较糟糕,对于大型数据库来讲,并不合适。更严重的是,它有很大的缺陷性。

    MOS 上有关使用Handlecollisions 方法的一个说明:

    1. When there isprimary key update (PKUpdate), the HANDLECOLLISIONS method may lose data. Thesolution in the case of a primary key update is for Extract to fetch whole rowby making a flashback query. However, this is not efficient and may not bepossible in certain environments.

    2. When a tabledoes not have any type of key, HANDLECOLLISIONS may introduce duplicate rows inthe table. One workaround is to add a primary or unique key to the targettable.

    3. The exacttimestamp when the initial load was started may not be known (and differenttables may have different start times), so it is almost inevitable thatHANDLECOLLISIONS will be used for certain records un-necessarily. This couldmask a real data integrity problem.

    1.2 Commit+SCN/CSN 方法

    可以使用一下几种方法来实现:

    (1)一致性的exp和imp

    (2)一致性的expdp和impdp

    (3) 基于备份的表空间搬移

    (4) Dataguard

    1.2.1 Transportable Tablespace

           TransporttableTablespace 可以使用Expdp/impdp实现,也可以使用RMAN 来实现,这里要注意的就是传输表空间需要将表空间设置为read only,但一般生产库不允许,所以这里可以使用基于备份的RMAN 传输表空间。

      基于备份的表空间搬移的一个最大的优势就是零停机,而且支持异构平台和跨版本(对于不同字节顺序的source-target平台初始化,需要进行convert),但是需要10g以上版本才支持,同样受到表空间搬移的那些限制条件。

    1.2.2 Data Guard

            Dataguard适合同平台同版本的系统环境初始化。

          

     有关Oracle DG 系列的文章,参考我的Blog:

    http://blog.csdn.net/tianlesoftware/article/category/700326

    1.2.3 exp/expdp

            通过exp和expdp的一致性参数(flashback_scn),导出特定SCN点上的一致性版本. FLASHBACK_SCN 参数用于指定导出特定SCN时刻的表数据,如:

    FLASHBACK_SCN=scn_value。

           Scn_value用于标识SCN值.FLASHBACK_SCN和FLASHBACK_TIME不能同时使用。

    使用这种方法初始化存在一个问题,就是构造一致性数据过程中会对undo造成比较大的压力,尤其对大型数据库来讲,可以通过分割的datapump来实现数据的分组同步,分散undo的压力,然后合适时间将分组的datapump合并即可。

           该方法的优点就是可以跨平台和跨版本初始化。

    Oracle 10g Data Pump Expdp/Impdp 详解

    http://blog.csdn.net/tianlesoftware/article/details/4674224

    Oracle expdp/impdp 使用示例

    http://blog.csdn.net/tianlesoftware/article/details/6260138

    ORACLE EXP/IMP 说明

    http://blog.csdn.net/tianlesoftware/article/details/4718366

    在基于SCN 的数据初始化完成之后,我们在Replicat时,从指定SCN 开始就ok了。如:

    start rep1,aftercsn 12345678

    二.示例

    这里只演示基于Expdp/Impdp, 这种方法最灵活。

    我们使用GG的架构是: Extract+Data Pump + Replicat.

    2.1 清除之前的GG环境

    --Target System

    SQL> conn dave/dave;

    Connected.

    SQL> drop table pdba;

    Table dropped.

    GGSCI (gg2) 46> stop rep1

    Sending STOP request to REPLICAT REP1 ...

    Request processed.

    GGSCI (gg2) 47> info all

    Program    Status      Group       Lag           Time Since Chkpt

    MANAGER    RUNNING                                          

    REPLICAT   STOPPED     REP1        00:00:00      00:00:04   

    GGSCI (gg2) 48> delete rep1

    ERROR: Could not delete DB checkpoint forREPLICAT REP1 (Database login required to delete database checkpoint).

    GGSCI (gg2) 49> dblogin useridggate,password ggate

    Successfully logged into database.

    GGSCI (gg2) 51> delete rep1

    Deleted REPLICAT REP1.

    GGSCI (gg2) 52> info all

    Program    Status      Group       Lag      Time Since Chkpt

    MANAGER    RUNNING                                          

    --Source System

    GGSCI (gg1) 55> info all

    Program    Status      Group      Lag           Time Since Chkpt

    MANAGER    RUNNING                                          

    EXTRACT    RUNNING     DPUMP       00:00:00      00:00:09   

    EXTRACT    RUNNING     EXT1        00:00:00      00:00:09   

    GGSCI (gg1) 56> dblogin userid ggate,password ggate

    Successfully logged into database.

    GGSCI (gg1) 57> stop ext1

    Sending STOP request to EXTRACT EXT1 ...

    Request processed.

    GGSCI (gg1) 58> stop dpump

    Sending STOP request to EXTRACT DPUMP ...

    Request processed.

    GGSCI (gg1) 59> delete ext1

    2011-11-17 16:51:48  INFO   OGG-01750  Successfullyunregistered EXTRACT EXT1 from database.

    Deleted EXTRACT EXT1.

    GGSCI (gg1) 60> delete dpump

    Deleted EXTRACT DPUMP.

    GGSCI (gg1) 61> info all

    Program    Status      Group       Lag           Time Since Chkpt

    MANAGER    RUNNING                                          

    2.2 重新配置GG 环境

    2.2.1 Source System创建Extract和 Data Pump

    --创建Extract: ext1

    GGSCI (gg1) 62> add extractext1,tranlog, begin now

    2011-11-17 16:56:21  INFO   OGG-01749  Successfully registeredEXTRACT EXT1 to start managing log retention at SCN 1374149.

    EXTRACT added.

    GGSCI (gg1) 63> add exttrail/u01/ggate/dirdat/lt, extract ext1

    EXTTRAIL added.

    GGSCI (gg1) 64> view params ext1

    extract ext1

    userid ggate@gg1, password ggate

    --rmthost gg2,mgrport 7809

    --rmttrail /u01/ggate/dirdat/lt

    exttrail /u01/ggate/dirdat/lt

    ddl include all objname dave.pdba;

    table dave.pdba;

    --创建DataPump:dpump

    GGSCI (gg1) 65> add extractdpump,exttrailsource /u01/ggate/dirdat/lt

    EXTRACT added.

    GGSCI (gg1) 66> add rmttrail/u01/ggate/dirdat/lt, extract dpump

    RMTTRAIL added.

    GGSCI (gg1) 67> view params dpump

    extract dpump

    userid ggate@gg1, password ggate

    rmthost gg2, mgrport 7809

    rmttrail /u01/ggate/dirdat/lt

    passthru

    table dave.pdba;

    2.2.2 Target System 创建Replicat

    --创建checkpoint 表

    GGSCI (gg2) 57> view param ./GLOBALS

    GGSCHEMA ggate

    CHECKPOINTTABLE ggate.checkpoint

    GGSCI (gg2) 58> dblogin userid ggate@gg2,password ggate

    Successfully logged into database.

    GGSCI (gg2) 59> add checkpointtable ggate.checkpoint

    Successfully created checkpoint tableGGATE.CHECKPOINT.

    --创建Replicat: rep1

    GGSCI (gg2) 60> add replicatrep1,exttrail /u01/ggate/dirdat/lt, checkpointtable ggate.checkpoint

    REPLICAT added.

    GGSCI (gg2) 61> view param rep1

    replicat rep1

    ASSUMETARGETDEFS

    userid ggate@gg2,password ggate

    discardfile/u01/ggate/dirdat/rep1_discard.txt, append, megabytes 10

    --HANDLECOLLISIONS

    ddl include all

    ddlerror default ignore retryop

    map dave.pdba, target dave.pdba;

    这里注意一点,因为我在删除进程时,并没有删除对应的参数文件,所以在创建同名的进程之后,就会默认使用之前的参数文件,在edit 进程时会显示参数文件的位置。

    默认放在$GGATE/dirprm目录下:

    gg2:/u01/ggate/dirprm> ls

    mgr.prm rep1.prm  rept2.prm

    2.3 启动Extract 和 Data Pump进程:ext1,dpump

           注意这里不要启动Replicat 进程,要等到我们用expdp/impdp 完成初始化之后,在用SCN 来启动replicat 进程。

    GGSCI (gg1) 70> start ext1

    Sending START request to MANAGER ...

    EXTRACT EXT1 starting

    GGSCI (gg1) 71> start dpump

    Sending START request to MANAGER ...

    EXTRACT DPUMP starting

    GGSCI (gg1) 73> info all

    Program    Status      Group       Lag           Time Since Chkpt

    MANAGER    RUNNING                                          

    EXTRACT    RUNNING     DPUMP       00:00:00      00:18:27   

    EXTRACT    RUNNING     EXT1        00:20:10      00:00:02   

    2.4 用带FLASHBACK_SCN 的expdp/impdp完成初始化

    2.4.1 在SourceDB 查询当前的SCN

    SQL> select current_scn from v$database;

    CURRENT_SCN

    -----------

       1376141

    2.4.2 导出pdba 表的数据

    gg1:/u01/backup> expdp  dave/dave directory=backup dumpfile=pdba.dmplogfile=table.log tables=pdba flashback_scn=1376141;

    Export: Release 11.2.0.3.0 - Production onThu Nov 17 17:24:49 2011

    Copyright (c) 1982, 2011, Oracle and/or itsaffiliates.  All rights reserved.

    Connected to: Oracle Database 11gEnterprise Edition Release 11.2.0.3.0 - 64bit Production

    With the Partitioning, OLAP, Data Miningand Real Application Testing options

    Starting"DAVE"."SYS_EXPORT_TABLE_02":  dave/******** directory=backupdumpfile=pdba.dmp logfile=table.log tables=pdba flashback_scn=1376141

    Estimate in progress using BLOCKS method...

    Processing object typeTABLE_EXPORT/TABLE/TABLE_DATA

    Total estimation using BLOCKS method: 60 MB

    Processing object typeTABLE_EXPORT/TABLE/TABLE

    Processing object typeTABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS

    . . exported"DAVE"."PDBA"                               48.51 MB 2678634rows

    Master table"DAVE"."SYS_EXPORT_TABLE_02" successfully loaded/unloaded

    ******************************************************************************

    Dump file set for DAVE.SYS_EXPORT_TABLE_02is:

     /u01/backup/pdba.dmp

    Job"DAVE"."SYS_EXPORT_TABLE_02" successfully completed at17:26:22

    2.4.3 删除pdba 表的部分数据

    SQL> delete from pdba whererownum<1000;

    999 rows deleted.

    SQL> commit;

    Commit complete.

    SQL> select current_scn from v$database;

    CURRENT_SCN

    -----------

       1377291

    2.4.4 将dumpscp到Target

    gg1:/u01/backup> scp pdba.dmp192.168.3.200:/u01/backup

    oracle@192.168.3.200's password:

    pdba.dmp                                         100%   49MB   6.1MB/s  00:08 

    2.4.5 impdp dump文件

    gg2:/u01/backup> impdp dave/davedirectory=backup dumpfile=pdba.dmp logfile=table.log tables=pdbatable_exists_action=replace;

    Import: Release 11.2.0.3.0 - Production onThu Nov 17 17:30:04 2011

    Copyright (c) 1982, 2011, Oracle and/or itsaffiliates.  All rights reserved.

    Connected to: Oracle Database 11gEnterprise Edition Release 11.2.0.3.0 - 64bit Production

    With the Partitioning, OLAP, Data Miningand Real Application Testing options

    Master table"DAVE"."SYS_IMPORT_TABLE_01" successfully loaded/unloaded

    Starting "DAVE"."SYS_IMPORT_TABLE_01":  dave/******** directory=backupdumpfile=pdba.dmp logfile=table.log tables=pdba table_exists_action=replace

    Processing object typeTABLE_EXPORT/TABLE/TABLE

    Processing object typeTABLE_EXPORT/TABLE/TABLE_DATA

    . . imported "DAVE"."PDBA"                               48.51 MB 2678634rows

    Processing object typeTABLE_EXPORT/TABLE/STATISTICS/TABLE_STATISTICS

    Job"DAVE"."SYS_IMPORT_TABLE_01" successfully completed at17:31:00

    2.5 用SCN 启动Replicat

    GGSCI (gg2) 63> start rep1, aftercsn 1376141

    Sending START request to MANAGER ...

    REPLICAT REP1 starting

    GGSCI (gg2) 64> info rep1

    REPLICAT  REP1      Last Started 2011-11-1717:32   Status RUNNING

    Checkpoint Lag       00:00:00 (updated 00:00:04 ago)

    Log Read Checkpoint  File /u01/ggate/dirdat/lt000000

                        First Record  RBA 978

    2.6 验证

    只要Source 和Target 表上的pdba 记录数一致,就说明,已经ok了。

    --Source DB:

    SQL> conn dave/dave;

    Connected.

    SQL> select count(*) from pdba;

     COUNT(*)

    ----------

      2678634

    --Target DB:

    SQL> conn dave/dave;

    Connected.

    SQL> select count(*) from pdba;

     COUNT(*)

    ----------

      2678634

    同步正常。基于SCN 的初始化示例到此结束。

    -------------------------------------------------------------------------------------------------------

    版权所有,文章允许转载,但必须以链接方式注明源地址,否则追究法律责任!

    Blog:     http://blog.csdn.net/tianlesoftware

    Weibo: http://weibo.com/tianlesoftware

    Email:   tianlesoftware@gmail.com

    Skype: tianlesoftware

    -------加群需要在备注说明Oracle表空间和数据文件的关系,否则拒绝申请----

    DBA1 群:62697716(满);   DBA2 群:62697977(满)  DBA3 群:62697850(满)  

    DBA 超级群:63306533(满);  DBA4 群:83829929(满) DBA5群: 142216823(满) 

    DBA6 群:158654907(满)   DBA7 群:69087192(满)  DBA8 群:172855474

    DBA 超级群2:151508914  DBA9群:102954821     聊天 群:40132017(满)

  • 相关阅读:
    JAVA中的SimpleDateFormat yyyy和YYYY的区别
    Mysql的MVCC
    SELECT语句中的for update的用法(锁的运用)
    今天简单说一下cdc 的使用
    sqlserver cdc用法
    JAVA | Java对象的内存分配过程是如何保证线程安全的?
    物联网(莹石云)WIFI一键配置原理分析(zz)
    Dell xps 13 9350待机时总是关机的处理方法
    Vue系列:在vux的popup组件中使用百度地图遇到显示不全的问题
    如何通过百度地图将经纬度转换为地址信息
  • 原文地址:https://www.cnblogs.com/tianlesoftware/p/3609491.html
Copyright © 2020-2023  润新知