• Otter双A同步搭建入门教程


    1.准备阶段

    1.1 环境

    虚拟机A :192.168.0.130

    虚拟机B :192.168.0.131

    系统:Ubuntu 16.04 LTS

    JRE:OpenJDK 1.8.0_151(A、B都要安装)

    DB:MySQL 5.7.20(A、B都要安装)

    ZooKeeper:3.4.11(只装A机)

    Otter Manager:4.2.14 (只装A机)(4.2.15启动Node时感觉有Bug,没有使用)

    Otter Node:4.2.14(A、B都要安装)

    1.2 安装环境

    1.2.1 JRE安装

    sudo apt-get install default-jre

    1.2.2 MySQL安装

    sudo apt-get install mysql-server

    sudo vim /etc/mysql/mysql.conf.d/mysqld.cnf

    character-set-server=utf8
    log-bin=mysql-bin   
    binlog-format=ROW   
    server-id=1 #A,B需要设置成不一样的id
    #bind-address=127.0.0.1 #注释该行

    service mysql restart 

    进入mysql,设置单独账号密码

    grant all on *.* to 'root'@'%' identified by 'password';

    1.2.3 ZooKeeper安装

    安装包地址 http://mirrors.hust.edu.cn/apache/zookeeper/zookeeper-3.4.11/zookeeper-3.4.11.tar.gz

    解压可见conf文件夹下有一个zoo_sample.cfg的文件,重命名为zoo.cfg并修改以下配置

    dataDir=../datas
    dataLogDir=../logs
    

    1.2.4 Otter Manager安装

    安装包地址 https://github.com/alibaba/otter/releases 请自己选择版本

    解压可见conf文件夹下otter.properties文件,修改以下配置

    #以下配置为最基本需要改的配置,其他配置可根据实际要用的功能进行修改
    otter.domainName = 192.168.0.130 #一定要改,不要用127.0.0.1
    otter.port = 9000 #manager 站点端口号
    otter.database.driver.url = jdbc:mysql://192.168.0.130:3306/otter
    otter.database.driver.username = root
    otter.database.driver.password = password
    otter.zookeeper.cluster.default = 192.168.0.130:2181
    

    1.2.5 Otter Node 安装

    安装包地址 https://github.com/alibaba/otter/releases 请自己选择版本

    解压可见conf文件夹下otter.properties文件,修改以下配置

    #以下配置为最基本需要改的配置,其他配置可根据实际要用的功能进行修改
    otter.nodeHome = ../
    otter.manager.address = 192.168.0.130:1099 

    conf文件夹下创建一个新的配置文件,命名为nid,写入node id,A机为1,B机为2。该id后面会介绍。

    1  

     

    2.启动

    2.1 启动ZooKeeper

    进入ZooKeeper目录下的bin文件夹,执行以下命令(不要使用sh,一定要用bash)

    bash zkServer.sh start
    

    2.2 启动Otter Manager

    在A机中执行以下数据库语句,创建Otter库

    CREATE DATABASE /*!32312 IF NOT EXISTS*/ `otter` /*!40100 DEFAULT CHARACTER SET utf8 COLLATE utf8_bin */;
    
    USE `otter`;
    
    CREATE TABLE `ALARM_RULE` (
      `ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
      `MONITOR_NAME` varchar(1024) DEFAULT NULL,
      `RECEIVER_KEY` varchar(1024) DEFAULT NULL,
      `STATUS` varchar(32) DEFAULT NULL,
      `PIPELINE_ID` bigint(20) NOT NULL,
      `DESCRIPTION` varchar(256) DEFAULT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      `MATCH_VALUE` varchar(1024) DEFAULT NULL,
      `PARAMETERS` text DEFAULT NULL,
      PRIMARY KEY (`ID`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `AUTOKEEPER_CLUSTER` (
      `ID` bigint(20) NOT NULL AUTO_INCREMENT,
      `CLUSTER_NAME` varchar(200) NOT NULL,
      `SERVER_LIST` varchar(1024) NOT NULL,
      `DESCRIPTION` varchar(200) DEFAULT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `CANAL` (
      `ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
      `NAME` varchar(200) DEFAULT NULL,
      `DESCRIPTION` varchar(200) DEFAULT NULL,
      `PARAMETERS` text DEFAULT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      UNIQUE KEY `CANALUNIQUE` (`NAME`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `CHANNEL` (
      `ID` bigint(20) NOT NULL AUTO_INCREMENT,
      `NAME` varchar(200) NOT NULL,
      `DESCRIPTION` varchar(200) DEFAULT NULL,
      `PARAMETERS` text DEFAULT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      UNIQUE KEY `CHANNELUNIQUE` (`NAME`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `COLUMN_PAIR` (
      `ID` bigint(20) NOT NULL AUTO_INCREMENT,
      `SOURCE_COLUMN` varchar(200) DEFAULT NULL,
      `TARGET_COLUMN` varchar(200) DEFAULT NULL,
      `DATA_MEDIA_PAIR_ID` bigint(20) NOT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      KEY `idx_DATA_MEDIA_PAIR_ID` (`DATA_MEDIA_PAIR_ID`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `COLUMN_PAIR_GROUP` (
      `ID` bigint(20) NOT NULL AUTO_INCREMENT,
      `DATA_MEDIA_PAIR_ID` bigint(20) NOT NULL,
      `COLUMN_PAIR_CONTENT` text DEFAULT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      KEY `idx_DATA_MEDIA_PAIR_ID` (`DATA_MEDIA_PAIR_ID`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `DATA_MEDIA` (
      `ID` bigint(20) NOT NULL AUTO_INCREMENT,
      `NAME` varchar(200) NOT NULL,
      `NAMESPACE` varchar(200) NOT NULL,
      `PROPERTIES` varchar(1000) NOT NULL,
      `DATA_MEDIA_SOURCE_ID` bigint(20) NOT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      UNIQUE KEY `DATAMEDIAUNIQUE` (`NAME`,`NAMESPACE`,`DATA_MEDIA_SOURCE_ID`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `DATA_MEDIA_PAIR` (
      `ID` bigint(20) NOT NULL AUTO_INCREMENT,
      `PULLWEIGHT` bigint(20) DEFAULT NULL,
      `PUSHWEIGHT` bigint(20) DEFAULT NULL,
      `RESOLVER` text DEFAULT NULL,
      `FILTER` text DEFAULT NULL,
      `SOURCE_DATA_MEDIA_ID` bigint(20) DEFAULT NULL,
      `TARGET_DATA_MEDIA_ID` bigint(20) DEFAULT NULL,
      `PIPELINE_ID` bigint(20) NOT NULL,
      `COLUMN_PAIR_MODE` varchar(20) DEFAULT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      KEY `idx_PipelineID` (`PIPELINE_ID`,`ID`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `DATA_MEDIA_SOURCE` (
      `ID` bigint(20) NOT NULL AUTO_INCREMENT,
      `NAME` varchar(200) NOT NULL,
      `TYPE` varchar(20) NOT NULL,
      `PROPERTIES` varchar(1000) NOT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      UNIQUE KEY `DATAMEDIASOURCEUNIQUE` (`NAME`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `DELAY_STAT` (
      `ID` bigint(20) NOT NULL AUTO_INCREMENT,
      `DELAY_TIME` int(21) NOT NULL,
      `DELAY_NUMBER` bigint(20) NOT NULL,
      `PIPELINE_ID` bigint(20) NOT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      KEY `idx_PipelineID_GmtModified_ID` (`PIPELINE_ID`,`GMT_MODIFIED`,`ID`),
      KEY `idx_Pipeline_GmtCreate` (`PIPELINE_ID`,`GMT_CREATE`),
      KEY `idx_GmtCreate_id` (`GMT_CREATE`,`ID`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `LOG_RECORD` (
      `ID` bigint(20) NOT NULL AUTO_INCREMENT,
      `NID` varchar(200) DEFAULT NULL,
      `CHANNEL_ID` varchar(200) NOT NULL,
      `PIPELINE_ID` varchar(200) NOT NULL,
      `TITLE` varchar(1000) DEFAULT NULL,
      `MESSAGE` text DEFAULT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      KEY `logRecord_pipelineId` (`PIPELINE_ID`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `NODE` (
      `ID` bigint(20) NOT NULL AUTO_INCREMENT,
      `NAME` varchar(200) NOT NULL,
      `IP` varchar(200) NOT NULL,
      `PORT` bigint(20) NOT NULL,
      `DESCRIPTION` varchar(200) DEFAULT NULL,
      `PARAMETERS` text DEFAULT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      UNIQUE KEY `NODEUNIQUE` (`NAME`,`IP`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `PIPELINE` (
      `ID` bigint(20) NOT NULL AUTO_INCREMENT,
      `NAME` varchar(200) NOT NULL,
      `DESCRIPTION` varchar(200) DEFAULT NULL,
      `PARAMETERS` text DEFAULT NULL,
      `CHANNEL_ID` bigint(20) NOT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      UNIQUE KEY `PIPELINEUNIQUE` (`NAME`,`CHANNEL_ID`),
      KEY `idx_ChannelID` (`CHANNEL_ID`,`ID`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `PIPELINE_NODE_RELATION` (
      `ID` bigint(20) NOT NULL AUTO_INCREMENT,
      `NODE_ID` bigint(20) NOT NULL,
      `PIPELINE_ID` bigint(20) NOT NULL,
      `LOCATION` varchar(20) NOT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      KEY `idx_PipelineID` (`PIPELINE_ID`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `SYSTEM_PARAMETER` (
      `ID` bigint(20) unsigned NOT NULL,
      `VALUE` text DEFAULT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`)
    ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
    
    CREATE TABLE `TABLE_HISTORY_STAT` (
      `ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
      `FILE_SIZE` bigint(20) DEFAULT NULL,
      `FILE_COUNT` bigint(20) DEFAULT NULL,
      `INSERT_COUNT` bigint(20) DEFAULT NULL,
      `UPDATE_COUNT` bigint(20) DEFAULT NULL,
      `DELETE_COUNT` bigint(20) DEFAULT NULL,
      `DATA_MEDIA_PAIR_ID` bigint(20) DEFAULT NULL,
      `PIPELINE_ID` bigint(20) DEFAULT NULL,
      `START_TIME` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `END_TIME` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      KEY `idx_DATA_MEDIA_PAIR_ID_END_TIME` (`DATA_MEDIA_PAIR_ID`,`END_TIME`),
      KEY `idx_GmtCreate_id` (`GMT_CREATE`,`ID`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `TABLE_STAT` (
      `ID` bigint(20) NOT NULL AUTO_INCREMENT,
      `FILE_SIZE` bigint(20) NOT NULL,
      `FILE_COUNT` bigint(20) NOT NULL,
      `INSERT_COUNT` bigint(20) NOT NULL,
      `UPDATE_COUNT` bigint(20) NOT NULL,
      `DELETE_COUNT` bigint(20) NOT NULL,
      `DATA_MEDIA_PAIR_ID` bigint(20) NOT NULL,
      `PIPELINE_ID` bigint(20) NOT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      KEY `idx_PipelineID_DataMediaPairID` (`PIPELINE_ID`,`DATA_MEDIA_PAIR_ID`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `THROUGHPUT_STAT` (
      `ID` bigint(20) NOT NULL AUTO_INCREMENT,
      `TYPE` varchar(20) NOT NULL,
      `NUMBER` bigint(20) NOT NULL,
      `SIZE` bigint(20) NOT NULL,
      `PIPELINE_ID` bigint(20) NOT NULL,
      `START_TIME` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `END_TIME` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      KEY `idx_PipelineID_Type_GmtCreate_ID` (`PIPELINE_ID`,`TYPE`,`GMT_CREATE`,`ID`),
      KEY `idx_PipelineID_Type_EndTime_ID` (`PIPELINE_ID`,`TYPE`,`END_TIME`,`ID`),
      KEY `idx_GmtCreate_id` (`GMT_CREATE`,`ID`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE `USER` (
      `ID` bigint(20) NOT NULL AUTO_INCREMENT,
      `USERNAME` varchar(20) NOT NULL,
      `PASSWORD` varchar(20) NOT NULL,
      `AUTHORIZETYPE` varchar(20) NOT NULL,
      `DEPARTMENT` varchar(20) NOT NULL,
      `REALNAME` varchar(20) NOT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      UNIQUE KEY `USERUNIQUE` (`USERNAME`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE  `DATA_MATRIX` (
      `ID` bigint(20) NOT NULL AUTO_INCREMENT,
      `GROUP_KEY` varchar(200) DEFAULT NULL,
      `MASTER` varchar(200) DEFAULT NULL,
      `SLAVE` varchar(200) DEFAULT NULL,
      `DESCRIPTION` varchar(200) DEFAULT NULL,
      `GMT_CREATE` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      `GMT_MODIFIED` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
      PRIMARY KEY (`ID`),
      KEY `GROUPKEY` (`GROUP_KEY`)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    
    insert into USER(ID,USERNAME,PASSWORD,AUTHORIZETYPE,DEPARTMENT,REALNAME,GMT_CREATE,GMT_MODIFIED) values(null,'admin','801fc357a5a74743894a','ADMIN','admin','admin',now(),now());
    insert into USER(ID,USERNAME,PASSWORD,AUTHORIZETYPE,DEPARTMENT,REALNAME,GMT_CREATE,GMT_MODIFIED) values(null,'guest','471e02a154a2121dc577','OPERATOR','guest','guest',now(),now());
    

    在A,B机中都要执行以下数据库语句,创建retl库(若只是单向同步,则不需要执行该语句。该语句用作双A同步时的算法)

    /*
    供 otter 使用, otter 需要对 retl.* 的读写权限,以及对业务表的读写权限
    1. 创建database retl
    */
    CREATE DATABASE retl;
    
    /* 2. 用户授权 给同步用户授权 */
    CREATE USER retl@'%' IDENTIFIED BY 'retl';
    GRANT USAGE ON *.* TO `retl`@'%';
    GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO `retl`@'%';
    GRANT SELECT, INSERT, UPDATE, DELETE, EXECUTE ON `retl`.* TO `retl`@'%';
    /* 业务表授权,这里可以限定只授权同步业务的表 */
    GRANT SELECT, INSERT, UPDATE, DELETE ON *.* TO `retl`@'%';  
    
    /* 3. 创建系统表 */
    USE retl;
    DROP TABLE IF EXISTS retl.retl_buffer;
    DROP TABLE IF EXISTS retl.retl_mark;
    DROP TABLE IF EXISTS retl.xdual;
    
    CREATE TABLE retl_buffer
    (	
    	ID BIGINT(20) AUTO_INCREMENT,
    	TABLE_ID INT(11) NOT NULL,
    	FULL_NAME varchar(512),
    	TYPE CHAR(1) NOT NULL,
    	PK_DATA VARCHAR(256) NOT NULL,
    	GMT_CREATE TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    	GMT_MODIFIED TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    	CONSTRAINT RETL_BUFFER_ID PRIMARY KEY (ID) 
    )  ENGINE=InnoDB DEFAULT CHARSET=utf8;
    
    CREATE TABLE retl_mark
    (	
    	ID BIGINT AUTO_INCREMENT,
    	CHANNEL_ID INT(11),
    	CHANNEL_INFO varchar(128),
    	CONSTRAINT RETL_MARK_ID PRIMARY KEY (ID) 
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    CREATE TABLE xdual (
      ID BIGINT(20) NOT NULL AUTO_INCREMENT,
      X timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      PRIMARY KEY (ID)
    ) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
    
    /* 4. 插入初始化数据 */
    INSERT INTO retl.xdual(id, x) VALUES (1,now()) ON DUPLICATE KEY UPDATE x = now();
    

    进入Otter Manager 下的bin目录

    bash startup.sh
    

    这时等待数秒,Otter Manager站点已启动,可以访问http://192.168.0.130:9000配置Otter Node 的信息了。

    2.3 启动Otter Node

    如果你未在Otter Manager站点配置Otter Node的信息,则请暂时跳过该节,直接查看下面配置章节,配置完成后再回来该节启动Otter Node

    进入Otter Node下的bin目录

    bash startup.sh
    

        

    3.配置

    进入Otter Manager站点后,使用账号:admin,密码:admin(默认),获得超级管理员权限。

    3.1 ZooKeeper配置

    机器管理  -->  ZooKeeper管理 ,添加,保存,如下图

     3.2 Node配置

    机器管理  -->  Node管理 ,添加,保存(2次,A机一个Node,B机一个Node),如下图(名称和IP需要根据不同机器修改,这里只列出A机图片)

    最后可在Node管理列表中看见,其中序号即我们上面安装过程中写入的nid文件的id号,配置好之后,即可启动Otter Node

    启动后可见配置列表

    3.3 数据源配置

    配置管理  -->  数据源配置,配置A机、B机两个MySQL 数据源

    配置完成后可见配置列表

    3.4 数据表配置

    配置管理  -->  数据表配置,配置A机、B机两个MySQL 数据源所需要同步的数据表(自己的数据库,我这里新建test库),其中table name配置如果想同步所有表则配置为.*,否则则配置对应表名即可

    配置完成后可见配置列表

    3.5 Canal配置

    将canal看作是A、B两机的从库即可,github上解释道,canal模拟mysql slave的交互协议,伪装自己为mysql slave,向mysql master发送dump协议。

    配置管理  -->  canal配置,配置A机、B机两个canal。实质上两个canal分别运行在A、B两机上的Node节点上。配置如下,其他使用默认配置。

    位点信息分别在A、B两个MySQL执行以下语句获取

    show master status;
    select unix_timestamp(now());

    配置完成后可见配置列表

    3.6 同步任务配置

    同步管理 -->  Channel管理 ,添加一个Channel

     

    添加后,点击进入Pipeline管理

    添加两个Pipeline,如图是其中一个方向,另外一个方向Select、Load机器相反。canal选择与Node机器选择一致方向,即要与Select机器一样。选择其中一个Pipeline作为主站点,并在高级设置中勾选支持ddl同步,另外一个主站点勾选否,支持ddl同步选择否。

    添加后,点击进入映射关系列表,选择好同步表的方向

    至此。配置完成。到Channel管理中开启同步任务,Otter将为我们自动双A同步数据啦!

     

     4. 参考资料

    https://github.com/alibaba/otter

    https://github.com/alibaba/canal

    https://junnan.org/2017/01/mysql-two-way-synchronization-use-otter.html

    https://yq.aliyun.com/articles/58388?spm=5176.100239.blogcont58420.15.ZyRocX

    http://download.csdn.net/download/yjx19930417/10170913

  • 相关阅读:
    6.素数和(素数的判断)
    6.素数和(素数的判断)
    5.明明的随机数(桶排序经典例题)
    5.明明的随机数(桶排序经典例题)
    5.明明的随机数(桶排序经典例题)
    5.明明的随机数(桶排序经典例题)
    Algs4-1.2.11根据Date的API实现一个SmartDate类型
    Algs4-1.2.10编写一个类VisualCounter
    Algs4-1.2.9使用Counter统计BinarySearch检查的key个数
    Algs4-1.2.8引用型变量赋值-数组复制
  • 原文地址:https://www.cnblogs.com/Inspire-Yi/p/8094325.html
Copyright © 2020-2023  润新知