• mycat+mysql集群:实现读写分离,分库分表


    本博客已迁移至www.lylblog.com,最新博文请访问新站点,欢迎各位来访。

    1.mycat文档:https://github.com/MyCATApache/Mycat-doc       官方网站:http://www.mycat.org.cn/ 2.mycat的优点

    • 配置简单,灵活
    • 可实现读写分离
    • 可利用多种规则实现分库分表
    • 心跳机制,自动踢出故障机组
    • 免费开源,长期维护,社区活跃

        mycat的缺点:

    • 主要是分片之后有一些限制,如只能2表join

    3.架构图:可实现读写分离,分库分表
    4.所需软件,机器配置与ip地址

    名称 版本 下载地址
    mysql server mysql  Ver 14.14 Distrib 5.6.27, for Linux (x86_64) yum源安装,yum源地址:http://dev.mysql.com/get/mysql57-community-release-el6-7.noarch.rpm
    mycat 1.3.0.3 https://github.com/MyCATApache/Mycat-download/blob/master/1.3.0.3-release/Mycat-server-1.3.0.3-release-20150527095523-linux.tar.gz
    keepalived   http://www.keepalived.org/download.html
    机器名  ip地址 配置 用途
    mycat1 192.168.2.155/192.168.10.30 4G,4C mycat+keepalived (主)
    mycat2 192.168.2.156/192.168.10.31 4G,4C mycat+keepalived (备)
    db1 192.168.10.155 8G,4C mysql master1
    db2 192.168.10.156 8G,4C mysql master2
    db3 192.168.10.157 8G,4C mysql slave1

    5.安装,配置mysql server

    • 获取官方yum源:  wget http://dev.mysql.com/get/mysql57-community-release-el6-7.noarch.rpm
    • 安装源:yum install mysql57-community-release-el6-7.noarch.rpm
    • 默认yum安装的是5.7,我们修改yum源,下载5.6的mysql server
    • 进入yum源配置文件:cd /etc/yum.repos.d/
    • 找到并编辑:vim mysql-community.repo
    • 找到下面这些内容:enabled=1就是可用,把[mysql56-community]段的enabled=0改为enabled=1,相应的把[mysql57-community]段的改为0,保存退出

                    # Enable to use MySQL 5.6                 [mysql56-community]                 name=MySQL 5.6 Community Server                 baseurl=http://repo.mysql.com/yum/mysql-5.6-community/el/6/$basearch/                 enabled=0                 gpgcheck=1                 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-mysql                                  [mysql57-community]                 name=MySQL 5.7 Community Server                 baseurl=http://repo.mysql.com/yum/mysql-5.7-community/el/6/$basearch/                 enabled=1                 gpgcheck=1                 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-mysql

    • 查找安装包(会更新源,时间几分钟,看你的网速咯):yum list |grep mysql
    • 如果顺利就会看到这一行:mysql-community-server.x86_64              5.6.27-2.el6
    • 没错就是它,安装:yum install -y mysql-community-server.x86_6
    • 启动(这一步会自动初始化一些内容):service mysqld start
    • 修改root密码:mysqladmin -uroot --password ‘xxxxx’
    • 修改配置文件,我测试时候的配置文件:
    • db1:

    [mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock character_set_server = utf8
    back_log=1024 max_connections = 10000 max_connect_errors = 1000 read_buffer_size = 4M query_cache_size = 64M key_buffer_size=400M max_allowed_packet=128M innodb_flush_log_at_trx_commit=0 innodb_log_file_size = 200M innodb_log_files_in_group = 3 innodb_buffer_pool_size = 2048M innodb_log_buffer_size = 16M query_cache_size = 0 #init_connect='SET autocommit=0' innodb_lock_wait_timeout = 50
    # Disabling symbolic-links is recommended to prevent assorted security risks symbolic-links=0
    # Recommended in standard MySQL setup sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES #replication setting server-id=1 log-bin=mysql-bin relay-log=mysql-relay-bin binlog-do-db=33hao binlog-do-db=jaydb binlog-do-db=sbtest binlog-do-db=shopnc binlog-ignore-db=mysql binlog-ignore-db=performance_schema binlog-ignore-db=information_schema log-slave-updates=1 master-info-repository=table relay-log-info-repository=table relay-log-recovery=1 #for mmm setting #read_only=1 #general log general_log=1 general_log_file=/var/log/mysql/mysql-row.log #no lower_case lower_case_table_names = 1 #slow log slow_query_log=1 slow_query_log_file=/var/log/mysql/mysql-slow.log long_query_time=2 log-queries-not-using-indexes #default timestamp explicit_defaults_for_timestamp=true
    [mysqld_safe] log-error=/var/log/mysql/mysqld.log pid-file=/var/run/mysqld/mysqld.pid

    • db2:

    [mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock character_set_server = utf8
    back_log=1024 max_connections = 10000 max_connect_errors = 1000 read_buffer_size = 4M query_cache_size = 64M key_buffer_size=400M max_allowed_packet=128M innodb_flush_log_at_trx_commit=0 innodb_log_file_size = 200M innodb_log_files_in_group = 3 innodb_buffer_pool_size = 2048M innodb_log_buffer_size = 16M query_cache_size = 0 #init_connect='SET autocommit=0' innodb_lock_wait_timeout = 50
    # Disabling symbolic-links is recommended to prevent assorted security risks symbolic-links=0
    # Recommended in standard MySQL setup sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES #replication setting server-id=2 log-bin=mysql-bin relay-log=mysql-relay-bin binlog-do-db=33hao binlog-do-db=jaydb binlog-do-db=sbtest binlog-do-db=shopnc binlog-ignore-db=mysql binlog-ignore-db=performance_schema binlog-ignore-db=information_schema log_slave_updates=1 master-info-repository=table relay-log-info-repository=table relay-log-recovery=1 #for mmm setting #read_only=1 #general log general_log=1 general_log_file=/var/log/mysql/mysql-row.log #no lower_case lower_case_table_names = 1 #slow log slow_query_log=1 slow_query_log_file=/var/log/mysql/mysql-slow.log long_query_time=2 log-queries-not-using-indexes #default timestamp explicit_defaults_for_timestamp=true
    [mysqld_safe] log-error=/var/log/mysql/mysqld.log pid-file=/var/run/mysqld/mysqld.pid

    • db3:    

    [mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock character_set_server = utf8
    back_log=1024 max_connections = 10000 max_connect_errors = 1000 read_buffer_size = 4M query_cache_size = 64M key_buffer_size=400M max_allowed_packet=128M innodb_flush_log_at_trx_commit=0 innodb_log_file_size = 200M innodb_log_files_in_group = 3 innodb_buffer_pool_size = 2048M innodb_log_buffer_size = 16M query_cache_size = 0 #init_connect='SET autocommit=0' innodb_lock_wait_timeout = 50
    # Disabling symbolic-links is recommended to prevent assorted security risks symbolic-links=0
    # Recommended in standard MySQL setup sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES #replication setting server-id=3 #log-bin=mysql-bin relay-log=mysql-relay-bin binlog-do-db=33hao binlog-do-db=jaydb binlog-do-db=sbtest binlog-do-db=shopnc binlog-ignore-db=mysql binlog-ignore-db=performance_schema binlog-ignore-db=information_schema master-info-repository=table relay-log-info-repository=table relay-log-recovery=1 #for mmm setting #read_only=1 #general log general_log=1 general_log_file=/var/log/mysql/mysql-row.log #no lower_case lower_case_table_names = 1 #slow log slow_query_log=1 slow_query_log_file=/var/log/mysql/mysql-slow.log long_query_time=2 log-queries-not-using-indexes #default timestamp explicit_defaults_for_timestamp=true
    [mysqld_safe] log-error=/var/log/mysql/mysqld.log pid-file=/var/run/mysqld/mysqld.pid

        

    6.从最上面的架构图中可以看出,这里只有一个主主(db1,db2),一个主从(db1,db3),下面分别配置主主,主从

    7.mysql主主复制配置

    (1)分别编辑db1,db2配置文件,把第5步相应的内容复制到/etc/my.cnf中 (2)接下来手动同步数据 (3)mysql>FLUSH TABLES WITH READ LOCK (4)不要退出终端,另外开一个,用mysqldump工具导出所有数据库数据,到db2数据库中 (5)在db1(192.168.10.155)的mysql上授权:grant replication slave on *.* to 'repl_user'@'192.168.10.156' identified by 'repl_password'; (6)查看db1的master状态,设置从(db2机器的mysql)的时候需要

                    mysql> show master status;                 +------------------+----------+--------------+------------------+-------------------+                 | File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |                 +------------------+----------+--------------+------------------+-------------------+                 | mysql-bin.000001 |      336 |              |                  |                   |                 +------------------+----------+--------------+------------------+-------------------+ (7)在db2中设置master参数,(master_log_file,master_log_pos,都来自上一步)                 mysql> change master to master_host='192.168.10.156', master_user='repl_user',master_password='repl_passwd',master_log_file='mysql-bin.000001',master_log_pos=336; (8)启动db2从的复制                 mysql>slave start; (9)查看从的状态                 mysql>show slave status\G;                 Slave_IO_Running: Yes                 Slave_SQL_Running: Yes                 Seconds_Behind_Master: 0                 看到这几行说明复制成功。 (10)把db1设置为db2的从                 方法同上,重复(5)-(9),只是把db1和db2,相应的IP地址互换即可,注意重复第(7)的时候master_log_file,master_log_pos是db2的master status了。

    8.把db3(192.168.10.157)设置为db1的从

    方法通与7步类似,重复其中的(1)-(7)步,只是把db2和192.168.10.156换成db3和192.168.10.157。

    9.至此,mysql的设置全部完成。

    10.下载,安装mycat

    • wget https://github.com/MyCATApache/Mycat-download/blob/master/1.3.0.3-release/Mycat-server-1.3.0.3-release-20150527095523-linux.tar.gz
    • tar -zxf Mycat-server-1.3.0.3-release-20150527095523-linux.tar.gz
    • 解压完成就可以使用,建议把解压后的文件夹放在/opt下,并建立mycat的软连接(ln -s “解压出来的文件夹” mycat),方便以后升级,维护。

    11.配置mycat/conf/schema.xml

    (1)在这里,我对33hao数据库实现读写分离 (2)对sbtest数据库所有表实现,读写不分离(当然,也可以针对某一张表,这种设置应对读写实时要求很高的情况,即单表或单库读写不分离) (3)对transdb数据库的表实现分片,users表不分片(type="global")提高join效率,orders表,用id通过mod-log规则分片(primaryKey="id" rule="mod-long")

    <?xml version="1.0"?> < !DOCTYPE mycat:schema SYSTEM "schema.dtd"> < mycat:schema xmlns:mycat="http://org.opencloudb/">
        <schema name="33hao" checkSQLschema="false" dataNode="dn1">     </schema>     <schema name="sbtest" checkSQLschema="false" dataNode="dn2">     </schema>     <schema name="jaydb" checkSQLschema="false" dataNode="dn3">     </schema>     <schema name="transdb" checkSQLschema="false">         <table name="users" primaryKey="id" type="global" dataNode="dn4,dn5,dn6" />
                    <!-- random sharding using mod sharind rule -->             <table name="orders" primaryKey="id" rule="mod-long" dataNode="dn4,dn5,dn6" />     </schema>     <schema name="shopnc" checkSQLschema="false" dataNode="dn7">     </schema>     <dataNode name="dn1" dataHost="localhost1" database="33hao" />     <dataNode name="dn2" dataHost="localhost2" database="sbtest" />     <dataNode name="dn3" dataHost="localhost1" database="jaydb" />     <dataNode name="dn4" dataHost="localhost4" database="transdb" />     <dataNode name="dn5" dataHost="localhost5" database="transdb" />     <dataNode name="dn6" dataHost="localhost6" database="transdb" />     <dataNode name="dn7" dataHost="localhost1" database="shopnc" />
        <dataHost name="localhost1" maxCon="100000" minCon="10" balance="1"         writeType="0" dbType="mysql" dbDriver="native">         <heartbeat>select user()</heartbeat>         <!-- can have multi write hosts -->         <writeHost host="hostM1" url="db1:3306" user="root"             password="123456">             <!-- can have multi read hosts -->             <readHost host="hostS1" url="db3:3306" user="root" password="123456"/>             <readHost host="hostS2" url="db2:3306" user="root" password="123456"/>         </writeHost>         <writeHost host="hostM2" url="db2:3306" user="root"             password="123456">             <!-- can have multi read hosts -->         </writeHost>         <!-- <writeHost host="hostM2" url="db2:3306" user="root" password="123456"/> -->     </dataHost>     <dataHost name="localhost2" maxCon="100000" minCon="10" balance="1"         writeType="0" dbType="mysql" dbDriver="native">         <heartbeat>select user()</heartbeat>         <!-- can have multi write hosts -->         <writeHost host="hostM1" url="db1:3306" user="root"             password="123456">         </writeHost>     </dataHost>     <dataHost name="localhost4" maxCon="100000" minCon="10" balance="0"         writeType="0" dbType="mysql" dbDriver="native">         <heartbeat>select user()</heartbeat>         <!-- can have multi write hosts -->         <writeHost host="hostM1" url="db1:3306" user="root" password="123456"></writeHost>     </dataHost>     <dataHost name="localhost5" maxCon="100000" minCon="10" balance="0"         writeType="0" dbType="mysql" dbDriver="native">         <heartbeat>select user()</heartbeat>         <!-- can have multi write hosts -->         <writeHost host="hostM2" url="db2:3306" user="root" password="123456"></writeHost>     </dataHost>     <dataHost name="localhost6" maxCon="100000" minCon="10" balance="0"         writeType="0" dbType="mysql" dbDriver="native">         <heartbeat>select user()</heartbeat>         <!-- can have multi write hosts -->         <writeHost host="hostS1" url="db3:3306" user="root" password="123456"></writeHost>     </dataHost>
    12.设置mycat/conf/server.xml

    • 这里是针对mycat的设置,如外部访问mycat的端口,用户名,密码,mycat的数据库有哪些等

    <?xml version="1.0" encoding="UTF-8"?> < !-- - - Licensed under the Apache License, Version 2.0 (the "License");         - you may not use this file except in compliance with the License. - You         may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0         - - Unless required by applicable law or agreed to in writing, software -         distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT         WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the         License for the specific language governing permissions and - limitations         under the License. --> < !DOCTYPE mycat:server SYSTEM "server.dtd"> < mycat:server xmlns:mycat="http://org.opencloudb/">         <system>         <property name="defaultSqlParser">druidparser</property>               <property name="processors">32</property>               <property name="processorExecutor">32</property>               <property name="serverPort">3306</property>               <property name="managerPort">9066</property>
            </system>         <user name="root">                 <property name="password">123456</property>                 <property name="schemas">33hao,sbtest,jaydb,transdb,shopnc</property>         </user> < /mycat:server> 13.启动mycat

    • /opt/mycat/bin/mycat start

    14.连接mycat

    • 和连接mysql方法一样,如果连接不上,看看是否在没个节点上都授予了mycat机器通过root/123456(在schema.xml中配置的)的方式访问:
    • mysql -uroot -p123456 -h127.0.0.1 -P3306

    15.测试mycat

    • 我开起了general_log,数据库任何操作都会被记录,依次打开三个db1 db2 db3的日志:tail -f /var/log/mysql/mysql-row.log
    • 透过mycat连接数据库 mysql -uroot -p123456 -h127.0.0.1 -P3306
    • 在相应的数据库中执行操作,并观察日志。
    • 比如对33hao的写总是在db1上,然后replication到db2,db3,如果db1宕机,写总是在db2上,db1恢复后,db2不会被抢占写的角色。读在db2,db3上随机执行。
    • 而transdb的orders表被均匀地复制到db1,db2,db3的transdb的orders表中,users则在所有mysql(这里为db1,db2,db3)中保存副本(因为在schema.xml中设置为:type="global")。
  • 相关阅读:
    mormot json操作
    DELPHI优秀的一些开源框架:QDAC,MORMOT,DIOCP
    数据库中间件支持数据库集群方案
    idhttp的用法
    firedac odbc sql server driver连接占线导致另一个hstmt
    论DATASNAP远程方法支持自定义对象作参数
    咏南中间件
    JS--截取字符串常用方法详细
    使用JavaScript进行进制转换将字符串转换为十进制
    Mac--安装kubernetes并运行echoserver
  • 原文地址:https://www.cnblogs.com/lylsr/p/10629061.html
Copyright © 2020-2023  润新知