• mycat 连续分片 -> 自己定义数字范围分片


    1,自己定义数字范围分片

    自己定义数字范围分片,提前规划好分片字段某个范围属于哪个分片,比方说将第一个500W的数据分片在第一个节点上面。第二个500W的数据分片在第二个节点上,依次类推


    2,加入配置文件

    在function.xml里面进行配置:

        <!-- 对自己定义数字分片规则rang-long-tr04的定义 -->
            <function name="rang-long-04" class="org.opencloudb.route.function.AutoPartitionByLong">
                    <property name="mapFile">autopartition-long04.txt</property>
                    <property name="defaultNode">0</property> 
                    <property name="type">0</property>
            </function>
    
            <!-- 对表进行分片的描写叙述。字段以及分片规则 -->
            <tableRule name="rang-long-tr04">
                     <rule>
                             <columns>ID</columns>
                              <algorithm>rang-long-04</algorithm>
                    </rule>
            </tableRule>

    在schema.xml里面进行配置:

        <table name="ORDER0401" primaryKey="ID" dataNode="dn21,dn22" rule="rang-long-tr04"/>
    

    autopartition-long04.txt文件的配置:

        [root@crm_idc_squid_1_11 conf]# more autopartition-long04.txt 
        0-500M=0
        500M-1000M=1
        1000M-1500M=2
        You have mail in /var/spool/mail/root
        [root@crm_idc_squid_1_11 conf]#     

    注意: 此配置很easy。即预先制定可能的id范围到某个分片。全部的节点配置都是从0開始。及0代表节点1


    3,建表而且录入数据

        CREATE TABLE ORDER0401(ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,PROVINCE VARCHAR(16),SN VARCHAR(64),CREATE_TIME DATETIME);
    mysql> CREATE TABLE ORDER0401(ID INT NOT NULL AUTO_INCREMENT PRIMARY KEY,PROVINCE VARCHAR(16),SN VARCHAR(64),CREATE_TIME DATETIME);
    Query OK, 0 rows affected (0.05 sec)
    
    mysql> 
        INSERT INTO ORDER0401(ID,PROVINCE,SN,CREATE_TIME) VALUES(10001,'shanghai','shanghai10006_10000',NOW());
        INSERT INTO ORDER0401(ID,PROVINCE,SN,CREATE_TIME) VALUES(5000001,'shanghai','shanghai10006_5000001',NOW());
        mysql> explain INSERT INTO ORDER0401(ID,PROVINCE,SN,CREATE_TIME) VALUES(10001,'shanghai','shanghai10006_10000',NOW());
        +-----------+--------------------------------------------------------------------------------------------------------+
        | DATA_NODE | SQL                                                                                                    |
        +-----------+--------------------------------------------------------------------------------------------------------+
        | dn21      | INSERT INTO ORDER0401(ID,PROVINCE,SN,CREATE_TIME) VALUES(10001,'shanghai','shanghai10006_10000',NOW()) |
        +-----------+--------------------------------------------------------------------------------------------------------+
        1 row in set (0.01 sec)
    
        mysql> INSERT INTO ORDER0401(ID,PROVINCE,SN,CREATE_TIME) VALUES(10001,'shanghai','shanghai10006_10000',NOW());
        Query OK, 1 row affected (0.01 sec)
    
        mysql> INSERT INTO ORDER0401(ID,PROVINCE,SN,CREATE_TIME) VALUES(5000001,'shanghai','shanghai10006_5000001',NOW());
        Query OK, 1 row affected (0.00 sec)
    
        mysql> 

    4,对insert操作流程进行分析。參考mycat.log对路由过程做完整的分析

    查看后台mycat.log分析例如以下:

    開始获取连接:
        02/12 21:17:13.614  DEBUG [$_NIOREACTOR-1-RW] (ServerQueryHandler.java:56) -ServerConnection [id=1, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=true, schema=TESTDB]INSERT INTO ORDER0401(ID,PROVINCE,SN,CREATE_TIME) VALUES(10001,'shanghai','shanghai10006_10000',NOW())
    
    開始建立缓存:
        02/12 21:17:13.615  DEBUG [$_NIOREACTOR-1-RW] (RouterUtil.java:951) -try to find cache by primary key
        02/12 21:17:13.616   INFO [$_NIOREACTOR-1-RW] (DefaultLayedCachePool.java:80) -create child Cache: TESTDB_ORDER0401 for layered cache TableID2DataNodeCache, size 10000, expire seconds 18000
        02/12 21:17:13.616  DEBUG [$_NIOREACTOR-1-RW] (CacheManager.java:794) -Attempting to create an existing singleton. Existing singleton returned.
        02/12 21:17:13.616  DEBUG [$_NIOREACTOR-1-RW] (Cache.java:955) -No BootstrapCacheLoaderFactory class specified. Skipping...
        02/12 21:17:13.617  DEBUG [$_NIOREACTOR-1-RW] (Cache.java:929) -CacheWriter factory not configured. Skipping...
        02/12 21:17:13.618  DEBUG [$_NIOREACTOR-1-RW] (MemoryStore.java:153) -Initialized net.sf.ehcache.store.NotifyingMemoryStore for TableID2DataNodeCache.TESTDB_ORDER0401
        02/12 21:17:13.619  DEBUG [$_NIOREACTOR-1-RW] (Cache.java:1165) -Initialised cache: TableID2DataNodeCache.TESTDB_ORDER0401
    
    開始路由匹配:
        02/12 21:17:13.621  DEBUG [$_NIOREACTOR-1-RW] (NonBlockingSession.java:113) -ServerConnection [id=1, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=true, schema=TESTDB]INSERT INTO ORDER0401(ID,PROVINCE,SN,CREATE_TIME) VALUES(10001,'shanghai','shanghai10006_10000',NOW()), route={
           1 -> dn21{INSERT INTO ORDER0401(ID,PROVINCE,SN,CREATE_TIME) VALUES(10001,'shanghai','shanghai10006_10000',NOW())}
        } rrs
    
    匹配好路由后。直接往相应的dn21节点開始运行insert数据录入操作:
        02/12 21:17:13.624  DEBUG [$_NIOREACTOR-0-RW] (NonBlockingSession.java:229) -release connection MySQLConnection [id=8, lastTime=1455283033608, user=root, schema=db3, old shema=db3, borrowed=true, fromSlaveDB=false, threadId=52, charset=latin1, txIsolation=3, autocommit=true, attachment=dn21{INSERT INTO ORDER0401(ID,PROVINCE,SN,CREATE_TIME) VALUES(10001,'shanghai','shanghai10006_10000',NOW())}, respHandler=SingleNodeHandler [node=dn21{INSERT INTO ORDER0401(ID,PROVINCE,SN,CREATE_TIME) VALUES(10001,'shanghai','shanghai10006_10000',NOW())}, packetId=0], host=13.217.1.11, port=3317, statusSync=null, writeQueue=0, modifiedSQLExecuted=true]
    
    连接释放:
        02/12 21:17:13.624  DEBUG [$_NIOREACTOR-0-RW] (PhysicalDatasource.java:403) -release channel MySQLConnection [id=8, lastTime=1455283033608, user=root, schema=db3, old shema=db3, borrowed=true, fromSlaveDB=false, threadId=52, charset=latin1, txIsolation=3, autocommit=true, attachment=null, respHandler=null, host=13.217.1.11, port=3317, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]

    5,运行简单的查询。假设指定分片字段ID字段,则走分片查询单个分片节点

    mycat命令台显示结果:

        mysql> explain select * from ORDER0401 t1 where t1.ID=5000001;
        +-----------+------------------------------------------------+
        | DATA_NODE | SQL                                            |
        +-----------+------------------------------------------------+
        | dn22      | select * from ORDER0401 t1 where t1.ID=5000001 |
        +-----------+------------------------------------------------+
        1 row in set (0.08 sec)
    
        mysql> select * from ORDER0401 t1 where t1.ID=5000001;
        +---------+----------+-----------------------+---------------------+
        | ID      | PROVINCE | SN                    | CREATE_TIME         |
        +---------+----------+-----------------------+---------------------+
        | 5000001 | shanghai | shanghai10006_5000001 | 2016-02-12 21:18:17 |
        +---------+----------+-----------------------+---------------------+
        1 row in set (0.00 sec)
    
        mysql> 

    查看后台mycat日志信息:

        02/12 21:44:01.116  DEBUG [$_NIOREACTOR-1-RW] (ServerQueryHandler.java:56) -ServerConnection [id=1, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=true, schema=TESTDB]select * from ORDER0401 t1 where t1.ID=5000001
        02/12 21:44:01.117  DEBUG [$_NIOREACTOR-1-RW] (EnchachePool.java:76) -SQLRouteCache  miss cache ,key:TESTDBselect * from ORDER0401 t1 where t1.ID=5000001
        02/12 21:44:01.118  DEBUG [$_NIOREACTOR-1-RW] (RouterUtil.java:951) -try to find cache by primary key
        02/12 21:44:01.118  DEBUG [$_NIOREACTOR-1-RW] (EnchachePool.java:76) -TableID2DataNodeCache.TESTDB_ORDER0401  miss cache ,key:5000001
        02/12 21:44:01.118  DEBUG [$_NIOREACTOR-1-RW] (NonBlockingSession.java:113) -ServerConnection [id=1, schema=TESTDB, host=127.0.0.1, user=test,txIsolation=3, autocommit=true, schema=TESTDB]select * from ORDER0401 t1 where t1.ID=5000001, route={
           1 -> dn22{select * from ORDER0401 t1 where t1.ID=5000001}
        } rrs
        02/12 21:44:01.119  DEBUG [$_NIOREACTOR-1-RW] (PhysicalDBPool.java:452) -select read source hostM2 for dataHost:m2
        02/12 21:44:01.120  DEBUG [$_NIOREACTOR-1-RW] (NonBlockingSession.java:229) -release connection MySQLConnection [id=17, lastTime=1455284641106, user=root, schema=db3, old shema=db3, borrowed=true, fromSlaveDB=false, threadId=59, charset=latin1, txIsolation=3, autocommit=true, attachment=dn22{select * from ORDER0401 t1 where t1.ID=5000001}, respHandler=SingleNodeHandler [node=dn22{select * from ORDER0401 t1 where t1.ID=5000001}, packetId=7], host=13.217.1.11, port=3327, statusSync=org.opencloudb.mysql.nio.MySQLConnection$StatusSync@6d1ef03, writeQueue=0, modifiedSQLExecuted=false]
        02/12 21:44:01.121  DEBUG [$_NIOREACTOR-1-RW] (PhysicalDatasource.java:403) -release channel MySQLConnection [id=17, lastTime=1455284641106, user=root, schema=db3, old shema=db3, borrowed=true, fromSlaveDB=false, threadId=59, charset=latin1, txIsolation=3, autocommit=true, attachment=null, respHandler=null, host=13.217.1.11, port=3327, statusSync=null, writeQueue=0, modifiedSQLExecuted=false]

    6。假设分片字段范围的查询,则走全部节点去检索,哪怕仅仅有一条数据在一个分片上。route路由也是走全部的分片进行检索查询

        mysql> explain select * from ORDER0401 t1 where t1.ID<50000;
        +-----------+----------------------------------------------------------+
        | DATA_NODE | SQL                                                      |
        +-----------+----------------------------------------------------------+
        | dn21      | SELECT * FROM ORDER0401 t1 WHERE t1.ID < 50000 LIMIT 100 |
        | dn22      | SELECT * FROM ORDER0401 t1 WHERE t1.ID < 50000 LIMIT 100 |
        +-----------+----------------------------------------------------------+
        2 rows in set (0.01 sec)
    
        mysql>  

    7,假设不走分片字段的查询。即使是单个数据,也要route路由全部的分片,走全部的分片进行查询

        mysql> explain select * from ORDER0401 t1 where t1.SN='shanghai10006_10000';
        +-----------+--------------------------------------------------------------------------+
        | DATA_NODE | SQL                                                                      |
        +-----------+--------------------------------------------------------------------------+
        | dn21      | SELECT * FROM ORDER0401 t1 WHERE t1.SN = 'shanghai10006_10000' LIMIT 100 |
        | dn22      | SELECT * FROM ORDER0401 t1 WHERE t1.SN = 'shanghai10006_10000' LIMIT 100 |
        +-----------+--------------------------------------------------------------------------+
        2 rows in set (0.00 sec)
    
        mysql> 
  • 相关阅读:
    自学传说中的php接口编写
    php数据类型
    php中的echo 与print 、var_dump 的区别
    vue 基础的一些字眼及路由
    初入 vue
    php 连接 数据库
    ExtJS获取父子、兄弟容器元素方法
    ext 的loadmask 与ajax的同步请求水火不容
    Ubuntu Server下配置UTF-8中文环境,ubuntu server zh_CN.UTF-8
    Proftpd快速搭建FTP服务器
  • 原文地址:https://www.cnblogs.com/cynchanpin/p/7091605.html
Copyright © 2020-2023  润新知