• ProxySQL 读写分离方法


    转载自:https://www.jianshu.com/p/597b840bf70c

    (使用正则表达式实现基本的读/写分离)
    在这一部分,我将通过一个示例来演示如何通过正则表达式来实现读/写分离。
    首先,我们需要将之前创建的查询规则删除:
    DELETE FROM mysql_query_rules;
    
    然后,为读/写分别创建对应的基本规则:
    UPDATE mysql_users SET default_hostgroup=10; # 所有的查询将默认路由到 HG10
    LOAD MYSQL USERS TO RUNTIME;
    SAVE MYSQL USERS TO DISK;     # 如果想要将配置持久化保存到磁盘,就执行该语句
    INSERT INTO mysql_query_rules (rule_id,active,match_digest,destination_hostgroup,apply)
    VALUES
    (1,1,'^SELECT.*FOR UPDATE$',10,1),
    (2,1,'^SELECT',20,1);
    LOAD MYSQL QUERY RULES TO RUNTIME;
    SAVE MYSQL QUERY RULES TO DISK;    # 如果想要将配置持久化保存到磁盘,就执行该语句
    
    现在,将按照如下规则对查询语句进行路由:
    
        所有SELECT FOR UPDATE语句将路由给HG10。
        其它所有的SELECT语句都将路由给HG20。
        其它所有查询也都将路由给HG10(这是默认主机组)
        注意,我并不认为上面的读/写分离规则是多好的方法。我经常使用这个例子来描述如何配置规则,但它经常被误解为配置读/写分离的方式 。稍后我将演示更好的读/写方法。
        现在,将上面的规则全部移除:
    
    DELETE FROM mysql_query_rules;
    LOAD MYSQL QUERY RULES TO RUNTIME;
    SAVE MYSQL QUERY RULES TO DISK; # 如果想将配置持久化到磁盘,执行该语句
    read/write split using regex and digest
    
    (使用正则表达式和digest配置读写分离)
    有效地设置读/写分离的配置过程如下:
    
        首先配置ProxySQL,让所有的请求都发送给单个MySQL节点master(既读又写)
        检查stats_mysql_query_digest表中开销最大的SELECT语句
        然后决定这些大开销的语句中哪些允许路由给负责读的节点
        然后配置mysql_query_rules表(创建规则),将大开销的SELECT语句路由给读组
        这个方案的本质非常简答:只发送那些想要发送给slaves/readers的查询,而不仅仅通过SELECT语句来判断。
    
    Find expensive queries using stats_mysql_query_digest
    
    (从stats_mysql_query_digest表中找出开销较大的查询语句)
    这里给出了几个示例,用来解释如何找出哪些应该路由给读组的潜在查询。
    由于ProxySQL将所有的统计数据和指标都导出到了各种表中,因此可以根据这些表创建复杂的查询来收集想要的信息。
    下面是在一个非常繁忙的ProxySQL实例下收集的数据,这个实例已经持续运行了几个月,处理的数据有上千亿。
    
        首先查找出总执行时间排在前5的查询:
    
    Admin> SELECT digest,SUBSTR(digest_text,0,25),count_star,sum_time FROM stats_mysql_query_digest WHERE digest_text LIKE 'SELECT%' ORDER BY sum_time DESC LIMIT 5;
    +--------------------+--------------------------+------------+---------------+
    | digest             | SUBSTR(digest_text,0,25) | count_star | sum_time      |
    +--------------------+--------------------------+------------+---------------+
    | 0x037C3E6D996DAFE2 | SELECT a.ip_id as ip_id, | 2030026798 | 1479082636017 |
    | 0xB081A85245DEA5B7 | SELECT a.ip_id as ip_id, | 2025902778 | 1206116187539 |
    | 0x38BE36BDFFDBE638 | SELECT instance.name as  | 59343662   | 1096236803754 |
    | 0xB4233552504E43B8 | SELECT ir.type as type,  | 1362897166 | 488971769571  |
    | 0x4A131A16DCFFD6C6 | SELECT i.id as id, i.sta | 934402293  | 475253770301  |
    +--------------------+--------------------------+------------+---------------+
    5 rows in set (0.01 sec)
    
        再找出执行次数排在前5的查询:
    
    Admin> SELECT digest,SUBSTR(digest_text,0,25),count_star,sum_time FROM stats_mysql_query_digest WHERE digest_text LIKE 'SELECT%' ORDER BY count_star DESC LIMIT 5;
    +--------------------+--------------------------+------------+---------------+
    | digest             | SUBSTR(digest_text,0,25) | count_star | sum_time      |
    +--------------------+--------------------------+------------+---------------+
    | 0x037C3E6D996DAFE2 | SELECT a.ip_id as ip_id, | 2030040688 | 1479092529369 |
    | 0xB081A85245DEA5B7 | SELECT a.ip_id as ip_id, | 2025916528 | 1206123010791 |
    | 0x22E0A5C585C53EAD | SELECT id as instanceid, | 1551361254 | 426419508609  |
    | 0x3DB4B9FA4B2CB36F | SELECT i.id as instancei | 1465274289 | 415565419867  |
    | 0xB4233552504E43B8 | SELECT ir.type as type,  | 1362906755 | 488974931108  |
    +--------------------+--------------------------+------------+---------------+
    5 rows in set (0.00 sec)
    
    这些查询可以被缓存吗?ProxySQL的查询缓存可以实现该功能。
    
        找出最长执行时间排在前5的查询(译注:每个查询都有最长执行时间max_time和最短执行时间min_time):
    
    Admin> SELECT digest,SUBSTR(digest_text,0,25),count_star,sum_time,sum_time/count_star avg_time, min_time, max_time FROM stats_mysql_query_digest WHERE digest_text LIKE 'SELECT%' ORDER BY max_time DESC LIMIT 5;
    +--------------------+--------------------------+------------+--------------+----------+----------+-----------+
    | digest             | SUBSTR(digest_text,0,25) | count_star | sum_time     | avg_time | min_time | max_time  |
    +--------------------+--------------------------+------------+--------------+----------+----------+-----------+
    | 0x36CE5295726DB5B4 | SELECT COUNT(*) as total | 146390     | 185951894994 | 1270249  | 445      | 237344243 |
    | 0xDA8C56B5644C0822 | SELECT COUNT(*) as total | 44130      | 24842335265  | 562935   | 494      | 231395575 |
    | 0x8C1B0405E1AAB9DB | SELECT COUNT(*) as total | 1194       | 1356742749   | 1136300  | 624      | 216677507 |
    | 0x6C03197B4A2C34BE | Select *, DateDiff(Date_ | 4796       | 748804483    | 156131   | 607      | 197881845 |
    | 0x1DEFCE9DEF3BDF87 | SELECT DISTINCT i.extid  | 592196     | 40209254260  | 67898    | 416      | 118055372 |
    +--------------------+--------------------------+------------+--------------+----------+----------+-----------+
    5 rows in set (0.01 sec)
    
    这个具体的结果表明,一些查询具有非常高的最大执行时间,而最短执行时间却非常小,并且平均执行速度也相当慢。
    例如,digest值为0x36CE5295726DB5B4的查询,平均执行时间为1.27秒,但最小执行时间才0.4毫秒,最大时间久到了237.34秒。对于这样的查询,也许有必要研究一下为什么执行时间不均匀。
    
        找出总执行时间排在前5,且最短执行时间至少为1毫秒的查询:
    
    Admin> SELECT digest,SUBSTR(digest_text,0,20),count_star,sum_time,sum_time/count_star avg_time, min_time, max_time FROM stats_mysql_query_digest WHERE digest_text LIKE 'SELECT%' AND min_time > 1000 ORDER BY sum_time DESC LIMIT 5;
    +--------------------+--------------------------+------------+-------------+----------+----------+----------+
    | digest             | SUBSTR(digest_text,0,20) | count_star | sum_time    | avg_time | min_time | max_time |
    +--------------------+--------------------------+------------+-------------+----------+----------+----------+
    | 0x9EED412C6E63E477 | SELECT a.id as acco      | 961733     | 24115349801 | 25074    | 10994    | 7046628  |
    | 0x8DDD43A9EA37750D | Select ( Coalesce((      | 107069     | 3156179256  | 29477    | 1069     | 24600674 |
    | 0x9EED412C6E63E477 | SELECT a.id as acco      | 91996      | 1883354396  | 20472    | 10095    | 497877   |
    | 0x08B23A268C35C08E | SELECT id as reward      | 49401      | 244088592   | 4940     | 1237     | 1483791  |
    | 0x437C846F935344F8 | SELECT Distinct i.e      | 164        | 163873101   | 999226   | 1383     | 7905811  |
    +--------------------+--------------------------+------------+-------------+----------+----------+----------+
    5 rows in set (0.01 sec)
    
        找出总执行时间排在前5,且平均执行时间至少为1秒的查询。同时输出每个查询的总时间占所有查询总时间的百分比:
    
    Admin> SELECT digest,SUBSTR(digest_text,0,25),count_star,sum_time,sum_time/count_star avg_time, ROUND(sum_time*100.00/(SELECT SUM(sum_time) FROM stats_mysql_query_digest),3) pct FROM stats_mysql_query_digest WHERE digest_text LIKE 'SELECT%' AND sum_time/count_star > 1000000 ORDER BY sum_time DESC LIMIT 5;
    +--------------------+--------------------------+------------+--------------+----------+-------+
    | digest             | SUBSTR(digest_text,0,25) | count_star | sum_time     | avg_time | pct   |
    +--------------------+--------------------------+------------+--------------+----------+-------+
    | 0x36CE5295726DB5B4 | SELECT COUNT(*) as total | 146390     | 185951894994 | 1270249  | 2.11  |
    | 0xD38895B4F4D2A4B3 | SELECT instance.name as  | 9783       | 12409642528  | 1268490  | 0.141 |
    | 0x8C1B0405E1AAB9DB | SELECT COUNT(*) as total | 1194       | 1356742749   | 1136300  | 0.015 |
    +--------------------+--------------------------+------------+--------------+----------+-------+
    3 rows in set (0.00 sec)
    
        找出总执行时间排在前5,且平均执行时间至少为15毫秒的查询。同时输出每个查询的总时间占所有查询总时间的百分比:
    
    Admin> SELECT digest,SUBSTR(digest_text,0,25),count_star,sum_time,sum_time/count_star avg_time, ROUND(sum_time*100.00/(SELECT SUM(sum_time) FROM stats_mysql_query_digest WHERE digest_text LIKE 'SELECT%'),3) pct FROM stats_mysql_query_digest WHERE digest_text LIKE 'SELECT%' AND sum_time/count_star > 15000 ORDER BY sum_time DESC LIMIT 5;
    +--------------------+--------------------------+------------+---------------+----------+--------+
    | digest             | SUBSTR(digest_text,0,25) | count_star | sum_time      | avg_time | pct    |
    +--------------------+--------------------------+------------+---------------+----------+--------+
    | 0x38BE36BDFFDBE638 | SELECT instance.name as  | 59360371   | 1096562204931 | 18472    | 13.006 |
    | 0x36CE5295726DB5B4 | SELECT COUNT(*) as total | 146390     | 185951894994  | 1270249  | 2.205  |
    | 0x1DEFCE9DEF3BDF87 | SELECT DISTINCT i.extid  | 592281     | 40215136635   | 67898    | 0.477  |
    | 0xDA8C56B5644C0822 | SELECT COUNT(*) as total | 44130      | 24842335265   | 562935   | 0.295  |
    | 0x9EED412C6E63E477 | SELECT a.id as accountid | 961768     | 24116011513   | 25074    | 0.286  |
    +--------------------+--------------------------+------------+---------------+----------+--------+
    5 rows in set (0.00 sec)
    
    难道所有这些查询都应用路由到master上去执行吗?如果一个查询的平均执行时间在1秒以上,那么答案可能是no。对于某些非常繁忙且对性能要求较高的应用程序,甚至可以考虑将平均执行时间在15毫秒的查询路由给slave。
    例如,我们可以决定将digest=0x38BE36BDFFDBE638的查询路由给slave:
    INSERT INTO mysql_query_rules (rule_id,active,digest,destination_hostgroup,apply)
    VALUES(1,1,'0x38BE36BDFFDBE638',20,1);
    
    类似地,在检查了如下语句的输出后:
    SELECT digest,digest_text,count_star,sum_time,sum_time/count_star avg_time, ROUND(sum_time*100.00/(SELECT SUM(sum_time) FROM stats_mysql_query_digest WHERE digest_text LIKE 'SELECT%'),3) pct FROM stats_mysql_query_digest WHERE digest_text LIKE 'SELECT COUNT%' ORDER BY sum_time DESC;
    
    我们决定将所有以SELECT COUNT(*)开头的语句路由给slave:
    INSERT INTO mysql_query_rules (rule_id,active,match_digest,destination_hostgroup,apply)
    VALUES(1,1,'^SELECT COUNT\(\*\)',20,1);
    
    最后,将配置加载到RUNTIME,并将它持久化保存到磁盘数据库中。
    LOAD MYSQL QUERY RULES TO RUNTIME;
    SAVE MYSQL QUERY RULES TO DISK; 
    Conclusion
    
    (结论:)
    ProxySQL可以非常有效、非常弹性地对各种查询进行选择性地路由。
    虽然对于某些应用程序来说,将所有SELECT语句路由给slaves/readers,其余都路由给writers/masters是可以接受的,但对于很多应用程序来说,其实并没这么简单。作为DBA,应当具备"使用复杂的规则配置ProxySQL,只将那些不应该路由给master的查询路由给slaves,而不需要更改任何应用程序"的能力。
    
    
  • 相关阅读:
    3--jsp编译指令和动作指令
    2-hello world
    1.搭建JSP开发环境
    4-Http通信
    P1879 [USACO06NOV]玉米田Corn Fields
    P3110 [USACO14DEC]驮运Piggy Back
    P2327 [SCOI2005]扫雷
    P1004 方格取数
    P2854 [USACO06DEC]牛的过山车Cow Roller Coaster
    P1353 [USACO08JAN]跑步Running
  • 原文地址:https://www.cnblogs.com/sanduzxcvbnm/p/16396969.html
Copyright © 2020-2023  润新知