• mysql慢查询日志按天切割归纳


    问题描述

    mysql开启慢查询功能,再正常不过,那么存在这样一种情况:慢查询写入的文件位置和文件名是指定好的,如果慢查询时间设定严苛,不出意外,记录慢查询的单个文件大小会日益增大,几十兆或者上百兆,带来的问题也就是慢查询日志分析处理不够方便。

    解决思路

    切割日志,再通过定时任务(crontab)按规定时间切分,一般按“天”切分。

    测试环境

    mysql5.6和mysql5.7

    实现方法

    方法一通过mysqladmin flush-logs [log_type] (注:mysql5.6无法指定日志类型,见测试过程)

    方法二通定时任务每次去设定慢查询路径实现

    说明:以下测试过程是方法一的测试,最终实现脚本见页底。

    测试过程

    1. 截取mysql5.7官方文档中的mysqladm部分说明

    flush-logs [log_type ...]
    
    Flush all logs.
    
    The mysqladmin flush-logs command permits optional log types to be given, to specify which logs to flush. Following the flush-logs command, you can provide a space-separated list of one or more of the following log types: binary, engine, error, general, relay, slow. These correspond to the log types that can be specified for the FLUSH LOGS SQL statement.

     注意“[log_type ...]” 这个附加参数仅适用于mysql5.7.x或以上版本,也就是说,mysql5.6可以使用mysqladm flush-logs命令,但不能指定刷新某种指定日志类型,这很重要,因为这样意味着mysql5.6执行该命令,就刷新了所有类型的日志,所以mysql5.6根据具体情况选择使用。

     2. 确定一下mysql版本和对应的数据存放目录(开起了binLog和慢查询)

    说明:以下只显示二进制日志和慢查询日志文件详情,留意文件时间变化

    [root@server-10 ~]# mysql -V
    mysql  Ver 14.14 Distrib 5.7.23, for Linux (x86_64) using  EditLine wrapper
    [root@server-10 ~]# cd /var/lib/mysql
    [root@server-10 mysql]# touch *
    [root@server-10 mysql]# ls -l |grep -E  "master-bin|server-10-slow.log"
    -rw-r----- 1 mysql mysql      177 Sep 15 10:53 master-bin.000001
    -rw-r----- 1 mysql mysql      421 Sep 15 10:53 master-bin.000002
    -rw-r----- 1 mysql mysql      885 Sep 15 10:53 master-bin.000003
    -rw-r----- 1 mysql mysql      634 Sep 15 10:53 master-bin.000004
    -rw-r----- 1 mysql mysql      202 Sep 15 10:53 master-bin.000005
    -rw-r----- 1 mysql mysql      202 Sep 15 10:53 master-bin.000006
    -rw-r----- 1 mysql mysql      154 Sep 15 10:53 master-bin.000007
    -rw-r----- 1 mysql mysql      140 Sep 15 10:53 master-bin.index
    -rw-r----- 1 mysql mysql     1473 Sep 15 10:53 server-10-slow.log

    3. 【测试】刷新二进制文件

    [root@server-10 mysql]# mysqladmin -uroot -p flush-logs binary
    Enter password: 
    [root@server-10 mysql]# ls -l |grep -E  "master-bin|server-10-slow.log"
    -rw-r----- 1 mysql mysql      177 Sep 15 10:53 master-bin.000001
    -rw-r----- 1 mysql mysql      421 Sep 15 10:53 master-bin.000002
    -rw-r----- 1 mysql mysql      885 Sep 15 10:53 master-bin.000003
    -rw-r----- 1 mysql mysql      634 Sep 15 10:53 master-bin.000004
    -rw-r----- 1 mysql mysql      202 Sep 15 10:53 master-bin.000005
    -rw-r----- 1 mysql mysql      202 Sep 15 10:53 master-bin.000006
    -rw-r----- 1 mysql mysql      202 Sep 15 10:57 master-bin.000007
    -rw-r----- 1 mysql mysql      154 Sep 15 10:57 master-bin.000008    //二进制日志已经更新为000008
    -rw-r----- 1 mysql mysql      160 Sep 15 10:57 master-bin.index    
    -rw-r----- 1 mysql mysql     1473 Sep 15 10:53 server-10-slow.log  //留意时间未变化,只指定了binary,故慢查询日志没有更新

    4. 刷新慢查询日志文件

    [root@server-10 mysql]# mysqladmin -uroot -p flush-logs slow
    Enter password: 
    [root@server-10 mysql]# ls -l |grep -E  "master-bin|server-10-slow.log"
    -rw-r----- 1 mysql mysql      177 Sep 15 10:53 master-bin.000001
    -rw-r----- 1 mysql mysql      421 Sep 15 10:53 master-bin.000002
    -rw-r----- 1 mysql mysql      885 Sep 15 10:53 master-bin.000003
    -rw-r----- 1 mysql mysql      634 Sep 15 10:53 master-bin.000004
    -rw-r----- 1 mysql mysql      202 Sep 15 10:53 master-bin.000005
    -rw-r----- 1 mysql mysql      202 Sep 15 10:53 master-bin.000006
    -rw-r----- 1 mysql mysql      202 Sep 15 10:57 master-bin.000007
    -rw-r----- 1 mysql mysql      298 Sep 15 11:04 master-bin.000008
    -rw-r----- 1 mysql mysql      160 Sep 15 10:57 master-bin.index
    -rw-r----- 1 mysql mysql     1656 Sep 15 11:04 server-10-slow.log  //慢查询文件刷新了(文件描述符已经改变),但这里不会生成多个文件

    5. 归纳旧慢查询日志,生成新慢查询日志文件

    [root@server-10 mysql]# mv server-10-slow.log server-10-slow.log.old
    [root@server-10 mysql]# ls -l |grep -E  "master-bin|server-10-slow.log"
    -rw-r----- 1 mysql mysql      177 Sep 15 10:53 master-bin.000001
    -rw-r----- 1 mysql mysql      421 Sep 15 10:53 master-bin.000002
    -rw-r----- 1 mysql mysql      885 Sep 15 10:53 master-bin.000003
    -rw-r----- 1 mysql mysql      634 Sep 15 10:53 master-bin.000004
    -rw-r----- 1 mysql mysql      202 Sep 15 10:53 master-bin.000005
    -rw-r----- 1 mysql mysql      202 Sep 15 10:53 master-bin.000006
    -rw-r----- 1 mysql mysql      202 Sep 15 10:57 master-bin.000007
    -rw-r----- 1 mysql mysql      298 Sep 15 11:04 master-bin.000008
    -rw-r----- 1 mysql mysql      160 Sep 15 10:57 master-bin.index
    -rw-r----- 1 mysql mysql     1656 Sep 15 11:04 server-10-slow.log.old
    [root@server-10 mysql]# mysqladmin -uroot -p flush-logs slow
    Enter password: 
    [root@server-10 mysql]# ls -l |grep -E  "master-bin|server-10-slow.log"
    -rw-r----- 1 mysql mysql      177 Sep 15 10:53 master-bin.000001
    -rw-r----- 1 mysql mysql      421 Sep 15 10:53 master-bin.000002
    -rw-r----- 1 mysql mysql      885 Sep 15 10:53 master-bin.000003
    -rw-r----- 1 mysql mysql      634 Sep 15 10:53 master-bin.000004
    -rw-r----- 1 mysql mysql      202 Sep 15 10:53 master-bin.000005
    -rw-r----- 1 mysql mysql      202 Sep 15 10:53 master-bin.000006
    -rw-r----- 1 mysql mysql      202 Sep 15 10:57 master-bin.000007
    -rw-r----- 1 mysql mysql      442 Sep 15 11:18 master-bin.000008
    -rw-r----- 1 mysql mysql      160 Sep 15 10:57 master-bin.index
    -rw-r----- 1 mysql mysql      183 Sep 15 11:18 server-10-slow.log    //新生成的慢查询日志,新的慢查询会写入该文件中
    -rw-r----- 1 mysql mysql     1656 Sep 15 11:04 server-10-slow.log.old
    [root@server-10 mysql]# date
    Sat Sep 15 11:18:20 CST 2018

    注意:不能单纯地仅重命名,上面重命名成了server-10-slow.log.old,如果不刷新,那么新的慢查询语句还会写入server-10-slow.log.old中,linux下存在“文件描述符”这么一个概念,改名不能修改文件描述符。

    解决方法

    回到主题:按天切分归纳慢查询日志

     方法一通过mysqladmin flush-logs [log_type] (注:mysql5.6无法指定日志类型,见上文)

    新建脚本:cut_slow_log.sh

    #!/bin/bash
    time=`date -d yesterday +"%Y-%m-%d"` user="root" passwd="xxxxxx" #提前创建好一个存放目录:/var/lib/mysql/slow_log/ mv /var/lib/mysql/server-10-slow.log /var/lib/mysql/slow_log/server-10-slow-$time.log mysqladmin -u$user -p$passwd --socket=/var/lib/mysql/mysql.sock flush-logs slow

    每天0点执行该脚本

    0    0    *    *    *    sh /root/cut_slow_log.sh > /dev/null 2>&1

    方法二通定时任务每次去设定慢查询路径实现

    新建脚本:cut_slow_log.sh

    #!/bin/bash
    
    time=`date -d yesterday +"%Y-%m-%d"`
    host="127.0.0.1"
    user="root"
    passwd="xxxxxx"
    #提前创建好一个存放目录:/var/lib/mysql/slow_log/
    mv /var/lib/mysql/server-10-slow.log /var/lib/mysql/slow_log/server-10-slow-$time.log
    mysql -u$user -p$passwd -e "set global slow_query_log_file='/var/lib/mysql/server-10-slow.log';"

    每天0点执行该脚本

    0    0    *    *    *    sh /root/cut_slow_log.sh > /dev/null 2>&1

    结束.

  • 相关阅读:
    Educational Codeforces Round 20 D. Magazine Ad
    Educational Codeforces Round 20 C. Maximal GCD
    紫书第三章训练2 暴力集
    Educational Codeforces Round 20 B. Distances to Zero
    Educational Codeforces Round 20 A. Maximal Binary Matrix
    紫书第三章训练1 D
    紫书第一章训练1 D -Message Decoding
    HAZU校赛 Problem K: Deadline
    Mutual Training for Wannafly Union #8 D
    紫书第三章训练1 E
  • 原文地址:https://www.cnblogs.com/ding2016/p/9649958.html
Copyright © 2020-2023  润新知