• MySQL Flashback 工具介绍


    MySQL Flashback 工具介绍

    • DML Flashback

      • 独立工具,通过伪装成slave拉取binlog来进行处理
      • patch形式集成到官方工具mysqlbinlog
      • 简单脚本。先用mysqlbinlog解析出文本格式的binlog,再根据回滚原理用正则进行匹配并替换
    • DDL Flashback

      DDL Falshback 要求离线,比 DML Flashback 要求更多,生产环境数据可能覆盖导致恢复不全

      因为 5.6 开始,有单独的 purge 线程,数据恢复可能性更低了

    部署位置

    • IP1

      /data1/dbatools/mysqlflashback

      • flashback_57 「mysqlbinlog」(因为系统原因,mysqlbinlog 放在 CentOS 7机器上)
      • binlog2sql 「建议使用」
      • MyFlash「建议使用」(测试暂时还不能在 CentOS 7上使用)
    • IP2

      /data1/dbatools/mysqlflashback

      • binlog2sql 「建议使用」
      • MyFlash「建议使用」

    MyFlash

    MyFlash——美团点评的开源MySQL闪回工具

    github:MyFlash

    MyFlash是由美团点评公司技术工程部开发维护的一个回滚DML操作的工具。该工具通过解析v4版本的binlog,完成回滚操作。相对已有的回滚工具,其增加了更多的过滤选项,让回滚更加容易。

    版本兼容

    • MySQL 5.6、5.7

    限制

    • binlog格式必须为row,且binlog_row_image=full
    • 目前只支持单个 binlog 文件解析
    • 只能回滚DML(增、删、改)

    依赖

    • glibc 版本

    编译

    git clone https://github.com/Meituan-Dianping/MyFlash.git
    cd MyFlash
    gcc -w  `pkg-config --cflags --libs glib-2.0` source/binlogParseGlib.c  -o binary/flashback
    

    使用

    基本用法

    cd binary
    
    # 回滚整个文件 && 应用
    ./flashback --binlogFileNames=mysql-bin.000005
    mysqlbinlog binlog_output_base.flashback | mysql -h<host> -u<user> -p
    
    # 回滚文件中 INSERT && 应用
    ./flashback --sqlTypes='INSERT'--binlogFileNames=mysql-bin.000005
    mysqlbinlog binlog_output_base.flashback | mysql -h<host> -u<user> -p
    
    # 回滚大文件 && 应用 (大于20M建议分割回滚文件)
    ./flashback --binlogFileNames=mysql-bin.000005
    ./flashback --maxSplitSize=1 --binlogFileNames=binlog_output_base.flashback
    
    mysqlbinlog binlog_output_base.flashback.000001 | mysql -h<host> -u<user> -p
    ...
    mysqlbinlog binlog_output_base.flashback.<N> | mysql -h<host> -u<user> -p
    
    • flashback 回滚的文件,还是mysqlbinlog格式的,如果想看 sql,还是需要原生 mysqlbinlog 进行解析

      /usr/local/mysql-5.7.18/bin/mysqlbinlog binlog_output_base.flashback

    • 开启 gtid MySQL 应用flashback报错

    ERROR 1782 (HY000) at line 16: @@SESSION.GTID_NEXT cannot be set to ANONYMOUS when @@GLOBAL.GTID_MODE = ON.

    在导入时加入--skip-gtids mysqlbinlog --skip-gtids | mysql -uxxx -pxxx

    选项

    ./flashback --help
    Usage:
      flashback [OPTION...]
    
    Help Options:
      -?, --help                  Show help options
    
    Application Options:
      --databaseNames             databaseName to apply. if multiple, seperate by comma(,)
      --tableNames                tableName to apply. if multiple, seperate by comma(,)
      --start-position            start position
      --stop-position             stop position
      --start-datetime            start time (format %Y-%m-%d %H:%M:%S)
      --stop-datetime             stop time (format %Y-%m-%d %H:%M:%S)
      --sqlTypes                  sql type to filter . support INSERT, UPDATE ,DELETE. if multiple, seperate by comma(,)
      --maxSplitSize              max file size after split, the uint is M
      --binlogFileNames           binlog files to process. if multiple, seperate by comma(,)  
      --outBinlogFileNameBase     output binlog file name base
      --logLevel                  log level, available option is debug,warning,error
      --include-gtids             gtids to process
      --exclude-gtids             gtids to skip
    
     参数解释
    1 databaseNames 指定需要回滚的数据库名。多个用“,”隔开。不指定该参数,相当于指定了所有数据库
    2 tableNames 指定需要回滚的表名。多个表可以用“,”隔开。如果不指定该参数,相当于指定了所有表
    3 start-position 指定回滚开始的位置。如不指定,从文件的开始处回滚。请指定正确的有效的位置,否则无法回滚
    4 stop-position 指定回滚结束的位置。如不指定,回滚到文件结尾。请指定正确的有效的位置,否则无法回滚
    5 start-datetime 指定回滚的开始时间。格式必须是 %Y-%m-%d %H:%M:%S。 如不指定,则不限定时间
    6 stop-datetime 定回滚的结束时间。格式必须是 %Y-%m-%d %H:%M:%S。 如不指定,则不限定时间
    7 sqlTypes 指定回滚的sql类型。支持的类型是INSERT, UPDATE ,DELETE。多个类型可以用“,”隔开。
    8 maxSplitSize 一旦指定该参数,对文件进行固定尺寸的分割(单位为M),过滤条件有效,但不进行回滚操作。该参数主要用来将大的binlog文件切割,防止单次应用的binlog尺寸过大,对线上造成压力
    9 binlogFileNames 需要回滚的binlog文件,目前只支持单个文件,后续会增加多个文件支持
    10 outBinlogFileNameBase 输出的binlog文件前缀,如不指定,则默认为binlog_output_base.flashback
    11 logLevel 仅供开发者使用,默认级别为error级别。在生产环境中不要修改这个级别,否则输出过多
    12 include-gtids 指定需要回滚的gtid,支持gtid的单个和范围两种形式。
    13 exclude-gtids 指定不需要回滚的gtid,用法同include-gtids

    binlog2sql

    伪装成slave拉取binlog,从MySQL binlog解析出你要的SQL。根据不同选项,你可以得到原始SQL、回滚SQL、去除主键的INSERT SQL等,建议表有主键。

    github: binlog2sql

    MySQL闪回原理与实战

    用途

    • 数据快速回滚(闪回)

    • 主从切换后新master丢数据的修复

    • 从binlog生成标准SQL,带来的衍生功能

    版本兼容

    • Python 2.6、2.7
    • MySQL 5.6 (简单测试在 5.5、5.7 gtid 均可用,不排除无 Bug,建议 5.6 以后版本使用)

    依赖

    限制 (对比mysqlbinlog)

    • mysql server必须开启,离线模式下不能解析

    • 参数 binlog_format 必须为ROW,参数 binlog_row_image 必须为FULL,暂不支持MINIMAL (5.6.2引入)

    • GEOMETRY 不能解析 (因为python-mysql-replication Limitations)

    • 解析速度不如 mysqlbinlog

    • 表有关联表,关联表并不会被回滚,需与业务方沟通清楚

    • 不支持DDL

    • 建议回滚的表有主键

    优点 (对比mysqlbinlog)

    • 纯Python开发,安装与使用都很简单
    • 自带flashback、no-primary-key解析模式,无需再装补丁
    • flashback模式下,更适合闪回实战
    • 解析为标准SQL,方便理解、调试
    • 代码容易改造,可以支持更多个性化解析

    安装

    git clone https://github.com/danfengcao/binlog2sql.git && cd binlog2sql
    pip install -r requirements.txt
    

    升级依赖模块

    # cat requirements.txt 
    PyMySQL==0.7.11
    wheel==0.29.0
    mysql-replication==0.13
    
    wget https://codeload.github.com/noplay/python-mysql-replication/tar.gz/0.15
    tar zxf 0.15
    cd python-mysql-replication-0.15
    python2.7 setup.py install
    

    使用

    MySQL server必须设置以下参数

    [mysqld]
    server_id = 1
    log_bin = /var/log/mysql/mysql-bin.log
    max_binlog_size = 1G
    binlog_format = row
    binlog_row_image = full
    

    user需要的最小权限集合

    select, super/replication client, replication slave
    
    建议授权
    GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 
    

    权限说明

    • select:需要读取server端information_schema.COLUMNS表,获取表结构的元信息,拼接成可视化的sql
    • super/replication client:两个权限都可以,需要执行'SHOW MASTER STATUS', 获取server端的binlog列表
    • replication slave:通过BINLOG_DUMP协议获取binlog内容的权限

    基本用法

    解析标准 SQL (<u>会有 DDL SQL,回滚模式是没有 DDL的</u>)

    python2.7 binlog2sql/binlog2sql.py -h127.0.0.1 -P6000 -umyadmin_common -p'password' -dtest -tt1 --start-file='mysql-bin.000192'
    
    INSERT INTO `test`.`t1`(`id`, `name`) VALUES (5, 'name5'); #start 4 end 259 time 2017-09-14 19:59:31
    DELETE FROM `test`.`t1` WHERE `id`=4 AND `name`='name4' LIMIT 1; #start 286 end 438 time 2017-09-14 19:59:42
    UPDATE `test`.`t1` SET `id`=3, `name`='name300' WHERE `id`=3 AND `name`='name3' LIMIT 1; #start 465 end 631 time 2017-09-14 19:59:54
    

    解析回滚SQL (在上面命令基础上加上 --flashback 或 -B)

    python2.7 binlog2sql/binlog2sql.py -h127.0.0.1 -P6000 -umyadmin_common -p'ZTlh916@mxq' -dtest -tt1 --start-file='mysql-bin.000192' -B
    
    UPDATE `test`.`t1` SET `id`=3, `name`='name3' WHERE `id`=3 AND `name`='name300' LIMIT 1; #start 465 end 631 time 2017-09-14 19:59:54
    INSERT INTO `test`.`t1`(`id`, `name`) VALUES (4, 'name4'); #start 286 end 438 time 2017-09-14 19:59:42
    DELETE FROM `test`.`t1` WHERE `id`=5 AND `name`='name5' LIMIT 1; #start 4 end 259 time 2017-09-14 19:59:31
    

    选项

    mysql连接配置

    -h host; -P port; -u user; -p password

    解析模式

    --stop-never 持续同步binlog。可选。不加则同步至执行命令时最新的binlog位置。

    -K, --no-primary-key 对INSERT语句去除主键。可选。

    -B, --flashback 生成回滚语句,可解析大文件,不受内存限制,每打印一千行加一句SLEEP SELECT(1)。可选。与stop-never或no-primary-key不能同时添加。

    解析范围控制

    --start-file 起始解析文件。必须。

    --start-position/--start-pos start-file的起始解析位置。可选。默认为start-file的起始位置。

    --stop-file/--end-file 末尾解析文件。可选。默认为start-file同一个文件。若解析模式为stop-never,此选项失效。

    --stop-position/--end-pos stop-file的末尾解析位置。可选。默认为stop-file的最末位置;若解析模式为stop-never,此选项失效。

    --start-datetime 从哪个时间点的binlog开始解析,格式必须为datetime,如'2016-11-11 11:11:11'。可选。默认不过滤。

    --stop-datetime 到哪个时间点的binlog停止解析,格式必须为datetime,如'2016-11-11 11:11:11'。可选。默认不过滤。

    对象过滤

    -d, --databases 只输出目标db的sql。可选。默认为空。

    -t, --tables 只输出目标tables的sql。可选。默认为空。

    恢复数据

    误操作大概时间范围

    python binlog2sql/binlog2sql.py -h127.0.0.1 -P3306 -uadmin -p'admin' -dtest -ttbl --start-file='mysql-bin.000052' --start-datetime='2016-12-13 20:25:00' --stop-datetime='2016-12-13 20:30:00'
    

    找到具体位点进行恢复

    python binlog2sql/binlog2sql.py -h127.0.0.1 -P3306 -uadmin -p'admin' -dtest -ttbl --start-file='mysql-bin.000052' --start-position=3346 --stop-position=3556 -B > rollback.sql | cat
    

    <u>回滚前,一定找业务确认回滚数据</u>

    TIPS

    • 闪回的目标:快速筛选出真正需要回滚的数据。

    • 先根据库、表、时间做一次过滤,再根据位置做更准确的过滤。

    • 由于数据一直在写入,要确保回滚sql中不包含其他数据。可根据是否是同一事务、误操作行数、字段值的特征等等来帮助判断。

    • 执行回滚sql时如有报错,需要查实具体原因,一般是因为对应的数据已发生变化。由于是严格的行模式,只要有唯一键(包括主键)存在,就只会报某条数据不存在的错,不必担心会更新不该操作的数据。业务如果有特殊逻辑,数据回滚可能会带来影响。

    • 如果只回滚某张表,并且该表有关联表,关联表并不会被回滚,需与业务方沟通清楚。

    mysqlbinlog_flashback

    跟 binlog2sql 一样,伪装成slave拉取binlog,从MySQL binlog解析出你要的SQL,目前已经稳定运行在阿里RDS、db字符集为 utf8 的产生环境,<u>其他环境要谨慎使用</u>

    github: mysqlbinlog_flashback

    版本兼容

    • 阿里RDS
    • 默认 utf8 DB (gbk等字符集的 DB 理论上支持)

    限制

    • mysql server必须开启,离线模式下不能解析
    • 参数 binlog_format 必须为ROW,参数 binlog_row_image 必须为FULL,暂不支持MINIMAL (5.6.2引入)
    • GEOMETRY 不能解析 (因为python-mysql-replication Limitations)
    • 解析速度不如 mysqlbinlog
    • 表有关联表,关联表并不会被回滚,需与业务方沟通清
    • 不支持DDL
    • <u>表必须有主键</u> (只限制在 mysqlbinlog_flashback,虽然 binlog2sql 不强制,但建议表有主键)
    • <u>阿里 RDS、db 字符集为 utf8 的环境稳定使用,其他环境要谨慎使用</u> (只限制在 mysqlbinlog_flashback)

    • <u>修改 python-mysql-replication</u> (只限制在 mysqlbinlog_flashback)

      • 某些数据被封装,没有暴露出来
      • Ali-rds导致的bug
    • <u>字段类型限制</u> (只限制在 mysqlbinlog_flashback)

      ALLOW_TYPE={ "varchar":True, "char":True, "datetime":True, "date":True, "time":True, "timestamp":True, "bigint":True, "mediumint":True, "smallint":True, "tinyint":True, "int":True, "smallint":True, "decimal":True, "float":True, "double":True, "longtext":True, "tinytext":True, "text":True, "mediumtext":True }

    • mysql 导入一定要指定 字符集 --default-character-set=utf8

    安装

    git clone https://github.com/58daojia-dba/mysqlbinlog_flashback.git
    cd mysqlbinlog_flashback
    

    使用

    MySQL server必须设置以下参数

    [mysqld]
    server_id = 1
    log_bin = /var/log/mysql/mysql-bin.log
    max_binlog_size = 1G
    binlog_format = row
    binlog_row_image = full
    

    基本用法

    python2.7 mysqlbinlog_back.py --host="127.0.0.1" --port=7000 --user="mysql_user" --password='password' --schema=test --table=t2
    
    ls -l log/*
    -rw-r--r-- 1 root root 1.9K Sep 19 17:40 save_data_dml_test_20170919_174001.sql
    -rw-r--r-- 1 root root  179 Sep 19 17:40 save_data_create_table_test_20170919_174001.sql
    -rw-r--r-- 1 root root 1.1K Sep 19 17:40 flashback_test_20170919_174001.sql <--- 反向sq文件
    
    mysql -uroot -p -h -P --default-character-set=utf8 < flashback_test_20170919_174001.sql
    

    选项

    -s SCHEMA, --schema=SCHEMA 必须的,指定 DB

    -t TABLES, --tables=TABLES 必须的,指定 Table,用逗号分隔

    -N BINLOG_END_TIME, --binlog_end_time=BINLOG_END_TIME

    format yyyy-mm-dd hh24:mi:ss,default is current time

    -S BINLOG_START_FILE_NAME, --binlog_start_file_name=BINLOG_START_FILE_NAME

    binlog start file name,default is current logfile of db

    -L BINLOG_START_FILE_POSITION, --binlog_start_file_position=BINLOG_START_FILE_POSITION

    binlog start file name

    -E BINLOG_START_TIME, --binlog_start_time=BINLOG_START_TIME

    binlog start time,format yyyy-mm-dd hh24:mi:ss

    -l OUTPUT_FILE_PATH, --output_file_path=OUTPUT_FILE_PATH

    file path that sql generated,,default ./log

    -I, --skip_insert skip insert(WriteRowsEvent) event
    -U, --skip_update skip update(UpdateRowsEvent) event
    -D, --skip_delete skip delete(DeleteRowsEvent) event

    -a, --add_schema_name add schema name for flashback sql

    mysqlbinlog

    以patch形式集成到官方工具mysqlbinlog中。以彭立勋提交的patch为代表 MySQL下实现闪回的设计思路 (MySQL Flashback Feature)

    mysqlbinlog 闪回工具

    优点

    • 上手成本低,原 mysqlbinlog 选项可以直接利用,只是多一个闪回选项

    • 支持离线解析

    • 支持事务闪回,语句逆转、顺序也颠倒

    缺点

    • 兼容性差、项目活跃度不高,mysql5.5,5.6,5.7 分别开发了patch
    • 难以添加新功能,实战效果欠佳,例如过滤一张表 (姜承尧版本也是支持的)
    • 安装稍显麻烦。需要对mysql源码打补丁再编译生成

    限制

    • 参数 binlog_format 必须为ROW,参数 binlog_row_image 必须为FULL,暂不支持MINIMAL

    • 不支持DDL

    使用

    MySQL server必须设置以下参数

    [mysqld]
    server_id = 1
    log_bin = /var/log/mysql/mysql-bin.log
    max_binlog_size = 1G
    binlog_format = row
    binlog_row_image = full
    

    基本用法

    解析 mysqlbinlog

    ./mysqlbinlog -vv  mysql-bin.000067 --start-datetime="2017-09-19 15:00:00" --stop-datetime="2017-09-19 15:38:10"
    

    回滚

    ./mysqlbinlog -vv  mysql-bin.000067 --start-position=1427 --stop-position=2588 -B | mysql -umysqlha_common -p'password' -h10.13.2.29 -P 7000 test
    

    position 位点找需要回滚事务 BEGIN 前面那个 position.

    选项

    -B, --flashback Flashback data to start_postition or start_datetime.

    -A, --skip-database=name 解析BinLog时过滤掉该数据库 (local log only)

    -a, --skip-table=name 解析BinLog时过滤掉该表,一般与skip_datebase配套使用 (local log only)

    --skip-database 可以单独使用,--skip-table需要与 skip-database配置使用,<u>多个db或table不生效</u>

    --skip-database=test --skip-table=t1

    -O, --split-size-interval=# 将BinLog文件按照指定的大小拆分为多个段,解析结果为打印每个段的起始offset位置

    注意,当进行flashback时,flashback的内容先保存在内存中。若你的binlog大小为10G,那么需要额外的10G内存先暂时保存这部分信息。在某些情况下,如云环境、或服务器内存较小,会导致无法输出flashback的日志。这时可以通过此参数来设置内存保存文件的大小,例如将此值设置为100M,那么每100M就会刷新到一个文件

    -D, --datetime-to-pos=name 基于输入的时间信息,解析出该时间对应的第一个BinLog event偏移位置,格式参照start-datetime

    flashback时要先找到起始的偏移量,DBA可以先通过此参数定位到具体位置,然后再进行flashback操作

    「不是很好用啊,少了一个」

    -T, --table=name 仅解析该表,一般与database配套使用 (local log only)

    -E, --fb-event=name 仅解析该类型的Log event,一般与database、table选项配套使用。(local log only)

    可选的值有:

    • DELETE
    • INSERT
    • UPDATE

    DDL Flashbackup

    DDL Falshback 要求离线,比 DML Flashback 要求更多,生产环境数据可能覆盖导致恢复不全。

    MySQL 对于空间的重用机制与Oracle 有很大区别,对于Oracle 而言,如果是delete的数据,还是很难被覆盖掉的,对于drop 和truncate 则领导别论。然而MySQL则有所不同,MySQL 默认会启动一些purge 进程来进行空间重用,这是MySQL 5.6的情况

    localhost.(none)>show variables like '%innodb_purge_threads';
    +----------------------+-------+
    | Variable_name        | Value |
    +----------------------+-------+
    | innodb_purge_threads | 1     |
    +----------------------+-------+
    1 row in set (0.00 sec)
    

    在MySQL 5.7 版本中更为坑爹,MySQL 默认会启动4个purge 线程,因此很容易就会导致空间被重用,最终导致数据无法恢复,如下是MySQL 5.7的purge相关参数

    localhost.(none)>show variables like '%innodb_purge_threads';
    +----------------------+-------+
    | Variable_name        | Value |
    +----------------------+-------+
    | innodb_purge_threads | 4     |
    +----------------------+-------+
    1 row in set (0.00 sec)
    

    Percona Data Recovery Tool for InnoDB

    code wiki

    限制

    • 只对InnoDB/XtraDB表有效,而无法恢复MyISAM表

    • 离线恢复,MySQL Server 是关闭的

    • 不能保证数据总一定可被恢复。

      被重写的数据不能被恢复,这种情况下可能需要针对系统或物理的方式来恢复,不属于本工具的范畴。

    • 恢复的最好时机是当你发现数据丢失时,尽快备份MySQL数据文件

    • innodb_file_per_table = 0

    安装

    现在可以下载还是 0.5,2011-08-28,0.6 还未发布

    yum install glibc glibc-static -y
    
    wget https://launchpad.net/percona-data-recovery-tool-for-innodb/trunk/release-0.5/+download/percona-data-recovery-tool-for-innodb-0.5.tar.gz
    
    tar -xvf percona-data-recovery-tool-for-innodb-0.5.tar.gz
    cd percona-data-recovery-tool-for-innodb-0/mysql-source/
    ./configure
    cd ..
    make
    
    # ll -rth
    total 3.0M
    -rw-r--r--  1  510 wheel 2.0K Aug 28  2011 tables_dict.c
    -rwxr-xr-x  1  510 wheel  302 Aug 28  2011 split_dump.pl
    -rw-r--r--  1  510 wheel  11K Aug 28  2011 print_data.c
    -rw-r--r--  1  510 wheel  15K Aug 28  2011 page_parser.c
    -rw-r--r--  1  510 wheel 2.7K Aug 28  2011 Makefile
    -rw-r--r--  1  510 wheel   74 Aug 28  2011 INSTALL
    -rw-r--r--  1  510 wheel 9.0K Aug 28  2011 innochecksum.c
    -rw-r--r--  1  510 wheel 8.1K Aug 28  2011 incrementalupdate.c
    -rw-r--r--  1  510 wheel  12K Aug 28  2011 ibdconnect.c
    -rwxr-xr-x  1  510 wheel 2.0K Aug 28  2011 fetch_data.sh
    -rwxr-xr-x  1  510 wheel  12K Aug 28  2011 create_defs.pl
    -rw-r--r--  1  510 wheel  22K Aug 28  2011 constraints_parser.c
    -rw-r--r--  1  510 wheel 6.2K Aug 28  2011 check_data.c
    drwxr-xr-x  2  510 wheel 4.0K Aug 28  2011 include
    drwxr-xr-x  2  510 wheel 4.0K Aug 28  2011 docs
    drwxr-xr-x 40  510 wheel 4.0K Sep 26 17:53 mysql-source
    drwxr-xr-x  2 root root  4.0K Sep 26 17:55 lib
    -rwxr-xr-x  1 root root  725K Sep 26 17:55 constraints_parser
    -rwxr-xr-x  1 root root  1.2M Sep 26 17:55 page_parser
    -rwxr-xr-x  1 root root   15K Sep 26 17:55 innochecksum
    -rwxr-xr-x  1 root root  973K Sep 26 17:55 ibdconnect
    

    page_parser工具将根据InnoDB的底层实现原理,解析表的页和行结构。constraints_parser工具暂时不使用,后续还需要在定义表结构之后,重新编译生成它。

    如果MySQL是5.0之前的版本,InnoDB采取的是REDUNDANT格式,运行以下命令:

    ./page_parser -4 -f /path/to/ibdata1
    

    如果MySQL是5.0以后的版本,InnoDB采取的是COMPACT格式,运行以下命令:

    ./page_parser -5 -f /path/to/ibdata1
    

    运行后,page_parser工具会创建一个pages-<TIMESTAMP>的目录,其中TIMESTAMP是UNIX系统时间戳。在这个目录下,为每个index ID,以页的index ID创建一个子目录

    ...

    ...

    ...

    这个工具限制很多,没有测试,具体步骤可以参考 "其他参考"

    其他参考:

    Percona Data Recovery Tool 单表恢复

    使用Percona Data Recovery Tool for InnoDB恢复数据

    无备份情况下恢复MySQL误删的表

    Undrop MySQL InnoDB 中恢复被drop的表,当 innodb_file_per_table=off时

    undrop-for-innodb

    github percona live ppt wiki

    「2017-01-01 已经闭源并收费」,来源wiki

    MySQL · 数据恢复 · undrop-for-innodb 「数据库内核月报 - 2017 / 11」

    如果 innodb_file_per_table = on 恢复可能性更低

    • idb 文件被脏写
    • 有单独的 purge 线程

    限制

    • innodb_purge_threads = 0

    innodb_purge_threads <=5.6.1,默认为0,范围 [0-1]

    ​ = 5.6.2, <= 5.6.4 默认为0,范围 [0-32]

    ​ = 5.6.5 默认为1,范围 [1-32]

    ​ = 5.7.8 默认为4,范围 [4-32]

    5.6 以后,DDL 恢复还是比较困难的

    安装

    git clone https://github.com/chhabhaiya/undrop-for-innodb
    cd undrop-for-innodb
    make
    

    使用 「MySQL 5.6.24」

    1、truncate table

    localhost.test>select * from t_recover;
    +----+--------+
    
    | id | name   |
    
    +----+--------+
    |  1 | test1  |
    |  2 | test2  |
    |  3 | test3  |
    |  4 | test4  |
    |  5 | test5  |
    |  6 | test6  |
    |  7 | test7  |
    |  8 | test8  |
    |  9 | test9  |
    | 10 | test10 |
    +----+--------+
    
    10 rows in set (0.00 sec)
    
    mysql> truncate table t_recover;
    
    

    2、获取数据字典

    # ./stream_parser -f /data1/mysql7000/ibdata1 
    
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    ID of device containing file:         2055
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    ID of device containing file:         2055
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    Opening file: /data1/mysql7000/ibdata1
    total size, in bytes:            104857600 (100.000 MiB)
    
    number of blocks allocated:         205008
    File information:
    
    Size to process:                 104857600 (100.000 MiB)
    ID of device containing file:         2055
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    ID of device containing file:         2055
    inode number:                      7012359
    protection:                         100640 (regular file)
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    total size, in bytes:            104857600 (100.000 MiB)
    
    Opening file: /data1/mysql7000/ibdata1
    total size, in bytes:            104857600 (100.000 MiB)
    
    File information:
    
    Size to process:                 104857600 (100.000 MiB)
    ID of device containing file:         2055
    Size to process:                 104857600 (100.000 MiB)
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    total size, in bytes:            104857600 (100.000 MiB)
    
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    Size to process:                 104857600 (100.000 MiB)
    ID of device containing file:         2055
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    Opening file: /data1/mysql7000/ibdata1
    user ID of owner:                      647
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    File information:
    
    group ID of owner:                     553
    device ID (if special file):             0
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    blocksize for filesystem I/O:         4096
    ID of device containing file:         2055
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    number of blocks allocated:         205008
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    total size, in bytes:            104857600 (100.000 MiB)
    
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    Size to process:                 104857600 (100.000 MiB)
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    ID of device containing file:         2055
    inode number:                      7012359
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    protection:                         100640 (regular file)
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    number of hard links:                    1
    user ID of owner:                      647
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    group ID of owner:                     553
    device ID (if special file):             0
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    total size, in bytes:            104857600 (100.000 MiB)
    
    Size to process:                 104857600 (100.000 MiB)
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    ID of device containing file:         2055
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    total size, in bytes:            104857600 (100.000 MiB)
    
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    Size to process:                 104857600 (100.000 MiB)
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    total size, in bytes:            104857600 (100.000 MiB)
    
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    Size to process:                 104857600 (100.000 MiB)
    ID of device containing file:         2055
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    total size, in bytes:            104857600 (100.000 MiB)
    
    Size to process:                 104857600 (100.000 MiB)
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    total size, in bytes:            104857600 (100.000 MiB)
    
    Size to process:                 104857600 (100.000 MiB)
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    ID of device containing file:         2055
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    total size, in bytes:            104857600 (100.000 MiB)
    
    Size to process:                 104857600 (100.000 MiB)
    Opening file: /data1/mysql7000/ibdata1
    File information:
    
    ID of device containing file:         2055
    inode number:                      7012359
    protection:                         100640 (regular file)
    number of hard links:                    1
    user ID of owner:                      647
    group ID of owner:                     553
    device ID (if special file):             0
    blocksize for filesystem I/O:         4096
    number of blocks allocated:         205008
    time of last access:            1506651425 Fri Sep 29 10:17:05 2017
    time of last modification:      1506651465 Fri Sep 29 10:17:45 2017
    time of last status change:     1506651465 Fri Sep 29 10:17:45 2017
    total size, in bytes:            104857600 (100.000 MiB)
    
    Size to process:                 104857600 (100.000 MiB)
    All workers finished in 0 sec
    
    

    stream_parser这个工具发现在ibdata1中,找到了存储页面类型(FIL_PAGE_INDEX或FIL_PAGE_TYPE_BLOB)由index_id的整理InnoDB的页面

    # cd pages-ibdata1/
    
    # ll -rth
    total 8.0K
    drwxr-xr-x 2 root root 4.0K Sep 29 10:18 FIL_PAGE_TYPE_BLOB
    drwxr-xr-x 2 root root 4.0K Sep 29 10:18 FIL_PAGE_INDEX
    
    

    SYS_TABLES

    SYS_TABLES本身是一个表,该表用于规范innodb各种表定义和保存innodb中各种表的基本信息,结合SYS_COLUMNS,SYS_INDEXES和SYS_FOREIGN系统表定义了特定某个表的所有信息。

    undrop-for-innodb]# ll pages-ibdata1/FIL_PAGE_INDEX/0000000000000001.page 
    -rw-r--r-- 1 root root 32768 Sep 29 10:18 pages-ibdata1/FIL_PAGE_INDEX/0000000000000001.page
    

    SYS_INDEXES

    SYS_INDEXES用于保存innodb中每个表定义的每个索引对象

    undrop-for-innodb]# ll pages-ibdata1/FIL_PAGE_INDEX/0000000000000003.page 
    -rw-r--r-- 1 root root 16384 Sep 29 10:18 pages-ibdata1/FIL_PAGE_INDEX/0000000000000003.page
    

    SYS_COLUMNS

    SYS_COLUMNS用于保存innodb引擎每个表定义的列,与SYS_TABLES相似。

    undrop-for-innodb]# ll pages-ibdata1/FIL_PAGE_INDEX/0000000000000002.page 
    -rw-r--r-- 1 root root 32768 Sep 29 10:18 pages-ibdata1/FIL_PAGE_INDEX/0000000000000002.page
    

    SYS_FIELDS

    SYS_FIELDS用于保存innodb中每个索引的每个列对象

    undrop-for-innodb]# ll pages-ibdata1/FIL_PAGE_INDEX/0000000000000004.page 
    -rw-r--r-- 1 root root 32768 Sep 29 10:18 pages-ibdata1/FIL_PAGE_INDEX/0000000000000004.page
    

    3、扫描逻辑卷

    ./stream_parser -f /dev/sda7 -t 105000000k
    
    Opening file: /dev/sda7
    File information:
    
    ID of device containing file:            5
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    Opening file: /dev/sda7
    File information:
    
    ID of device containing file:            5
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    total size, in bytes:                    0 (0.000 exp(+0))
    
    Opening file: /dev/sda7
    File information:
    
    Size to process:              107520000000 (100.136 GiB)
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    ID of device containing file:            5
    inode number:                         6047
    protection:                          60660 total size, in bytes:                    0 (0.000 exp(+0))
    
    (block device)
    Opening file: /dev/sda7
    number of hard links:                    1
    Size to process:              107520000000 (100.136 GiB)
    File information:
    
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    ID of device containing file:            5
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    Opening file: /dev/sda7
    File information:
    
    ID of device containing file:            5
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    Opening file: /dev/sda7
    user ID of owner:                        0
    group ID of owner:                       6
    File information:
    
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    ID of device containing file:            5
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    user ID of owner:                        0
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    group ID of owner:                       6
    total size, in bytes:                    0 (0.000 exp(+0))
    
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    total size, in bytes:                    0 (0.000 exp(+0))
    
    Size to process:              107520000000 (100.136 GiB)
    Size to process:              107520000000 (100.136 GiB)
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    Opening file: /dev/sda7
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    File information:
    
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    ID of device containing file:            5
    total size, in bytes:                    0 (0.000 exp(+0))
    
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    Size to process:              107520000000 (100.136 GiB)
    number of blocks allocated:              0
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    Opening file: /dev/sda7
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    File information:
    
    total size, in bytes:                    0 (0.000 exp(+0))
    
    ID of device containing file:            5
    inode number:                         6047
    Size to process:              107520000000 (100.136 GiB)
    protection:                          60660 (block device)
    number of hard links:                    1
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    Opening file: /dev/sda7
    total size, in bytes:                    0 (0.000 exp(+0))
    
    File information:
    
    ID of device containing file:            5
    Size to process:              107520000000 (100.136 GiB)
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    total size, in bytes:                    0 (0.000 exp(+0))
    
    Size to process:              107520000000 (100.136 GiB)
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    total size, in bytes:                    0 (0.000 exp(+0))
    
    Size to process:              107520000000 (100.136 GiB)
    Opening file: /dev/sda7
    File information:
    
    Opening file: /dev/sda7
    File information:
    
    ID of device containing file:            5
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    ID of device containing file:            5
    user ID of owner:                        0
    inode number:                         6047
    group ID of owner:                       6
    protection:                          60660 device ID (if special file):          2055
    (block device)
    blocksize for filesystem I/O:         4096
    number of hard links:                    1
    number of blocks allocated:              0
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    total size, in bytes:                    0 (0.000 exp(+0))
    
    total size, in bytes:                    0 (0.000 exp(+0))
    
    Size to process:              107520000000 (100.136 GiB)
    Size to process:              107520000000 (100.136 GiB)
    Opening file: /dev/sda7
    File information:
    
    ID of device containing file:            5
    inode number:                         6047
    protection:                          60660 (block device)
    number of hard links:                    1
    user ID of owner:                        0
    group ID of owner:                       6
    device ID (if special file):          2055
    blocksize for filesystem I/O:         4096
    number of blocks allocated:              0
    time of last access:            1490933835 Fri Mar 31 12:17:15 2017
    time of last modification:      1477372695 Tue Oct 25 13:18:15 2016
    time of last status change:     1477372695 Tue Oct 25 13:18:15 2016
    total size, in bytes:                    0 (0.000 exp(+0))
    
    Size to process:              107520000000 (100.136 GiB)
    Worker(2): 1.12% done. 2017-09-29 10:22:56 ETA(in 00:03:12). Processing speed: 44.000 MiB/sec
    Worker(3): 1.12% done. 2017-09-29 10:22:56 ETA(in 00:03:12). Processing speed: 44.000 MiB/sec
    
    ....
    ....
    ....
    
    Worker(11): 97.77% done. 2017-09-29 10:23:21 ETA(in 00:00:02). Processing speed: 87.828 MiB/sec
    Worker(0): 97.74% done. 2017-09-29 10:23:21 ETA(in 00:00:02). Processing speed: 87.844 MiB/sec
    Worker(9): 97.74% done. 2017-09-29 10:23:21 ETA(in 00:00:02). Processing speed: 87.844 MiB/sec
    Worker(8): 99.80% done. 2017-09-29 10:23:20 ETA(in 00:00:00). Processing speed: 87.828 MiB/sec
    Worker(11): 98.80% done. 2017-09-29 10:23:21 ETA(in 00:00:01). Processing speed: 87.828 MiB/sec
    Worker(0): 98.77% done. 2017-09-29 10:23:21 ETA(in 00:00:01). Processing speed: 87.828 MiB/sec
    Worker(9): 98.77% done. 2017-09-29 10:23:21 ETA(in 00:00:01). Processing speed: 87.828 MiB/sec
    Worker(0): 99.80% done. 2017-09-29 10:23:21 ETA(in 00:00:00). Processing speed: 87.828 MiB/sec
    Worker(11): 99.83% done. 2017-09-29 10:23:21 ETA(in 00:00:00). Processing speed: 87.828 MiB/sec
    Worker(9): 99.80% done. 2017-09-29 10:23:21 ETA(in 00:00:00). Processing speed: 87.844 MiB/sec
    All workers finished in 219 sec
    [root@idaross29 undrop-for-innodb]# ./stream_parser -f /dev/sda7 -t 105000000k
    Could not create directory pages-sda7
    mkdir(): File exists
    

    4、创建数据字典表

    recover_dictionary.sh
    
    修改 mysql 登陆相关命令 mysql -u -p -S
    
    # ./recover_dictionary.sh 
    Generating dictionary tables dumps... OK
    Creating test database ... OK
    Creating dictionary tables in database test:
    SYS_TABLES ... OK
    SYS_COLUMNS ... OK
    SYS_INDEXES ... OK
    SYS_FIELDS ... OK
    All OK
    Loading dictionary tables data:
    SYS_TABLES ... 24 recs OK
    SYS_COLUMNS ... 148 recs OK
    SYS_INDEXES ... 14 recs OK
    SYS_FIELDS ... 34 recs OK
    All OK
    
    localhost.test>show tables;
    +----------------+
    | Tables_in_test |
    +----------------+
    | SYS_COLUMNS    |
    | SYS_FIELDS     |
    | SYS_INDEXES    |
    | SYS_TABLES     |
    

    5、查询需要恢复的表的index_id信息

    localhost.test>select * from SYS_TABLES where name like 'test/t_recover';
    +----------------+----+--------+------+--------+---------+--------------+-------+
    | NAME           | ID | N_COLS | TYPE | MIX_ID | MIX_LEN | CLUSTER_NAME | SPACE |
    +----------------+----+--------+------+--------+---------+--------------+-------+
    | test/t_recover | 25 |      2 |    1 |      0 |      80 |              |    11 |
    +----------------+----+--------+------+--------+---------+--------------+-------+
    1 row in set (0.02 sec)
    
    localhost.test>select * from SYS_INDEXES where table_id = 25;
    +----------+----+---------+----------+------+-------+---------+
    | TABLE_ID | ID | NAME    | N_FIELDS | TYPE | SPACE | PAGE_NO |
    +----------+----+---------+----------+------+-------+---------+
    |       25 | 28 | PRIMARY |        1 |    3 |    11 |       3 |
    +----------+----+---------+----------+------+-------+---------+
    1 row in set (0.01 sec)
    

    可以看到被truncate的表的index_id 为28,我们应该进一步从28 的page中获取数据。

    6、拿到表结构,并检查数据是否可以恢复

    • 表结构
    cat t_recover.sql 
    create table t_recover(id int primary key,name varchar(20));
    
    • 确认数据是否存在
    undrop-for-innodb]# ./c_parser -6f pages-ibdata1/FIL_PAGE_INDEX/0000000000000028.page -t t_recover.sql | head -5
    SET FOREIGN_KEY_CHECKS=0;
    -- Page id: 3, Format: COMPACT, Records list: Valid, Expected records: (0 0)
    LOAD DATA LOCAL INFILE '-- Page id: 3, Found records: 0, Lost records: NO, Leaf page: YES
    -- Page id: 3, Format: COMPACT, Records list: Valid, Expected records: (0 0)
    /data1/jinlin3/undrop-for-innodb/dumps/default/t_recover' REPLACE INTO TABLE `t_recover` FIELDS TERMINATED BY '	' OPTIONALLY ENCLOSED BY '"' LINES STARTING BY 't_recover	' (`id`, `name`);
    

    7、抽取 Page 中的数据

    undrop-for-innodb]# ./c_parser -6f pages-ibdata1/FIL_PAGE_INDEX/0000000000000028.page -t t_recover.sql > dumps/default/t_recover 2> dumps/default/t_recover_load.sql
    

    8、加载数据

    source dumps/default/t_recover_load.sql
    

    但是,我测试是 5.6 ,数据被 purge 掉了

    undrop-for-innodb]# cat dumps/default/t_recover
    -- Page id: 3, Format: COMPACT, Records list: Valid, Expected records: (0 0)
    -- Page id: 3, Found records: 0, Lost records: NO, Leaf page: YES
    -- Page id: 3, Format: COMPACT, Records list: Valid, Expected records: (0 0)
    -- Page id: 3, Found records: 0, Lost records: NO, Leaf page: YES
    
    
    undrop-for-innodb]# cat dumps/default/t_recover_load.sql 
    SET FOREIGN_KEY_CHECKS=0;
    LOAD DATA LOCAL INFILE '/data1/jinlin3/undrop-for-innodb/dumps/default/t_recover' REPLACE INTO TABLE `t_recover` FIELDS TERMINATED BY '	' OPTIONALLY ENCLOSED BY '"' LINES STARTING BY 't_recover	' (`id`, `name`);
    

    若 truncate table 后,立刻 shutdown MySQL,文件是有内容,但不对啊!

    undrop-for-innodb]# cat dumps/default/t_truncate
    -- Page id: 3, Format: COMPACT, Records list: Valid, Expected records: (17 17)
    0000000B0000    00000000000025  t_truncate      -2147483648     ""
    0000000C0000    00000000000025  t_truncate      -2147483648     "@�"
    0000000D0000    00000000000025  t_truncate      -2147483648     "@"
    0000000E0000    00000000000025  t_truncate      -2147483648     ""
    0000000E0000    00010000000025  t_truncate      -2147483648     ""
    0000000F0000    00000000000025  t_truncate      -2147483648     ""
    000000100000    00000000000025  t_truncate      -2147483648     ""
    000000120000    00030000000025  t_truncate      -2147483648     "@ps"
    000000130000    00000000000025  t_truncate      -2147483648     ""
    000000140000    00000000000025  t_truncate      -2147483648     ""
    000000140000    00010000000025  t_truncate      -2147483648     ""
    000000150000    00000000000025  t_truncate      -2147483648     ""
    000000160000    00000000000025  t_truncate      -2147483648     ""
    000000170000    00000000000025  t_truncate      -2147483648     ""
    000000180000    00000000000025  t_truncate      -2147483648     "@ka"
    0000001A0000    00000000000025  t_truncate      -2147483648     ""
    0000001C0000    00000000000025  t_truncate      -2147483648     ""
    -- Page id: 3, Found records: 17, Lost records: NO, Leaf page: YES
    
    

    <u>这是一个悲伤的故事!</u>

    其他参考:

    undrop for innodb c_parser 源码分析

    Recover InnoDB dictionary

    MySQL 如何对InnoDB使用Undrop来恢复InnoDB数据

    希望小伙伴永远不需要用到此方法,备份才是王道!



    作者:saup007
    链接:https://www.jianshu.com/p/31818e1727be
    來源:简书
    简书著作权归作者所有,任何形式的转载都请联系作者获得授权并注明出处。
  • 相关阅读:
    d3.js--04(enter和exit)
    d3.js--03(增删改查)
    d3.js--02(data和datum原理)
    css文件和js文件后面带一个问号----2015-1103
    json_decode()和json_encode()区别----2015-0929
    随记--更新于20150925
    个人火狐插件汇总--20150924
    『jQuery』.html(),.text()和.val()的概述及使用--2015-08-11
    矩阵求导
    傅里叶变换
  • 原文地址:https://www.cnblogs.com/DataArt/p/9873365.html
Copyright © 2020-2023  润新知