mysqldump不得不说的秘密

原文链接：http://www.mysqlperformanceblog.com/2012/03/23/best-kept-mysqldump-secret/

Many people use mysqldump –single-transaction to get consistent backup for their Innodb tables without making database read only.  In most cases it works, but did you know there are some cases when you can get table entirely missing from the backup if you use this technique ?

很多人用在mysqldump的时候带上–single-transaction，这样就可以不锁表获取一致性备份集了。在绝大多数场景下确实是这样的，但是你知不知道有时候用这个技巧会导致备份集完全丢失数据？

The problem comes from the fact how MySQL’s Transactions work with DDL, In particular ALTER TABLE.   When ALTER TABLE is Performed in many cases it will Create temporary table with modified structure, copy data to that table and when drop original table and rename such temporary table to original name.

这个问题的实际上是由于mysql的事务操作DDL造成的，特别是ALTER TABLE操作。在很多时候，ALTER TABLE 执行方式是：创建新结构的临时表，拷贝数据到临时表，删除原表，把临时表重命名为原表。

How does data visibility works in this case ?  DDLs are not transactional and as such the running transaction will not see the contents of old table once it is dropped,  transaction also will see the new table which was created after transaction was started, including table created by ALTER TABLE statement.     Transactions however apply to DATA which is stored in this table and so data which was inserted after start of transaction (by ALTER TABLE statement) will not be visible.  In the end we will get new structure in the dump but no data.

在这个场景下，数据的可见性是怎么工作的？DDL是非事务性的，与此同时，一旦旧表被删除，活动的事务是看不到旧表的数据的，事务能看到事务开始后创建的新表，包括由ALTER TABLE语句创建的新表。然而事务涉及的数据存储在这个新表里，而这些数据是在事务开始之后插入到新表的（通过ALTER TABLE语句），这些数据是不可见的。最后的结果是我们有了最新的表表结构，但是却没有数据。

Here is example:  

SESSION1:
mysql> show tables;
+--------------------+
| Tables_in_dumptest |
+--------------------+
| A                  |
| B                  |
| C                  |
+--------------------+
3 rows in set (0.00 sec)

mysql> select count(*) from A;
+----------+
| count(*) |
+----------+
|  2359296 |
+----------+
1 row in set (1.73 sec)


mysql> select * from C;
+------+
| t    |
+------+
| test |
+------+
1 row in set (0.00 sec)



SESSION2:
root@ubuntu:~/dump# mysqldump --single-transaction dumptest > dump.sql

SESSION1:  (before dump has completed)
mysql> alter table C add i int not null;
Query OK, 1 row affected (0.65 sec)
Records: 1  Duplicates: 0  Warnings: 0

SESSION2:

root@ubuntu:~/dump# tail -29 dump.sql

DROP TABLE IF EXISTS `C`;
/*!40101 SET @saved_cs_client     = @@character_set_client */;
/*!40101 SET character_set_client = utf8 */;
CREATE TABLE `C` (
  `t` char(255) NOT NULL,
  `i` int(11) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;

--
-- Dumping data for table `C`
--

LOCK TABLES `C` WRITE;
/*!40000 ALTER TABLE `C` DISABLE KEYS */;
/*!40000 ALTER TABLE `C` ENABLE KEYS */;
UNLOCK TABLES;
/*!40103 SET TIME_ZONE=@OLD_TIME_ZONE */;

/*!40101 SET SQL_MODE=@OLD_SQL_MODE */;
/*!40014 SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS */;
/*!40014 SET UNIQUE_CHECKS=@OLD_UNIQUE_CHECKS */;
/*!40101 SET CHARACTER_SET_CLIENT=@OLD_CHARACTER_SET_CLIENT */;
/*!40101 SET CHARACTER_SET_RESULTS=@OLD_CHARACTER_SET_RESULTS */;
/*!40101 SET COLLATION_CONNECTION=@OLD_COLLATION_CONNECTION */;
/*!40111 SET SQL_NOTES=@OLD_SQL_NOTES */;

-- Dump completed on 2012-03-23  7:27:18

SESSION1:

mysql> select * from C;
+------+---+
| t    | i |
+------+---+
| test | 0 |
+------+---+
1 row in set (0.00 sec)

As you can see as we altered table C at the same time as mysqldump was running we got table empty table with new structure in mysqldump instead of table with valuable data.

如你所见，我们在mysqldump运行时对C表执行alter table操作，在备份文件里我们得到了新结构，但是是没有数据的。

This is a pretty edge case scenario neither the less it can be problem for some workloads which run ALTER TABLE regularly during normal operation.  I also hope if you get some empty tables in your mysqldump –single-transaction backups you will know the potential cause for it.

What are potential solutions for this problem ?  you can use mysqldump –lock-all-tables instead which does not have this problem at the cost of having database read only for the duration of operation.  You can also use Percona Xtrabackup, LVM or other database backup approach which does not relay on transaction visibility.

这是一个非常极端的例子，在日常工作中去下运行ALTER TABLE 是不会出现问题的。我也希望当你使用mysqldump –single-transaction备份表却没数据时，知道它可能导致这样的情况的发生。

对于这个问题有什么办法解决呢？你可以用mysqldump –lock-all-tables ，它没有这个问题，在备份运行时数据库是只读的。你也可以尝试Percona Xtrabackup,LVM或其他的在事务可见度上没有问题的备份工具

##############################################################################################

一楼：

Hi Peter!
 
It is not really a secret… Quoting MySQL manual from http://dev.mysql.com/doc/refman/5.5/en/mysqldump.html :
 
“While a –single-transaction dump is in process, to ensure a valid dump file (correct table contents and binary log coordinates), no other connection should use the following statements: ALTER TABLE, CREATE TABLE, DROP TABLE, RENAME TABLE, TRUNCATE TABLE. A consistent read is not isolated from those statements, so use of them on a table to be dumped can cause the SELECT that is performed by mysqldump to retrieve the table contents to obtain incorrect contents or fail.”