• 阅读xtrabackup代码的一点笔记


    xtrabackup binary最重要的两个过程是backup和prepare,对应的函数分别是xtrabackup_backup_func()和xtrabackup_prepare_func(),这里做一些阅读代码时的笔记。

    xtrabackup backup的线程模型:
    1. 一个log拷贝线程;
    2. n个ibd文件拷贝线程;
    3. 一个io监控线程;
    4. 通过suspend_start/suspend_end文件来标注是否启动终止线程;
     
    typedef struct {
        datafiles_iter_t *it;
        uint num;
        uint *count;
        os_ib_mutex_t  count_mutex;
        os_thread_id_t  id;
    }data_thread_ctxt_t;
    数据线程上下文
     
    切换工作目录;
     
     116 /** Set if InnoDB must operate in read-only mode. We don't do any
     117 recovery and open all tables in RO mode instead of RW mode. We don't
     118 sync the max trx id to disk either. */
    xb_set_innodb_read_only()  将innodb设成只读模式
    srv_backup_mode=TRUE;  将innodb设成backup模式;
     
    设置innodb的一系列参数
    innodb_init_param()
     
    xb_normalize_init_values(void)
     
    修改srv_unix_file_flush_method
     
    根据bp大小,调整srv_max_n_threads参数
     
    1017 /*********************************************************************//**
    1018 Initializes the synchronization primitives, memory system, and the thread
    1019 local storage. */
    srv_general_init()
     
    ut_crc32_init()
     
    xb_filters_init()
     
    2567 /************************************************************************
    2568 Initializes the I/O and tablespace cache subsystems. */
    xb_fil_io_init(void)
     
     838 /******************************************************//**
     839 Initializes the log. */
    log_init(void)
     
     585 /*********************************************************************//**
     586 Creates the lock system at database start. */
    lock_sys_create()
     
    open_or_create_log_file
     
    创建xtrabackup_extra_lsndir/extrabackup_traget_dir
     
    表空间memory cache
    fil_system_t* f_system = fil_system;
     
    recv_find_max_checkpoint(&max_cp_group, &max_cp_field)
     
    log_group_read_checkpoint_info(max_cp_group, max_cp_field)
     
    checkpoint_lsn_start/checkpoint_no_start
    确认一致的checkpoint状态;
     
    创建XB_LOG_FILENAME文件,写入文件头信息;
     
    创建io_watching_thread;
     
    从checkpoint位置开始copy log文件;
    xtrabackup_copy_logfile(checkpoint_lsn_start, FALSE)
     
    log_copying/log_copying_stop
     
    创建日志copy线程
    os_thread_create(log_copying_thread, NULL, &log_copying_thread_id);
     
    2591 /****************************************************************************
    2592 Populates the tablespace memory cache by scanning for and opening data files.
    2593 @returns DB_SUCCESS or error code.*/
    xb_load_tablespaces()
     
    挂起,等待XB_FN_SUSPENDED_AT_START文件被删除
    xtrabackup_suspend
     
    xb_page_bitmap_init()
     
    根据xtrabackup_parallel设置,创建data_copy_thread_func线程
     
    等待所有data_copy_thread_func线程退出
     
    挂起,等待XB_FN_SUSPENDED_AT_END文件被删除
    xtrabackup_suspend
     
    读取最新的checkpoint, 记录在metadata的to_lsn字段;
    通过设置log_copying=FALSE && set log_copying_stop,停止log_copying_thread;
     
    创建一个文件XB_FN_LOG_COPIED,通知外部脚本,log_copying_thread已经结束;
     
    写metadata;
     
     
    prepare_func
     
    1. 切换到xtrabackup_real_target_dir
    2. 读取XTRABACKUP_METADATA_FILENAME,获取原信息;
        metadata_type
    3. xtrabackup_init_temp_log()
    4. innodb_init_param()
     
    2670 /************************************************************************
    2671 Initialize the tablespace memory cache and populate it by scanning for and
    2672 opening data files.
    2673 @returns DB_SUCCESS or error code.*/
    xb_data_files_init()
     
    应用增量到全量
    xtrabackup_apply_deltas()
     
    重设innodb初始化参数
    innodb_init_param()
     
     
    innodb_init()
     
    遍历文件
    mtr_start -> mtr_commit
     
     
    trx_sys_print_mysql_binlog_offset()
    将binlog位置信息输出到 xtrabackup_binlog_pos_innodb文件中
     
    xtrabackup_close_temp_log(TRUE)
     
    输出记录metadata_log
     
     
    backup
    $./xtrabackup_56 --defaults-file=/u01/my3928/my.cnf --backup --target_dir=/u01/xianlin.lh/backup_dir/
    ./xtrabackup_56 version 2.1.8 for MySQL server 5.6.15 Linux (x86_64) (revision id: undefined)
    xtrabackup: uses posix_fadvise().
    xtrabackup: cd to /u01/my3928/data
    xtrabackup: using the following InnoDB configuration:
    xtrabackup: innodb_data_home_dir = /u01/my3928/data
    xtrabackup: innodb_data_file_path = ibdata1:4G;ibdata2:16M:autoextend
    xtrabackup: innodb_log_group_home_dir = /u01/my3928/data
    xtrabackup: innodb_log_files_in_group = 4
    xtrabackup: innodb_log_file_size = 1073741824
    2014-05-05 17:29:35 2ac06bc4a2c0 InnoDB: Using Linux native AIO
    xtrabackup: using O_DIRECT
    >> log scanned up to (1451746590)
    [01] Copying /u01/my3928/data/ibdata1 to /u01/xianlin.lh/backup_dir/ibdata1
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    [01] ...done
    [01] Copying /u01/my3928/data/ibdata2 to /u01/xianlin.lh/backup_dir/ibdata2
    [01] ...done
    [01] Copying ./test/t2.ibd to /u01/xianlin.lh/backup_dir/test/t2.ibd
    [01] ...done
    [01] Copying ./test/t4.ibd to /u01/xianlin.lh/backup_dir/test/t4.ibd
    [01] ...done
    [01] Copying ./test/t3.ibd to /u01/xianlin.lh/backup_dir/test/t3.ibd
    [01] ...done
    [01] Copying ./test/sbtest1.ibd to /u01/xianlin.lh/backup_dir/test/sbtest1.ibd
    >> log scanned up to (1451746590)
    >> log scanned up to (1451746590)
    [01] ...done
    [01] Copying ./test/t1.ibd to /u01/xianlin.lh/backup_dir/test/t1.ibd
    [01] ...done
    [01] Copying ./mysql/innodb_index_stats.ibd to /u01/xianlin.lh/backup_dir/mysql/innodb_index_stats.ibd
    [01] ...done
    [01] Copying ./mysql/slave_worker_info.ibd to /u01/xianlin.lh/backup_dir/mysql/slave_worker_info.ibd
    [01] ...done
    [01] Copying ./mysql/innodb_table_stats.ibd to /u01/xianlin.lh/backup_dir/mysql/innodb_table_stats.ibd
    [01] ...done
    [01] Copying ./mysql/slave_relay_log_info.ibd to /u01/xianlin.lh/backup_dir/mysql/slave_relay_log_info.ibd
    [01] ...done
    [01] Copying ./mysql/slave_master_info.ibd to /u01/xianlin.lh/backup_dir/mysql/slave_master_info.ibd
    [01] ...done
    >> log scanned up to (1451746590)
    xtrabackup: The latest check point (for incremental): '1451746590'
    xtrabackup: Stopping log copying thread.
    .>> log scanned up to (1451746590)
     
    xtrabackup: Transaction log of lsn (1451746590) to (1451746590) was copied.
     
     
    创建备份的流程
     
    1 backup_type = full-backuped
    2 from_lsn = 0
    3 to_lsn = 1451746590
    4 last_lsn = 1451746590
    5 compact = 0
     
    prepare
    $./xtrabackup_56 --defaults-file=/u01/my3928/my.cnf --prepare --target-dir=/u01/xianlin.lh/backup_dir/
    ./xtrabackup_56 version 2.1.8 for MySQL server 5.6.15 Linux (x86_64) (revision id: undefined)
    xtrabackup: cd to /u01/xianlin.lh/backup_dir/
    xtrabackup: This target seems to be not prepared yet.
    xtrabackup: xtrabackup_logfile detected: size=2097152, start_lsn=(1451746590)
    xtrabackup: using the following InnoDB configuration for recovery:
    xtrabackup: innodb_data_home_dir = ./
    xtrabackup: innodb_data_file_path = ibdata1:4G;ibdata2:16M:autoextend
    xtrabackup: innodb_log_group_home_dir = ./
    xtrabackup: innodb_log_files_in_group = 1
    xtrabackup: innodb_log_file_size = 2097152
    2014-05-05 18:38:32 2b7f2f5202c0 InnoDB: Using Linux native AIO
    xtrabackup: using the following InnoDB configuration for recovery:
    xtrabackup: innodb_data_home_dir = ./
    xtrabackup: innodb_data_file_path = ibdata1:4G;ibdata2:16M:autoextend
    xtrabackup: innodb_log_group_home_dir = ./
    xtrabackup: innodb_log_files_in_group = 1
    xtrabackup: innodb_log_file_size = 2097152
    2014-05-05 18:38:32 2b7f2f5202c0 InnoDB: Using Linux native AIO
    xtrabackup: Starting InnoDB instance for recovery.
    xtrabackup: Using 104857600 bytes for buffer pool (set by --use-memory parameter)
    InnoDB: The InnoDB memory heap is disabled
    InnoDB: Mutexes and rw_locks use GCC atomic builtins
    InnoDB: Compressed tables use zlib 1.2.3
    InnoDB: Using Linux native AIO
    InnoDB: Using CPU crc32 instructions
    InnoDB: Initializing buffer pool, size = 100.0M
    InnoDB: Completed initialization of buffer pool
    InnoDB: Highest supported file format is Barracuda.
    InnoDB: The log sequence numbers 552524932 and 552524932 in ibdata files do not match the log sequence number 1451746590 in the ib_logfiles!
    InnoDB: Database was not shutdown normally!
    InnoDB: Starting crash recovery.
    InnoDB: Reading tablespace information from the .ibd files...
    InnoDB: Restoring possible half-written data pages
    InnoDB: from the doublewrite buffer...
    InnoDB: Last MySQL binlog file position 0 26767310, file name mysql-bin.000023
    InnoDB: 128 rollback segment(s) are active.
    InnoDB: Waiting for purge to start
    2014-05-05 18:38:33 2b7f49056700 InnoDB: Warning: table 'test/sbtest1'
    InnoDB: in InnoDB data dictionary has unknown flags 50.
    InnoDB: 5.6.15 started; log sequence number 1451746590
     
    [notice (again)]
      If you use binary log and don't use any hack of group commit,
      the binary log position seems to be:
    InnoDB: Last MySQL binlog file position 0 26767310, file name mysql-bin.000023
     
    xtrabackup: starting shutdown with innodb_fast_shutdown = 1
    InnoDB: FTS optimize thread exiting.
    InnoDB: Starting shutdown...
    InnoDB: Shutdown completed; log sequence number 1451747281
     
    prepare的流程:
     
     
      1 backup_type = full-prepared
      2 from_lsn = 0
      3 to_lsn = 1451746590
      4 last_lsn = 1451746590
      5 compact = 0 
  • 相关阅读:
    高可用性GRE+IPSEC中心—分支
    高可用性GRE+IPSEC中心—分支
    高可用性GRE+IPSEC中心—分支
    mysql数据库移植
    mysql数据库移植
    mysql数据库移植
    mysql数据库移植
    Linux 查看进程资源--ps、top命令
    比特币底层设计剖析
    比特币的P2P网络协议
  • 原文地址:https://www.cnblogs.com/liuhao/p/3709854.html
Copyright © 2020-2023  润新知