mysql 高可用架构 mha 之一安装

os:centos 7.4
mysql: 5.7
mha: 0.58

MHA（Master High Availability）目前在MySQL高可用方面是一个相对成熟的解决方案,是一套优秀的作为MySQL高可用性环境下故障切换和主从提升的高可用软件。

该软件由两部分组成：MHA Manager（管理节点）和MHA Node（数据节点）。

MHA Manager
可以单独部署在一台独立的机器上管理多个master-slave集群，也可以部署在一台slave节点上。MHA Manager会定时探测集群中的master节点，当master出现故障时，它可以自动将最新数据的slave提升为新的master，然后将所有其他的slave重新指向新的master。整个故障转移过程对应用程序完全透明。

MHA Node
运行在每台MySQL服务器上，定时和 MHA Manager交互信息。

在MHA自动故障切换过程中，MHA试图从宕机的主服务器上保存二进制日志，最大程度的保证数据的不丢失，但这并不总是可行的。
例如，如果主服务器硬件故障或无法通过ssh访问，MHA没法保存二进制日志，只进行故障转移而丢失了最新的数据。

使用MySQL 的半同步复制，可以大大降低数据丢失的风险。MHA可以与半同步复制结合起来。如果只有一个slave已经收到了最新的二进制日志，MHA可以将最新的二进制日志应用于其他所有的slave服务器上，因此可以保证所有节点的数据一致性。

目前MHA主要支持一主多从的架构，要搭建MHA,要求一个复制集群中必须最少有三台数据库服务器，一主二从，即一台充当master，一台充当备用master，另外一台充当从库。

三台机器是相对比较可靠的一个集群。

ip 规划如下：

192.168.56.101 node1 (mha manager)

192.168.56.102 node2 (mysql master)
192.168.56.103 node3 (mysql master candicate)
192.168.56.104 node4 (mysql slave)

os的ssh免密登录

详细过称略，参考另外一篇博客。

安装好mysql 5.7

# wget https://dev.mysql.com/get/mysql80-community-release-el7-1.noarch.rpm
# yum install mysql-community-client mysql-community-common mysql-community-devel mysql-community-embedded mysql-community-embedded-compat mysql-community-embedded-devel mysql-community-libs mysql-community-libs-compat mysql-community-minimal-debuginfo mysql-community-server mysql-community-test

详细过称略

初始化mysql 5.7，配置好master slave

注意每个节点的 binlog-do-db 和 replicate-ignore-db 设置必须相同，此处这两个参数都为空，没有设置。

master 修改密码

mysql> set password for 'root'@'localhost'= password('1qaz@WSX');

master 创建监控用户

mysql> create user 'mha_mon'@'192.168.56.%' identified by '2wsx3edc';
mysql> grant all privileges on *.* to 'mha_mon'@'192.168.56.%';
mysql> flush privileges;

master 创建复制用户

mysql> create user 'replicator'@'192.168.56.%' identified by '2wsx3edc';
mysql> grant replication slave on *.* to 'replicator'@'192.168.56.%';
mysql> flush privileges;

master 使用 mysqldump 出集合,再在slave端导入

# mysqldump -uroot -p --master-data=2 --single-transaction -R --triggers -A > mysql_all.sql

其中
–master-data=2代表备份时刻记录master的Binlog位置和Position,
–single-transaction意思是获取一致性快照，
-R意思是备份存储过程和函数，
–triggres的意思是备份触发器，
-A代表备份所有的库。

mysql> change master to
  master_host='192.168.56.102',
  master_user='replicator',
  master_password='2wsx3edc',
  master_port=3306,
  master_log_file='mysql-bin.000005',
  master_log_pos=1099;
mysql> start slave;

salve设置为 read only，从库对外提供读服务，只所以没有写进配置文件，是因为随时slave会提升为master。

mysql> set global read_only=1;

至此，已经配置好了1master、2slave

安装 mha node

在node2、node3、node4分别安装 mha node 软件 mha4mysql-node-0.58

# yum install perl-CPAN perl-DBD-MySQL perl-Config-Tiny perl-Email-Date-Format perl-Log-Dispatch perl-Mail-Sender  perl-Mail-Sendmail perl-MIME-Lite perl-MIME-Types perl-Parallel-ForkManager perl-Params-Validate perl-Time-HiRes

# cd /tmp/mha
# wget https://github.com/yoshinorim/mha4mysql-node/releases/download/v0.58/mha4mysql-node-0.58.tar.gz
# tar -zxvf mha4mysql-node-0.58.tar.gz
# cd mha4mysql-node-0.58
# perl Makefile.PL
# make 
# make install

安装好后会存在如下文件

# cd /usr/local/bin/
# ls -l
total 48
-r-xr-xr-x 1 root root 17639 Aug  6 13:32 apply_diff_relay_logs
-r-xr-xr-x 1 root root  4807 Aug  6 13:32 filter_mysqlbinlog
-r-xr-xr-x 1 root root  8337 Aug  6 13:32 purge_relay_logs
-r-xr-xr-x 1 root root  7525 Aug  6 13:32 save_binary_logs

简单描述下这几个文件
apply_diff_relay_logs 识别差异的中继日志事件并将其差异的事件应用于其他的slave
filter_mysqlbinlog 去除不必要的ROLLBACK事件（MHA已不再使用这个工具）
purge_relay_logs 清除中继日志（不会阻塞SQL线程）
save_binary_logs 保存和复制master的二进制日志

安装 mha manager

node1 节点安装 mha4mysql-manager-0.58,在安装 mha manager 之前，必须要先安装 mha node，参照上面完成安装后，再进行下面的操作。

# cd /tmp/mha
# wget https://github.com/yoshinorim/mha4mysql-manager/releases/download/v0.58/mha4mysql-manager-0.58.tar.gz
# tar -zxvf mha4mysql-manager-0.58.tar.gz
# cd mha4mysql-manager-0.58
# perl Makefile.PL
# make 
# make install

安装好后会存在如下文件

# cd /usr/local/bin
# ls -l |grep -i master
-r-xr-xr-x 1 root root  1995 Aug  6 14:28 masterha_check_repl
-r-xr-xr-x 1 root root  1779 Aug  6 14:28 masterha_check_ssh
-r-xr-xr-x 1 root root  1865 Aug  6 14:28 masterha_check_status
-r-xr-xr-x 1 root root  3201 Aug  6 14:28 masterha_conf_host
-r-xr-xr-x 1 root root  2517 Aug  6 14:28 masterha_manager
-r-xr-xr-x 1 root root  2165 Aug  6 14:28 masterha_master_monitor
-r-xr-xr-x 1 root root  2373 Aug  6 14:28 masterha_master_switch
-r-xr-xr-x 1 root root  5172 Aug  6 14:28 masterha_secondary_check
-r-xr-xr-x 1 root root  1739 Aug  6 14:28 masterha_stop

简单描述下这几个文件
masterha_check_repl 检查MySQL复制状况
masterha_check_ssh 检查MHA的SSH配置状况
masterha_check_status 检测当前MHA运行状态
masterha_conf_host 添加或删除配置的server信息
masterha_manger 启动MHA
masterha_master_monitor 检测master是否宕机
masterha_master_switch 控制故障转移（自动或者手动）

拷贝一些有用的脚本到 /usr/local/bin

# cd /tmp/mha/mha4mysql-manager-0.58/samples/scripts
# ls -l
total 32
-rwxr-xr-x 1 peiyb peiyb  3648 Mar 23 05:53 master_ip_failover
-rwxr-xr-x 1 peiyb peiyb  9870 Mar 23 05:53 master_ip_online_change
-rwxr-xr-x 1 peiyb peiyb 11867 Mar 23 05:53 power_manager
-rwxr-xr-x 1 peiyb peiyb  1360 Mar 23 05:53 send_report

# cp ./* /usr/local/bin

简单描述下这几个文件
master_ip_failover 自动切换时vip管理的脚本，不是必须
master_ip_online_change 在线切换时vip的管理，不是必须
power_manager 故障发生后关闭主机的脚本，不是必须
send_report 因故障切换后发送报警的脚本，不是必须

配置

创建mha工作目录

# mkdir -p /etc/masterha
# cp /tmp/mha/mha4mysql-manager-0.58/samples/conf/* /etc/masterha/

app1.cnf 的一些内容

mha manager 节点上

# vi /etc/masterha/app1.cnf 

[server default]
#manager的工作目录
manager_workdir=/etc/masterha/app1
#manager的日志
manager_log=/var/log/masterha/app1-manager.log
#master 保存binlog的位置，以便MHA可以找到master的日志
master_binlog_dir=/var/lib/mysql
#自动failover时候的切换脚本
master_ip_failover_script= /usr/local/bin/master_ip_failover
#手动切换时候的切换脚本
master_ip_online_change_script= /usr/local/bin/master_ip_online_change
#mysql中创建的监控用户的密码
password=2wsx3edc
#mysql中创建的监控用户
user=mha_mon
#设置监控主库，发送ping包的时间间隔，默认是3秒，尝试三次没有回应的时候自动进行railover            
ping_interval=5
#远端mysql在发生切换时binlog的保存位置
remote_workdir=/tmp
#mysql复制用户的密码
repl_password=2wsx3edc
#mysql复制用户名
repl_user=replicator
#发生切换后发送的报警的脚本
report_script=/usr/local/send_report

#MHA到node2的监控之间出现问题，MHA Manager将会尝试从node3登录到node2
secondary_check_script=/usr/local/bin/masterha_secondary_check -s node3 -s node2 

#故障发生后关闭故障主机脚本
shutdown_script=""
#ssh的登录用户名,需事先配置好ssh免密登录
ssh_user=root

[server1]
#master 信息
hostname=192.168.56.102
port=3306

[server2]
#candidate master 信息
hostname=192.168.56.103
port=3306
#候选master，如果设置该参数以后，发生主从切换以后将会将此从库提升为主库，即使这个主库不是集群中事件最新的slave
candidate_master=1
#默认情况下如果一个slave落后master 100M的relay logs的话，MHA将不会选择该slave作为一个新的master，
#因为对于这个slave的恢复需要花费很长时间，通过设置check_repl_delay=0,MHA触发切换在选择一个新的master的时候将会忽略复制延时，
#这个参数对于设置了candidate_master=1的主机非常有用，因为这个候选主在切换的过程中一定是新的master
check_repl_delay=0   

[server3]
#slave 信息
hostname=192.168.56.104
port=3306

relay log 的一些内容

在每个slave上执行,将relay log的自动设置清除为OFF

mysql> set global relay_log_purge=0;

pure_relay_logs 的一些内容

每个slave上操作

# vi /usr/local/bin/purge_relay_log.sh 

#!/bin/bash
user=root
passwd=1qaz@WSX
port=3306
log_dir='/var/log/masterha'
work_dir='/var/lib'
purge='/usr/local/bin/purge_relay_logs'

if [ ! -d $log_dir ]
then
   mkdir $log_dir -p
fi

$purge --user=$user --password=$passwd --disable_relay_log_purge --port=$port --workdir=$work_dir >> $log_dir/purge_relay_logs1.log 2>&1


# crontab -l
0 5 * * * /bin/bash /usr/local/bin/purge_relay_log.sh

purge_relay_logs脚本删除中继日志不会阻塞SQL线程。

检查

检查MHA Manger到所有MHA Node的SSH连接

# masterha_check_ssh --conf=/etc/masterha/app1.cnf 

Mon Aug  6 19:00:29 2018 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Mon Aug  6 19:00:29 2018 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Mon Aug  6 19:00:29 2018 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Mon Aug  6 19:00:29 2018 - [info] Starting SSH connection tests..
Mon Aug  6 19:00:30 2018 - [debug] 
Mon Aug  6 19:00:29 2018 - [debug]  Connecting via SSH from root@192.168.56.102(192.168.56.102:22) to root@192.168.56.103(192.168.56.103:22)..
Mon Aug  6 19:00:29 2018 - [debug]   ok.
Mon Aug  6 19:00:29 2018 - [debug]  Connecting via SSH from root@192.168.56.102(192.168.56.102:22) to root@192.168.56.104(192.168.56.104:22)..
Mon Aug  6 19:00:29 2018 - [debug]   ok.
Mon Aug  6 19:00:30 2018 - [debug] 
Mon Aug  6 19:00:29 2018 - [debug]  Connecting via SSH from root@192.168.56.103(192.168.56.103:22) to root@192.168.56.102(192.168.56.102:22)..
Mon Aug  6 19:00:29 2018 - [debug]   ok.
Mon Aug  6 19:00:29 2018 - [debug]  Connecting via SSH from root@192.168.56.103(192.168.56.103:22) to root@192.168.56.104(192.168.56.104:22)..
Mon Aug  6 19:00:30 2018 - [debug]   ok.
Mon Aug  6 19:00:31 2018 - [debug] 
Mon Aug  6 19:00:30 2018 - [debug]  Connecting via SSH from root@192.168.56.104(192.168.56.104:22) to root@192.168.56.102(192.168.56.102:22)..
Mon Aug  6 19:00:30 2018 - [debug]   ok.
Mon Aug  6 19:00:30 2018 - [debug]  Connecting via SSH from root@192.168.56.104(192.168.56.104:22) to root@192.168.56.103(192.168.56.103:22)..
Mon Aug  6 19:00:30 2018 - [debug]   ok.
Mon Aug  6 19:00:31 2018 - [info] All SSH connection tests passed successfully.

看到最后一行就开心了。

检查整个复制环境

先注销掉这几个参数，后面再仔细研究实现

# vi /etc/masterha/app1.cnf

#master_ip_failover_script= /usr/local/bin/master_ip_failover
#master_ip_online_change_script= /usr/local/bin/master_ip_online_change
#report_script=/usr/local/send_report

# masterha_check_repl --conf=/etc/masterha/app1.cnf

Mon Aug  6 22:34:07 2018 - [info] Reading default configuration from /etc/masterha_default.cnf..
Mon Aug  6 22:34:07 2018 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Mon Aug  6 22:34:07 2018 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Mon Aug  6 22:34:07 2018 - [info] MHA::MasterMonitor version 0.58.
Mon Aug  6 22:34:08 2018 - [info] GTID failover mode = 0
Mon Aug  6 22:34:08 2018 - [info] Dead Servers:
Mon Aug  6 22:34:08 2018 - [info] Alive Servers:
Mon Aug  6 22:34:08 2018 - [info]   192.168.56.102(192.168.56.102:3306)
Mon Aug  6 22:34:08 2018 - [info]   192.168.56.103(192.168.56.103:3306)
Mon Aug  6 22:34:08 2018 - [info]   192.168.56.104(192.168.56.104:3306)
Mon Aug  6 22:34:08 2018 - [info] Alive Slaves:
Mon Aug  6 22:34:08 2018 - [info]   192.168.56.103(192.168.56.103:3306)  Version=5.7.22-log (oldest major version between slaves) log-bin:enabled
Mon Aug  6 22:34:08 2018 - [info]     Replicating from 192.168.56.102(192.168.56.102:3306)
Mon Aug  6 22:34:08 2018 - [info]   192.168.56.104(192.168.56.104:3306)  Version=5.7.23-log (oldest major version between slaves) log-bin:enabled
Mon Aug  6 22:34:08 2018 - [info]     Replicating from 192.168.56.102(192.168.56.102:3306)
Mon Aug  6 22:34:08 2018 - [info] Current Alive Master: 192.168.56.102(192.168.56.102:3306)
Mon Aug  6 22:34:08 2018 - [info] Checking slave configurations..
Mon Aug  6 22:34:08 2018 - [info] Checking replication filtering settings..
Mon Aug  6 22:34:08 2018 - [info]  binlog_do_db= , binlog_ignore_db= 
Mon Aug  6 22:34:08 2018 - [info]  Replication filtering check ok.
Mon Aug  6 22:34:08 2018 - [info] GTID (with auto-pos) is not supported
Mon Aug  6 22:34:08 2018 - [info] Starting SSH connection tests..
Mon Aug  6 22:34:10 2018 - [info] All SSH connection tests passed successfully.
Mon Aug  6 22:34:10 2018 - [info] Checking MHA Node version..
Mon Aug  6 22:34:11 2018 - [info]  Version check ok.
Mon Aug  6 22:34:11 2018 - [info] Checking SSH publickey authentication settings on the current master..
Mon Aug  6 22:34:11 2018 - [info] HealthCheck: SSH to 192.168.56.102 is reachable.
Mon Aug  6 22:34:11 2018 - [info] Master MHA Node version is 0.58.
Mon Aug  6 22:34:11 2018 - [info] Checking recovery script configurations on 192.168.56.102(192.168.56.102:3306)..
Mon Aug  6 22:34:11 2018 - [info]   Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/var/lib/mysql --output_file=/tmp/save_binary_logs_test --manager_version=0.58 --start_file=mysql-bin.000005 
Mon Aug  6 22:34:11 2018 - [info]   Connecting to root@192.168.56.102(192.168.56.102:22).. 
  Creating /tmp if not exists..    ok.
  Checking output directory is accessible or not..
   ok.
  Binlog found at /var/lib/mysql, up to mysql-bin.000005
Mon Aug  6 22:34:11 2018 - [info] Binlog setting check done.
Mon Aug  6 22:34:11 2018 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Mon Aug  6 22:34:11 2018 - [info]   Executing command : apply_diff_relay_logs --command=test --slave_user='mha_mon' --slave_host=192.168.56.103 --slave_ip=192.168.56.103 --slave_port=3306 --workdir=/tmp --target_version=5.7.22-log --manager_version=0.58 --relay_log_info=/var/lib/mysql/relay-log.info  --relay_dir=/var/lib/mysql/  --slave_pass=xxx
Mon Aug  6 22:34:11 2018 - [info]   Connecting to root@192.168.56.103(192.168.56.103:22).. 
  Checking slave recovery environment settings..
    Opening /var/lib/mysql/relay-log.info ... ok.
    Relay log found at /var/lib/mysql, up to node3-relay-bin.000002
    Temporary relay log file is /var/lib/mysql/node3-relay-bin.000002
    Checking if super_read_only is defined and turned on.. not present or turned off, ignoring.
    Testing mysql connection and privileges..
mysql: [Warning] Using a password on the command line interface can be insecure.
 done.
    Testing mysqlbinlog output.. done.
    Cleaning up test file(s).. done.
Mon Aug  6 22:34:12 2018 - [info]   Executing command : apply_diff_relay_logs --command=test --slave_user='mha_mon' --slave_host=192.168.56.104 --slave_ip=192.168.56.104 --slave_port=3306 --workdir=/tmp --target_version=5.7.23-log --manager_version=0.58 --relay_log_info=/var/lib/mysql/relay-log.info  --relay_dir=/var/lib/mysql/  --slave_pass=xxx
Mon Aug  6 22:34:12 2018 - [info]   Connecting to root@192.168.56.104(192.168.56.104:22).. 
  Checking slave recovery environment settings..
    Opening /var/lib/mysql/relay-log.info ... ok.
    Relay log found at /var/lib/mysql, up to node4-relay-bin.000005
    Temporary relay log file is /var/lib/mysql/node4-relay-bin.000005
    Checking if super_read_only is defined and turned on.. not present or turned off, ignoring.
    Testing mysql connection and privileges..
mysql: [Warning] Using a password on the command line interface can be insecure.
 done.
    Testing mysqlbinlog output.. done.
    Cleaning up test file(s).. done.
Mon Aug  6 22:34:12 2018 - [info] Slaves settings check done.
Mon Aug  6 22:34:12 2018 - [info] 
192.168.56.102(192.168.56.102:3306) (current master)
 +--192.168.56.103(192.168.56.103:3306)
 +--192.168.56.104(192.168.56.104:3306)

Mon Aug  6 22:34:12 2018 - [info] Checking replication health on 192.168.56.103..
Mon Aug  6 22:34:12 2018 - [info]  ok.
Mon Aug  6 22:34:12 2018 - [info] Checking replication health on 192.168.56.104..
Mon Aug  6 22:34:12 2018 - [info]  ok.
Mon Aug  6 22:34:12 2018 - [warning] master_ip_failover_script is not defined.
Mon Aug  6 22:34:12 2018 - [warning] shutdown_script is not defined.
Mon Aug  6 22:34:12 2018 - [info] Got exit code 0 (Not master dead).

MySQL Replication Health is OK.

仔细观察后，出现两个 warning，先不理会。

Mon Aug  6 22:37:53 2018 - [warning] master_ip_failover_script is not defined.
Mon Aug  6 22:37:53 2018 - [warning] shutdown_script is not defined.

此处顺带搜索了常用的一些错误 https://www.cnblogs.com/polestar/p/5371080.html

启动 mha manager

启动 manager

# nohup masterha_manager --conf=/etc/masterha/app1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/masterha/app1-manager.log 2>&1 &

查看 manager status

# masterha_check_status --conf=/etc/masterha/app1.cnf
app1 (pid:17385) is running(0:PING_OK), master:192.168.56.102

查看 manager log

# tail -n 1000 -f /var/log/masterha/app1-manager.log

Mon Aug  6 22:45:38 2018 - [info] Reading default configuration from /etc/masterha_default.cnf..
Mon Aug  6 22:45:38 2018 - [info] Reading application default configuration from /etc/masterha/app1.cnf..
Mon Aug  6 22:45:38 2018 - [info] Reading server configuration from /etc/masterha/app1.cnf..
Mon Aug  6 22:45:38 2018 - [info] MHA::MasterMonitor version 0.58.
Mon Aug  6 22:45:39 2018 - [info] GTID failover mode = 0
Mon Aug  6 22:45:39 2018 - [info] Dead Servers:
Mon Aug  6 22:45:39 2018 - [info] Alive Servers:
Mon Aug  6 22:45:39 2018 - [info]   192.168.56.102(192.168.56.102:3306)
Mon Aug  6 22:45:39 2018 - [info]   192.168.56.103(192.168.56.103:3306)
Mon Aug  6 22:45:39 2018 - [info]   192.168.56.104(192.168.56.104:3306)
Mon Aug  6 22:45:39 2018 - [info] Alive Slaves:
Mon Aug  6 22:45:39 2018 - [info]   192.168.56.103(192.168.56.103:3306)  Version=5.7.22-log (oldest major version between slaves) log-bin:enabled
Mon Aug  6 22:45:39 2018 - [info]     Replicating from 192.168.56.102(192.168.56.102:3306)
Mon Aug  6 22:45:39 2018 - [info]   192.168.56.104(192.168.56.104:3306)  Version=5.7.23-log (oldest major version between slaves) log-bin:enabled
Mon Aug  6 22:45:39 2018 - [info]     Replicating from 192.168.56.102(192.168.56.102:3306)
Mon Aug  6 22:45:39 2018 - [info] Current Alive Master: 192.168.56.102(192.168.56.102:3306)
Mon Aug  6 22:45:39 2018 - [info] Checking slave configurations..
Mon Aug  6 22:45:39 2018 - [info] Checking replication filtering settings..
Mon Aug  6 22:45:39 2018 - [info]  binlog_do_db= , binlog_ignore_db= 
Mon Aug  6 22:45:39 2018 - [info]  Replication filtering check ok.
Mon Aug  6 22:45:39 2018 - [info] GTID (with auto-pos) is not supported
Mon Aug  6 22:45:39 2018 - [info] Starting SSH connection tests..
Mon Aug  6 22:45:42 2018 - [info] All SSH connection tests passed successfully.
Mon Aug  6 22:45:42 2018 - [info] Checking MHA Node version..
Mon Aug  6 22:45:42 2018 - [info]  Version check ok.
Mon Aug  6 22:45:42 2018 - [info] Checking SSH publickey authentication settings on the current master..
Mon Aug  6 22:45:42 2018 - [info] HealthCheck: SSH to 192.168.56.102 is reachable.
Mon Aug  6 22:45:43 2018 - [info] Master MHA Node version is 0.58.
Mon Aug  6 22:45:43 2018 - [info] Checking recovery script configurations on 192.168.56.102(192.168.56.102:3306)..
Mon Aug  6 22:45:43 2018 - [info]   Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/var/lib/mysql --output_file=/tmp/save_binary_logs_test --manager_version=0.58 --start_file=mysql-bin.000005 
Mon Aug  6 22:45:43 2018 - [info]   Connecting to root@192.168.56.102(192.168.56.102:22).. 
  Creating /tmp if not exists..    ok.
  Checking output directory is accessible or not..
   ok.
  Binlog found at /var/lib/mysql, up to mysql-bin.000005
Mon Aug  6 22:45:43 2018 - [info] Binlog setting check done.
Mon Aug  6 22:45:43 2018 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..
Mon Aug  6 22:45:43 2018 - [info]   Executing command : apply_diff_relay_logs --command=test --slave_user='mha_mon' --slave_host=192.168.56.103 --slave_ip=192.168.56.103 --slave_port=3306 --workdir=/tmp --target_version=5.7.22-log --manager_version=0.58 --relay_log_info=/var/lib/mysql/relay-log.info  --relay_dir=/var/lib/mysql/  --slave_pass=xxx
Mon Aug  6 22:45:43 2018 - [info]   Connecting to root@192.168.56.103(192.168.56.103:22).. 
  Checking slave recovery environment settings..
    Opening /var/lib/mysql/relay-log.info ... ok.
    Relay log found at /var/lib/mysql, up to node3-relay-bin.000002
    Temporary relay log file is /var/lib/mysql/node3-relay-bin.000002
    Checking if super_read_only is defined and turned on.. not present or turned off, ignoring.
    Testing mysql connection and privileges..
mysql: [Warning] Using a password on the command line interface can be insecure.
 done.
    Testing mysqlbinlog output.. done.
    Cleaning up test file(s).. done.
Mon Aug  6 22:45:43 2018 - [info]   Executing command : apply_diff_relay_logs --command=test --slave_user='mha_mon' --slave_host=192.168.56.104 --slave_ip=192.168.56.104 --slave_port=3306 --workdir=/tmp --target_version=5.7.23-log --manager_version=0.58 --relay_log_info=/var/lib/mysql/relay-log.info  --relay_dir=/var/lib/mysql/  --slave_pass=xxx
Mon Aug  6 22:45:43 2018 - [info]   Connecting to root@192.168.56.104(192.168.56.104:22).. 
  Checking slave recovery environment settings..
    Opening /var/lib/mysql/relay-log.info ... ok.
    Relay log found at /var/lib/mysql, up to node4-relay-bin.000005
    Temporary relay log file is /var/lib/mysql/node4-relay-bin.000005
    Checking if super_read_only is defined and turned on.. not present or turned off, ignoring.
    Testing mysql connection and privileges..
mysql: [Warning] Using a password on the command line interface can be insecure.
 done.
    Testing mysqlbinlog output.. done.
    Cleaning up test file(s).. done.
Mon Aug  6 22:45:44 2018 - [info] Slaves settings check done.
Mon Aug  6 22:45:44 2018 - [info] 
192.168.56.102(192.168.56.102:3306) (current master)
 +--192.168.56.103(192.168.56.103:3306)
 +--192.168.56.104(192.168.56.104:3306)

Mon Aug  6 22:45:44 2018 - [warning] master_ip_failover_script is not defined.
Mon Aug  6 22:45:44 2018 - [warning] shutdown_script is not defined.
Mon Aug  6 22:45:44 2018 - [info] Set master ping interval 5 seconds.
Mon Aug  6 22:45:44 2018 - [info] Set secondary check script: /usr/local/bin/masterha_secondary_check -s node3 -s node2
Mon Aug  6 22:45:44 2018 - [info] Starting ping health check on 192.168.56.102(192.168.56.102:3306)..
Mon Aug  6 22:45:44 2018 - [info] Ping(SELECT) succeeded, waiting until MySQL doesn't respond..

关闭 mha manager

关闭 manager

# masterha_stop --conf=/etc/masterha/app1.cnf
Stopped app1 successfully.

查看 manager log

# tail -n 1000 -f /var/log/masterha/app1-manager.log

Mon Aug  6 22:52:54 2018 - [info] Got terminate signal. Exit.

会追加一条退出日志。

至此，mysql mha 就算安装好了，后面会继续就 mha的使用写一些blog。

参考：
https://github.com/yoshinorim/mha4mysql-manager/releases
https://github.com/yoshinorim/mha4mysql-node/releases

https://github.com/yoshinorim/mha4mysql-manager/wiki
https://github.com/yoshinorim/mha4mysql-manager/wiki/Parameters

https://github.com/yoshinorim/mha4mysql-manager
https://github.com/yoshinorim/mha4mysql-node

https://code.google.com/p/mysql-master-ha/
https://code.google.com/archive/p/mysql-master-ha/
https://code.google.com/archive/p/mysql-master-ha/downloads

下面是google官网对 mha 的概要描述

MHA for MySQL: Master High Availability Manager and tools for MySQL

A primary objective of MHA is automating master failover and slave promotion within short (usually 10-30 seconds) downtime, without suffering from replication consistency problems, without spending money for lots of new servers, without performance penalty, without complexity (easy-to-install), and without changing existing deployments.

MHA also provides a way for scheduled online master switch: changing currently running master to a new master safely, within a few seconds (0.5-2 seconds) of downtime (blocking writes only).

MHA provides the following functionality, and can be useful in many deployments where requirements such as high availability, data integrity, almost non-stop master maintenance are desired. * Automated master monitoring and failover MHA has a functionality to monitor MySQL master in an existing replication environment, detecting master failure, and doing master failover automatically. Even though some of slaves have not received the latest relay log events, MHA automatically identifies differential relay log events from the latest slave, and applies differential events to other slaves. So all slaves can be consistent. MHA normally can do failover in seconds (9-12 seconds to detect master failure, optionally 7-10 seconds to power off the master machine to avoid split brain, a few seconds for applying differential relay logs to the new master, so total downtime is normally 10-30 seconds). In addition, you can define a specific slave as a candidate master (setting priorities) in a configuration file. Since MHA fixes consistencies between slaves, you can promote any slave to a new master and consistency problems (which might cause sudden replication failure) will not happen. * Interactive (manual) Master Failover You can also use MHA for just failover, not for monitoring master. You can use MHA for master failover interactively. * Non-interactive master failover Non-interactive master failover (not monitoring master, but doing failover automatically) is also supported. This feature is useful especially when you have already used a software that monitors MySQL master. For example, you can use Pacemaker(Heartbeat) for detecting master failure and virtual ip address takeover, and use MHA for master failover and slave promotion. * Online switching master to a different host In many cases, it is necessary to migrate an existing master to a different machine (i.e. the current master has H/W problems on RAID controller or RAM, you want to replace with faster machine, etc). This is not a master crash, but scheduled master maintenance is needed to do that. Scheduled master maintenance causes downtime (at least you can not write master) so should be done as quickly as possible. On the other hand, you should block/kill current running sessions very carefully because consistency problems between different masters might happen (i.e "updating master1, updating master 2, committing master1, getting error on committing master 2" will result in data inconsistency). Both fast master switch and graceful blocking writes are required. MHA provides a way to do that. You can switch master gracefully within 0.5-2 seconds of writer block. In many cases 0.5-2 seconds of writer downtime is acceptable and you can switch master even without allocating scheduled maintenance window. This means you can take actions such as upgrading to higher versions, faster machine, etc much more easily.

mha4mysql-node-0.58 的一些输出

[root@node4 mha4mysql-node-0.58]# perl Makefile.PL
*** Module::AutoInstall version 1.06
*** Checking for Perl dependencies...
[Core Features]
- DBI        ...loaded. (1.627)
- DBD::mysql ...loaded. (4.023)
*** Module::AutoInstall configuration finished.
Checking if your kit is complete...
Looks good
Writing Makefile for mha4mysql::node


[root@node4 mha4mysql-node-0.58]# make
cp lib/MHA/BinlogManager.pm blib/lib/MHA/BinlogManager.pm
cp lib/MHA/BinlogPosFindManager.pm blib/lib/MHA/BinlogPosFindManager.pm
cp lib/MHA/BinlogPosFinderXid.pm blib/lib/MHA/BinlogPosFinderXid.pm
cp lib/MHA/BinlogHeaderParser.pm blib/lib/MHA/BinlogHeaderParser.pm
cp lib/MHA/BinlogPosFinder.pm blib/lib/MHA/BinlogPosFinder.pm
cp lib/MHA/BinlogPosFinderElp.pm blib/lib/MHA/BinlogPosFinderElp.pm
cp lib/MHA/NodeUtil.pm blib/lib/MHA/NodeUtil.pm
cp lib/MHA/SlaveUtil.pm blib/lib/MHA/SlaveUtil.pm
cp lib/MHA/NodeConst.pm blib/lib/MHA/NodeConst.pm
cp bin/filter_mysqlbinlog blib/script/filter_mysqlbinlog
/usr/bin/perl "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/filter_mysqlbinlog
cp bin/apply_diff_relay_logs blib/script/apply_diff_relay_logs
/usr/bin/perl "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/apply_diff_relay_logs
cp bin/purge_relay_logs blib/script/purge_relay_logs
/usr/bin/perl "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/purge_relay_logs
cp bin/save_binary_logs blib/script/save_binary_logs
/usr/bin/perl "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/save_binary_logs
Manifying blib/man1/filter_mysqlbinlog.1
Manifying blib/man1/apply_diff_relay_logs.1
Manifying blib/man1/purge_relay_logs.1
Manifying blib/man1/save_binary_logs.1


[root@node4 mha4mysql-node-0.58]# make install
Installing /usr/local/share/perl5/MHA/BinlogManager.pm
Installing /usr/local/share/perl5/MHA/BinlogPosFindManager.pm
Installing /usr/local/share/perl5/MHA/BinlogPosFinderXid.pm
Installing /usr/local/share/perl5/MHA/BinlogHeaderParser.pm
Installing /usr/local/share/perl5/MHA/BinlogPosFinder.pm
Installing /usr/local/share/perl5/MHA/BinlogPosFinderElp.pm
Installing /usr/local/share/perl5/MHA/NodeUtil.pm
Installing /usr/local/share/perl5/MHA/SlaveUtil.pm
Installing /usr/local/share/perl5/MHA/NodeConst.pm
Installing /usr/local/share/man/man1/filter_mysqlbinlog.1
Installing /usr/local/share/man/man1/apply_diff_relay_logs.1
Installing /usr/local/share/man/man1/purge_relay_logs.1
Installing /usr/local/share/man/man1/save_binary_logs.1
Installing /usr/local/bin/filter_mysqlbinlog
Installing /usr/local/bin/apply_diff_relay_logs
Installing /usr/local/bin/purge_relay_logs
Installing /usr/local/bin/save_binary_logs
Appending installation info to /usr/lib64/perl5/perllocal.pod

mha4mysql-manager-0.58 的一些输出

[root@node1 mha4mysql-manager-0.58]# perl Makefile.PL
*** Module::AutoInstall version 1.06
*** Checking for Perl dependencies...
[Core Features]
- DBI                   ...loaded. (1.627)
- DBD::mysql            ...loaded. (4.023)
- Time::HiRes           ...loaded. (1.9725)
- Config::Tiny          ...loaded. (2.14)
- Log::Dispatch         ...loaded. (2.41)
- Parallel::ForkManager ...loaded. (1.18)
- MHA::NodeConst        ...loaded. (0.58)
*** Module::AutoInstall configuration finished.
Checking if your kit is complete...
Looks good
Writing Makefile for mha4mysql::manager


[root@node1 mha4mysql-manager-0.58]# make 
cp lib/MHA/ManagerUtil.pm blib/lib/MHA/ManagerUtil.pm
cp lib/MHA/Config.pm blib/lib/MHA/Config.pm
cp lib/MHA/HealthCheck.pm blib/lib/MHA/HealthCheck.pm
cp lib/MHA/ServerManager.pm blib/lib/MHA/ServerManager.pm
cp lib/MHA/ManagerConst.pm blib/lib/MHA/ManagerConst.pm
cp lib/MHA/FileStatus.pm blib/lib/MHA/FileStatus.pm
cp lib/MHA/ManagerAdmin.pm blib/lib/MHA/ManagerAdmin.pm
cp lib/MHA/ManagerAdminWrapper.pm blib/lib/MHA/ManagerAdminWrapper.pm
cp lib/MHA/MasterFailover.pm blib/lib/MHA/MasterFailover.pm
cp lib/MHA/MasterRotate.pm blib/lib/MHA/MasterRotate.pm
cp lib/MHA/MasterMonitor.pm blib/lib/MHA/MasterMonitor.pm
cp lib/MHA/Server.pm blib/lib/MHA/Server.pm
cp lib/MHA/SSHCheck.pm blib/lib/MHA/SSHCheck.pm
cp lib/MHA/DBHelper.pm blib/lib/MHA/DBHelper.pm
cp bin/masterha_stop blib/script/masterha_stop
/usr/bin/perl "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/masterha_stop
cp bin/masterha_conf_host blib/script/masterha_conf_host
/usr/bin/perl "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/masterha_conf_host
cp bin/masterha_check_repl blib/script/masterha_check_repl
/usr/bin/perl "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/masterha_check_repl
cp bin/masterha_check_status blib/script/masterha_check_status
/usr/bin/perl "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/masterha_check_status
cp bin/masterha_master_monitor blib/script/masterha_master_monitor
/usr/bin/perl "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/masterha_master_monitor
cp bin/masterha_check_ssh blib/script/masterha_check_ssh
/usr/bin/perl "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/masterha_check_ssh
cp bin/masterha_master_switch blib/script/masterha_master_switch
/usr/bin/perl "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/masterha_master_switch
cp bin/masterha_secondary_check blib/script/masterha_secondary_check
/usr/bin/perl "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/masterha_secondary_check
cp bin/masterha_manager blib/script/masterha_manager
/usr/bin/perl "-Iinc" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/masterha_manager
Manifying blib/man1/masterha_stop.1
Manifying blib/man1/masterha_conf_host.1
Manifying blib/man1/masterha_check_repl.1
Manifying blib/man1/masterha_check_status.1
Manifying blib/man1/masterha_master_monitor.1
Manifying blib/man1/masterha_check_ssh.1
Manifying blib/man1/masterha_master_switch.1
Manifying blib/man1/masterha_secondary_check.1
Manifying blib/man1/masterha_manager.1


[root@node1 mha4mysql-manager-0.58]# make install
Installing /usr/local/share/perl5/MHA/ManagerUtil.pm
Installing /usr/local/share/perl5/MHA/Config.pm
Installing /usr/local/share/perl5/MHA/HealthCheck.pm
Installing /usr/local/share/perl5/MHA/ServerManager.pm
Installing /usr/local/share/perl5/MHA/ManagerConst.pm
Installing /usr/local/share/perl5/MHA/FileStatus.pm
Installing /usr/local/share/perl5/MHA/ManagerAdmin.pm
Installing /usr/local/share/perl5/MHA/ManagerAdminWrapper.pm
Installing /usr/local/share/perl5/MHA/MasterFailover.pm
Installing /usr/local/share/perl5/MHA/MasterRotate.pm
Installing /usr/local/share/perl5/MHA/MasterMonitor.pm
Installing /usr/local/share/perl5/MHA/Server.pm
Installing /usr/local/share/perl5/MHA/SSHCheck.pm
Installing /usr/local/share/perl5/MHA/DBHelper.pm
Installing /usr/local/share/man/man1/masterha_stop.1
Installing /usr/local/share/man/man1/masterha_conf_host.1
Installing /usr/local/share/man/man1/masterha_check_repl.1
Installing /usr/local/share/man/man1/masterha_check_status.1
Installing /usr/local/share/man/man1/masterha_master_monitor.1
Installing /usr/local/share/man/man1/masterha_check_ssh.1
Installing /usr/local/share/man/man1/masterha_master_switch.1
Installing /usr/local/share/man/man1/masterha_secondary_check.1
Installing /usr/local/share/man/man1/masterha_manager.1
Installing /usr/local/bin/masterha_stop
Installing /usr/local/bin/masterha_conf_host
Installing /usr/local/bin/masterha_check_repl
Installing /usr/local/bin/masterha_check_status
Installing /usr/local/bin/masterha_master_monitor
Installing /usr/local/bin/masterha_check_ssh
Installing /usr/local/bin/masterha_master_switch
Installing /usr/local/bin/masterha_secondary_check
Installing /usr/local/bin/masterha_manager
Appending installation info to /usr/lib64/perl5/perllocal.pod

相关阅读:
Java应用开发与实践
 大话存储：存储系统底层架构原理极限剖析(终极版)
Excel 2016公式与函数应用大全
 Excel高效办公应用技巧
 业务弯路池子
 一个现象，
为什么有时候进入这么多次，一次是 38次，一次是 114次，
恶心从判断开始，
but,
这两个的意思是不同的。。。
原文地址：https://www.cnblogs.com/ctypyb2002/p/9792913.html

mysql 高可用架构 mha 之一 安装