一、哨兵模式原理
1、基本原理
（1）定时任务：每个哨兵节点维护了3个定时任务。定时任务的功能分别如下：通过向主节点发送info命令获取最新的主从结构；通过发布订阅功能获取其他哨兵节点的信息；通过向其他节点发送ping命令进行心跳检测，判断是否下线。
###向主节点发送info命令获取最新的主从结构### 10:06:08.746434 IP 192.168.0.91.39782 > 192.168.0.183.6379: Flags [P.], seq 168:182, ack 19, win 1424, options [nop,nop,TS val 1196253045 ecr 932478239], length 14: RESP "INFO" 10:06:08.747141 IP 192.168.0.183.6379 > 192.168.0.91.39782: Flags [P.], seq 19:3533, ack 182, win 1432, options [nop,nop,TS val 932478476 ecr 1196253045], length 3514: RESP "# Server^M^Jredis_version:5.0.12^M^Jredis_git_sha1:00000000^M^Jredis_git_dirty:0^M^Jredis_build_id:73d7f12b5b0c37b9^M^Jredis_mode:standalone^M^Jos:Linux 3.10.0-957.el7.x86_64 x86_64^M^Jarch_bits:64^M^Jmultiplexing_api:epoll^M^Jatomicvar_api:atomic-builtin^M^Jgcc_version:4.8.5^M^Jprocess_id:10795^M^Jrun_id:e5004ad6cf91960ff61e3e416a8ea500d440e1f1^M^Jtcp_port:6379^M^Juptime_in_seconds:61580^M^Juptime_in_days:0^M^Jhz:10^M^Jconfigured_hz:10^M^Jlru_clock:4902032^M^Jexecutable:/usr/local/redis/bin/redis-server^M^Jconfig_file:/usr/local/redis/bin/redis.conf^M^J^M^J# Clients^M^Jconnected_clients:4^M^Jclient_recent_max_input_buffer:2^M^Jclient_recent_max_output_buffer:0^M^Jblocked_clients:0^M^J^M^J# Memory^M^Jused_memory:2050184^M^Jused_memory_human:1.96M^M^Jused_memory_rss:3825664^M^Jused_memory_rss_human:3.65M^M^Jused_memory_peak:2132072^M^Jused_memory_peak_human:2.03M^M^Jused_memory_peak_perc:96.16%^M^Jused_memory_overhead:2007652^M^Jused_memory_startup:791440^M^Jused_memory_dataset:42532^M^Jused_memory_dataset_perc:3.38%^M^Jallocator_allocated:2015976^M^Jallocator_active:2293760^M^Jallocator_resident:4620288^M^Jtotal_system_memory:1019797504^M^Jtotal_system_memory_human:972.55M^M^Jused_memory_lua:37888^M^Jused_memory_lua_human:37.00K^M^Jused_memory_scripts:0^M^Jused_memory_scripts_human:0B^M^Jnumber_of_cached_scripts:0^M^Jmaxmemory:0^M^Jmaxmemory_human:0B^M^Jmaxmemory_policy:noeviction^M^Jallocator_frag_ratio:1.14^M^Jallocator_frag_bytes:277784^M^Jallocator_rss_ratio:2.01^M^Jallocator_rss_bytes:2326528^M^Jrss_overhead_ratio:0.83^M^Jrss_overhead_bytes:-794624^M^Jmem_fragmentation_ratio:1.94^M^Jmem_fragmentation_bytes:1858688^M^Jmem_not_counted_for_evict:0^M^Jmem_replication_backlog:1048576^M^Jmem_clients_slaves:66616^M^Jmem_clients_normal:100460^M^Jmem_aof_buffer:0^M^Jmem_allocator:jemalloc-5.1.0^M^Jactive_defrag_running:0^M^Jlazyfree_pending_objects:0^M^J^M^J# Persistence^M^Jloading:0^M^Jrdb_changes_since_last_save:0^M^Jrdb_bgsave_in_progress:0^M^Jrdb_last_save_time:1615453188^M^Jrdb_last_bgsave_status:ok^M^Jrdb_last_bgsave_time_sec:0^M^Jrdb_current_bgsave_time_sec:-1^M^Jrdb_last_cow_size:335872^M^Jaof_enabled:0^M^Jaof_rewrite_in_progress:0^M^Jaof_rewrite_scheduled:0^M^Jaof_last_rewrite_time_sec:-1^M^Jaof_current_rewrite_time_sec:-1^M^Jaof_last_bgrewrite_status:ok^M^Jaof_last_write_status:ok^M^Jaof_last_cow_size:0^M^J^M^J# Stats^M^Jtotal_connections_received:6832^M^Jtotal_commands_processed:295598^M^Jinstantaneous_ops_per_sec:4^M^Jtotal_net_input_bytes:16065006^M^Jtotal_net_output_bytes:72932454^M^Jinstantaneous_input_kbps:0.26^M^Jinstantaneous_output_kbps:0.69^M^Jrejected_connections:0^M^Jsync_full:2^M^Jsync_partial_ok:1^M^Jsync_partial_err:2^M^Jexpired_keys:0^M^Jexpired_stale_perc:0.00^M^Jexpired_time_cap_reached_count:0^M^Jevicted_keys:0^M^Jkeyspace_hits:0^M^Jkeyspace_misses:0^M^Jpubsub_channels:1^M^Jpubsub_patterns:0^M^Jlatest_fork_usec:252^M^Jmigrate_cached_sockets:0^M^Jslave_expires_tracked_keys:0^M^Jactive_defrag_hits:0^M^Jactive_defrag_misses:0^M^Jactive_defrag_key_hits:0^M^Jactive_defrag_key_misses:0^M^J^M^J# Replication^M^Jrole:master^M^Jconnected_slaves:2^M^Jslave0:ip=192.168.0.51,port=6379,state=online,offset=7863711,lag=0^M^Jslave1:ip=192.168.0.91,port=6379,state=online,offset=7863711,lag=0^M^Jmaster_replid:26f7b1b48361cf24d999eb2c81336068f1a70d21^M^Jmaster_replid2:0000000000000000000000000000000000000000^M^Jmaster_repl_offset:7863711^M^Jsecond_repl_offset:-1^M^Jrepl_backlog_active:1^M^Jrepl_backlog_size:1048576^M^Jrepl_backlog_first_byte_offset:6815136^M^Jrepl_backlog_histlen:1048576^M^J^M^J# CPU^M^Jused_cpu_sys:141.963877^M^Jused_cpu_user:3.311267^M^Jused_cpu_sys_children:0.005190^M^Jused_cpu_user_children:0.000827^M^J^M^J# Cluster^M^Jcluster_enabled:0^M^J^M^J# Keyspace^M^Jdb0:keys=10,expires=0,avg_ttl=0^M^J" ###通过发布订阅功能获取其他哨兵节点的信息### 10:06:10.434633 IP 192.168.0.183.6379 > 192.168.0.91.39784: Flags [P.], seq 420:560, ack 1, win 235, options [nop,nop,TS val 932480163 ecr 1196254438], length 140: RESP "message" "__sentinel__:hello" "192.168.0.51,26379,53d4d0aa6766c3c7b82e87db4707a046e8b697f7,1,mymaster,192.168.0.183,6379,0" ###通过向其他节点发送ping命令进行心跳检测### 10:06:09.535248 IP 192.168.0.91.39782 > 192.168.0.183.6379: Flags [P.], seq 182:196, ack 3533, win 1424, options [nop,nop,TS val 1196253834 ecr 932478476], length 14: RESP "PING" 10:06:09.535680 IP 192.168.0.183.6379 > 192.168.0.91.39782: Flags [P.], seq 3533:3540, ack 196, win 1432, options [nop,nop,TS val 932479264 ecr 1196253834], length 7: RESP "PONG" 10:06:09.535756 IP 192.168.0.91.39782 > 192.168.0.183.6379: Flags [.], ack 3540, win 1424, options [nop,nop,TS val 1196253834 ecr 932479264], length 0 10:06:09.710353 IP 192.168.0.51.46508 > 192.168.0.183.6379: Flags [P.], seq 80:120, ack 281, win 6059, options [nop,nop,TS val 774008738 ecr 932478432], length 40: RESP "REPLCONF" "ACK" "7863711" 10:06:09.710574 IP 192.168.0.183.6379 > 192.168.0.51.46508: Flags [.], ack 120, win 227, options [nop,nop,TS val 932479439 ecr 774008738], length 0 10:06:09.994816 IP 192.168.0.51.53640 > 192.168.0.183.6379: Flags [P.], seq 168:182, ack 19, win 1424, options [nop,nop,TS val 774009022 ecr 932478653], length 14: RESP "PING" 10:06:09.995046 IP 192.168.0.183.6379 > 192.168.0.51.53640: Flags [P.], seq 19:26, ack 182, win 1432, options [nop,nop,TS val 932479724 ecr 774009022], length 7: RESP "PONG"
###抓包命令###
192.168.0.91和192.168.0.51为部署的两个哨兵节点
抓包命令是在192.168.0.91上执行的，命令如下：
tcpdump -ni ens37 host 192.168.0.183 and port 6379
tcpdump -ni ens37 host 192.168.0.51 and port 26379 -A
（2）主观下线：在心跳检测的定时任务中，如果其他节点超过一定时间没有回复，哨兵节点就会将其进行主观下线。顾名思义，主观下线的意思是一个哨兵节点“主观地”判断下线；与主观下线相对应的是客观下线。
（3）客观下线：哨兵节点在对主节点进行主观下线后，会通过sentinel is-master-down-by-addr命令询问其他哨兵节点该主节点的状态；如果判断主节点下线的哨兵数量达到一定数值，则对该主节点进行客观下线。
二、哨兵模式配置
1、环境介绍
操作系统：CentOS7.6
redis版本：redis-5.0.12
主节点：192.168.0.183
从节点：192.168.0.51 和 192.168.0.91
哨兵节点：192.168.0.51 和 192.168.0.91
2、配置哨兵模式
在配置哨兵模式之前，需要先做主从复制(一主多从)，在这里不做讲解；
分别在192.168.0.91/192.168.0.51上配置哨兵模式：
logfile "/usr/local/redis/logs/redis-sentinel.log"
# sentinel monitor <master-name> <ip> <redis-port> <quorum>
#master-name自定义，quorum表示：至少需要多少个哨兵节点同意，才能判定主节点故障并进行故障转移。
sentinel monitor mymaster 192.168.0.183 6379 2 #这里表示至少需要2个哨兵节点同意，才能判定主节点故障并进行故障转移； sentinel auth-pass mymaster 123.com #这里由于我在master上配置了认证，所以需要在哨兵节点配置上master的访问密码；
3、查看哨兵模式信息
配置完哨兵模式后，需要启动哨兵服务，监听26379端口；
[root@node1 ~]# redis -p 26379 info Sentinel # Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 master0:name=mymaster,status=ok,address=192.168.0.183:6379,slaves=2,sentinels=2
#
4、演示故障转移
4.1、手动down掉master主机
systemctl stop redis.service
4.2、查看slave主机的redis.log日志
41510:S 12 Mar 2021 14:20:33.806 * MASTER <-> REPLICA sync started 41510:S 12 Mar 2021 14:20:33.806 # Error condition on socket for SYNC: Connection refused 41510:S 12 Mar 2021 14:20:34.812 * Connecting to MASTER 192.168.0.183:6379 41510:S 12 Mar 2021 14:20:34.813 * MASTER <-> REPLICA sync started 41510:S 12 Mar 2021 14:20:34.814 # Error condition on socket for SYNC: Connection refused 41510:S 12 Mar 2021 14:20:35.820 * Connecting to MASTER 192.168.0.183:6379 41510:S 12 Mar 2021 14:20:35.820 * MASTER <-> REPLICA sync started 41510:S 12 Mar 2021 14:20:35.820 # Error condition on socket for SYNC: Connection refused 41510:M 12 Mar 2021 14:20:36.244 # Setting secondary replication ID to e680f4e597e6c4db75f9483ed97995f518e96b89, valid up to offset: 25196. New replication ID is a79f9c80b854a6b823b7aa58d730c41b1c166f26 41510:M 12 Mar 2021 14:20:36.244 * Discarding previously cached master state. 41510:M 12 Mar 2021 14:20:36.245 * MASTER MODE enabled (user request from 'id=6 addr=192.168.0.91:51234 fd=11 name=sentinel-a6f12e21-cmd age=183 idle=0 flags=x db=0 sub=0 psub=0 multi=3 qbuf=140qbuf-free=32628 obl=36 oll=0 omem=0 events=r cmd=exec') 41510:M 12 Mar 2021 14:20:36.248 # CONFIG REWRITE executed with success. 41510:M 12 Mar 2021 14:20:36.845 * Replica 192.168.0.51:6379 asks for synchronization 41510:M 12 Mar 2021 14:20:36.845 * Partial resynchronization request from 192.168.0.51:6379 accepted. Sending 302 bytes of backlog starting from offset 25196.
##########
slave日志中是从14:20:06开始连接超时的，14:20:36新的master选举成功后连接成功；期间经过了30s；
4.3、查看sentinel.log的日志
39131:X 12 Mar 2021 14:20:35.955 # +sdown master mymaster 192.168.0.183 6379 39131:X 12 Mar 2021 14:20:36.057 # +odown master mymaster 192.168.0.183 6379 #quorum 2/2 #主节点下线 39131:X 12 Mar 2021 14:20:36.057 # +new-epoch 7 #开启新的纪元 39131:X 12 Mar 2021 14:20:36.057 # +try-failover master mymaster 192.168.0.183 6379 #尝试故障转移 39131:X 12 Mar 2021 14:20:36.073 # +vote-for-leader a6f12e218ee3306e792d35ac399be04b0d323ee4 7 #投票选举领导人 39131:X 12 Mar 2021 14:20:36.100 # 53d4d0aa6766c3c7b82e87db4707a046e8b697f7 voted for a6f12e218ee3306e792d35ac399be04b0d323ee4 7 39131:X 12 Mar 2021 14:20:36.131 # +elected-leader master mymaster 192.168.0.183 6379 39131:X 12 Mar 2021 14:20:36.131 # +failover-state-select-slave master mymaster 192.168.0.183 6379 39131:X 12 Mar 2021 14:20:36.188 # 39131:X 12 Mar 2021 14:20:36.188 * +failover-state-send-slaveof-noone slave 192.168.0.91:6379 192.168.0.91 6379 @ mymaster 192.168.0.183 6379 39131:X 12 Mar 2021 14:20:36.244 * +failover-state-wait-promotion slave 192.168.0.91:6379 192.168.0.91 6379 @ mymaster 192.168.0.183 6379 39131:X 12 Mar 2021 14:20:36.631 # +promoted-slave slave 192.168.0.91:6379 192.168.0.91 6379 @ mymaster 192.168.0.183 6379 39131:X 12 Mar 2021 14:20:36.631 # +failover-state-reconf-slaves master mymaster 192.168.0.183 6379 39131:X 12 Mar 2021 14:20:36.669 * +slave-reconf-sent slave 192.168.0.51:6379 192.168.0.51 6379 @ mymaster 192.168.0.183 6379 39131:X 12 Mar 2021 14:20:37.098 * +slave-reconf-inprog slave 192.168.0.51:6379 192.168.0.51 6379 @ mymaster 192.168.0.183 6379 39131:X 12 Mar 2021 14:20:37.098 * +slave-reconf-done slave 192.168.0.51:6379 192.168.0.51 6379 @ mymaster 192.168.0.183 6379 39131:X 12 Mar 2021 14:20:37.181 # +failover-end master mymaster 192.168.0.183 6379 39131:X 12 Mar 2021 14:20:37.182 # +switch-master mymaster 192.168.0.183 6379 192.168.0.91 6379 #切换master主机，完成故障转移； 39131:X 12 Mar 2021 14:20:37.182 * +slave slave 192.168.0.51:6379 192.168.0.51 6379 @ mymaster 192.168.0.91 6379 39131:X 12 Mar 2021 14:20:37.182 * +slave slave 192.168.0.183:6379 192.168.0.183 6379 @ mymaster 192.168.0.91 6379 39131:X 12 Mar 2021 14:21:07.203 # +sdown slave 192.168.0.183:6379 192.168.0.183 6379 @ mymaster 192.168.0.91 6379 39131:X 12 Mar 2021 16:43:57.903 # -sdown slave 192.168.0.183:6379 192.168.0.183 6379 @ mymaster 192.168.0.91 6379 #原master节点0.183启动后成为从节点；
4.4、查看sentinel信息
[root@localhost redis]# redis -p 26379 info Sentinel # Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 master0:name=mymaster,status=ok,address=192.168.0.183:6379,slaves=2,sentinels=2 [root@localhost redis]# redis -p 26379 info Sentinel # Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 master0:name=mymaster,status=ok,address=192.168.0.91:6379,slaves=2,sentinels=2
####################
查看Sentinel信息，已经选举了新的master，但是此时的slave还是2，这是因为将已经挂了的master踢掉之后将它变成了slave，当前master再次启动的时候，它就是slave了；
三、总结
1、在配置哨兵模式的时候，无论master还是slave，没有配置认证还则罢了；如果配置了认证，一定要在所有设备的redis.conf中配置如下两个参数：
masterauth "123.com" requirepass "123.com"
######
这两个参数的密码一定要保持一致，少配置或者密码不一致，则会因为当主节点挂了，由于认证而无法选举出新的主节点导致哨兵模式迁移失败；
2、哨兵节点支持的命令
2.1、基础查询：通过这些命令，可以查询哨兵系统的拓扑结构、节点信息、配置信息等。
info sentinel：获取监控的所有主节点的基本信息
sentinel masters：获取监控的所有主节点的详细信息
sentinel master mymaster：获取监控的主节点mymaster的详细信息
sentinel slaves mymaster：获取监控的主节点mymaster的从节点的详细信息
sentinel sentinels mymaster：获取监控的主节点mymaster的哨兵节点的详细信息
sentinel get-master-addr-by-name mymaster：获取监控的主节点mymaster的地址信息
sentinel is-master-down-by-addr：哨兵节点之间可以通过该命令询问主节点是否下线，从而对是否客观下线做出判断
2.2、增加/移除对主节点的监控
sentinel monitor mymaster2 192.168.92.128 16379 2：与部署哨兵节点时配置文件中的sentinel monitor功能完全一样，不再详述
sentinel remove mymaster2：取消当前哨兵节点对主节点mymaster2的监控
2.3、强制故障转移
sentinel failover mymaster：该命令可以强制对mymaster执行故障转移，即便当前的主节点运行完好；例如，如果当前主节点所在机器即将报废，便可以提前通过failover命令进行故障转移。
四、感谢
感谢(https://www.cnblogs.com/kismetv/p/9609938.html)这篇博客的指导；向作者编程迷思致敬；