本次实验环境准备用一台服务器模拟3台redis服务器,1主2从
主从集群搭建
第一步:安装Redis
安装Redis,参考前面安装Redis文章,保证单机使用没有问题。
第二步:配置服务器文件
定位到安装后的redis目录:cd /usr/local/redis
对单机的redis配置文件拷贝出3份出来
cp redis.conf redis6381.conf
cp redis.conf redis6382.conf
cp redis.conf redis6383.conf
清空新拷贝的三份配置文件,命令为 “> 文件名”
[root@localhost redis]# > redis6381.conf
[root@localhost redis]# > redis6382.conf
[root@localhost redis]# > redis6383.conf
再对三份配置文件分别配置内容
6381端口的配置文件
include /usr/local/redis/redis.conf #包含redis.conf文件 daemonize yes port 6381 pidfile /var/run/redis_6381.pid logfile 6381.log dbfilename dump6381.rdb
6382端口的配置文件
include /usr/local/redis/redis.conf #包含redis.conf文件 daemonize yes port 6382 pidfile /var/run/redis_6382.pid logfile 6382.log dbfilename dump6382.rdb slaveof 127.0.0.1 6381 #说明是从属于6381端口的主机
6383端口的配置文件
include /usr/local/redis/redis.conf #包含redis.conf文件 daemonize yes port 6383 pidfile /var/run/redis_6383.pid logfile 6383.log dbfilename dump6383.rdb slaveof 127.0.0.1 6381 #说明是从属于6381端口的主机
第三步:启动redis服务器
模拟启动三台redis服务器
[root@localhost redis]# ./bin/redis-server redis6381.conf
[root@localhost redis]# ./bin/redis-server redis6382.conf
[root@localhost redis]# ./bin/redis-server redis6383.conf
查看进程
[root@localhost redis]# ps -ef|grep redis
root 90648 1 0 17:17 ? 00:00:00 ./bin/redis-server 127.0.0.1:6381
root 90652 1 0 17:17 ? 00:00:00 ./bin/redis-server 127.0.0.1:6382
root 90657 1 1 17:17 ? 00:00:00 ./bin/redis-server 127.0.0.1:6383
root 90664 4536 0 17:17 pts/2 00:00:00 grep --color=auto redis
说明3台服务器已经启动成功,端口分别是指定的6381,6382,6383
第四步,客户端连接redis服务器
分别打开三个窗口,定位到redis目录:cd /usr/local/redis
通过客户端工具连接不同端口的服务器
如连接6381端口服务器
[root@localhost redis]# ./bin/redis-cli -p 6381
查看信息,如下:
127.0.0.1:6381> info replication
# Replication
role:master
connected_slaves:2
slave0:ip=127.0.0.1,port=6382,state=online,offset=253,lag=1
slave1:ip=127.0.0.1,port=6383,state=online,offset=253,lag=1
master_repl_offset:253
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:252
可以看到它是主服务器,我们在前面配置文件中指定它就是主服务器。
因为是主机,它可读可写,写的数据同步到从机。
写:
127.0.0.1:6381> set key1 v1
OK
127.0.0.1:6381> set key2 v2
OK
读:
127.0.0.1:6381> get key1
"v1"
127.0.0.1:6381> get key2
"v2"
如连接6382端口服务器
[root@localhost redis]# ./bin/redis-cli -p 6382
查看信息,如下:
127.0.0.1:6382> info replication
# Replication
role:slave
master_host:127.0.0.1
master_port:6381
master_link_status:up
master_last_io_seconds_ago:10
master_sync_in_progress:0
slave_repl_offset:883
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
可以看到,它是从服务器,我们在前面配置文件中指定它就是从服务器。
因为是从机,它只读不可写。数据只能从主机6381同步。
读:
127.0.0.1:6382> keys *
1) "key1"
2) "key2"
127.0.0.1:6382> get key2
"v2"
写:
127.0.0.1:6382> set key3 v3
(error) READONLY You can't write against a read only slave.
哨兵模式
配置哨兵配置文件
定位到redis的源文件目录,将哨兵配置文件sentinel.conf作为模板,拷贝出3份
[root@localhost local]# cd redis-3.2.9/
[root@localhost redis-3.2.9]# cp sentinel.conf sentinel26381.conf
[root@localhost redis-3.2.9]# cp sentinel.conf sentinel26382.conf
[root@localhost redis-3.2.9]# cp sentinel.conf sentinel26383.conf
分别修改拷贝的配置文件
[root@localhost redis-3.2.9]# vim sentinel26381.conf
将哨兵的工作端口端口26379和监控的redis主机127.0.0.1 6379 2修改为自己定义的值
模板中的两处要被修改的地方:
port 26379
sentinel monitor mymaster 127.0.0.1 6379 2
我这里修改为该哨兵端口为26381,监控的redis主机为127.0.0.1 6381
port 26381
sentinel monitor mymaster 127.0.0.1 6381 2
其他两台哨兵配置文件也按照这样修改,只是哨兵的端口要不一样,监控的redis主机应该一样。
启动哨兵程序
分别打开3个窗口,定为到/usr/local/redis-3.2.9/src,找到redis-sentinel,配合上步的配置文件启动
注意需要3个窗口,因为这里是前台启动,需要看监控信息
[root@localhost src]# ./redis-sentinel ../sentinel26381.conf
[root@localhost src]# ./redis-sentinel ../sentinel26382.conf
[root@localhost src]# ./redis-sentinel ../sentinel26383.conf
等3台哨兵都启动成功以后,
分别查看监控信息,26381哨兵的监控信息,如下:
[root@localhost src]# ./redis-sentinel ../sentinel26381.conf
94744:X 01 Feb 18:29:03.171 * Increased maximum number of open files to 10032 (it was originally set to 1024).
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 3.2.9 (00000000/0) 64 bit
.-`` .-```. ```/ _.,_ ''-._
( ' , .-` | `, ) Running in sentinel mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 26381
| `-._ `._ / _.-' | PID: 94744
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
94744:X 01 Feb 18:29:03.172 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
94744:X 01 Feb 18:29:03.174 # Sentinel ID is 9044885c3cd355b168a93bc68ed4c2b7c78ebca6
94744:X 01 Feb 18:29:03.174 # +monitor master mymaster 127.0.0.1 6381 quorum 2
94744:X 01 Feb 18:29:03.175 * +slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6381
94744:X 01 Feb 18:29:03.176 * +slave slave 127.0.0.1:6383 127.0.0.1 6383 @ mymaster 127.0.0.1 6381
/usr/local/redis-3.2.9/src/usr/local/redis-3.2.9/src94744:X 01 Feb 18:33:35.645 * +sentinel sentinel 7bc01d8db6cf93c5d1f9d0a146c9967c0c4b339b 127.0.0.1 26382 @ mymaster 127.0.0.1 6381
94744:X 01 Feb 18:34:08.801 * +sentinel sentinel 574de03c2a3e5ace4798af0df4945961398efdf1 127.0.0.1 26383 @ mymaster 127.0.0.1 6381
26382哨兵的监控信息,如下:
[root@localhost src]# ./redis-sentinel ../sentinel26382.conf
95009:X 01 Feb 18:33:33.626 * Increased maximum number of open files to 10032 (it was originally set to 1024).
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 3.2.9 (00000000/0) 64 bit
.-`` .-```. ```/ _.,_ ''-._
( ' , .-` | `, ) Running in sentinel mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 26382
| `-._ `._ / _.-' | PID: 95009
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
95009:X 01 Feb 18:33:33.627 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
95009:X 01 Feb 18:33:33.648 # Sentinel ID is 7bc01d8db6cf93c5d1f9d0a146c9967c0c4b339b
95009:X 01 Feb 18:33:33.648 # +monitor master mymaster 127.0.0.1 6381 quorum 2
95009:X 01 Feb 18:33:33.649 * +slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:33:33.651 * +slave slave 127.0.0.1:6383 127.0.0.1 6383 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:33:34.651 * +sentinel sentinel 9044885c3cd355b168a93bc68ed4c2b7c78ebca6 127.0.0.1 26381 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:34:08.801 * +sentinel sentinel 574de03c2a3e5ace4798af0df4945961398efdf1 127.0.0.1 26383 @ mymaster 127.0.0.1 6381
26383哨兵的监控信息,如下:
[root@localhost src]# ./redis-sentinel ../sentinel26383.conf
95097:X 01 Feb 18:34:06.716 * Increased maximum number of open files to 10032 (it was originally set to 1024).
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 3.2.9 (00000000/0) 64 bit
.-`` .-```. ```/ _.,_ ''-._
( ' , .-` | `, ) Running in sentinel mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 26383
| `-._ `._ / _.-' | PID: 95097
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
95097:X 01 Feb 18:34:06.717 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
95097:X 01 Feb 18:34:06.719 # Sentinel ID is 574de03c2a3e5ace4798af0df4945961398efdf1
95097:X 01 Feb 18:34:06.719 # +monitor master mymaster 127.0.0.1 6381 quorum 2
95097:X 01 Feb 18:34:06.719 * +slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6381
95097:X 01 Feb 18:34:06.721 * +slave slave 127.0.0.1:6383 127.0.0.1 6383 @ mymaster 127.0.0.1 6381
95097:X 01 Feb 18:34:07.784 * +sentinel sentinel 9044885c3cd355b168a93bc68ed4c2b7c78ebca6 127.0.0.1 26381 @ mymaster 127.0.0.1 6381
95097:X 01 Feb 18:34:08.425 * +sentinel sentinel 7bc01d8db6cf93c5d1f9d0a146c9967c0c4b339b 127.0.0.1 26382 @ mymaster 127.0.0.1 6381
模拟故障,让主从自动切换
主动让redis的主服务器6381关掉,模拟主机挂机的状况
127.0.0.1:6381> shutdown
观看哨兵服务器,会发现以下信息
[root@localhost src]# ./redis-sentinel ../sentinel26382.conf
95009:X 01 Feb 18:33:33.626 * Increased maximum number of open files to 10032 (it was originally set to 1024).
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 3.2.9 (00000000/0) 64 bit
.-`` .-```. ```/ _.,_ ''-._
( ' , .-` | `, ) Running in sentinel mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 26382
| `-._ `._ / _.-' | PID: 95009
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
95009:X 01 Feb 18:33:33.627 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
95009:X 01 Feb 18:33:33.648 # Sentinel ID is 7bc01d8db6cf93c5d1f9d0a146c9967c0c4b339b
95009:X 01 Feb 18:33:33.648 # +monitor master mymaster 127.0.0.1 6381 quorum 2
95009:X 01 Feb 18:33:33.649 * +slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:33:33.651 * +slave slave 127.0.0.1:6383 127.0.0.1 6383 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:33:34.651 * +sentinel sentinel 9044885c3cd355b168a93bc68ed4c2b7c78ebca6 127.0.0.1 26381 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:34:08.801 * +sentinel sentinel 574de03c2a3e5ace4798af0df4945961398efdf1 127.0.0.1 26383 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:42.232 # +sdown master mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:42.316 # +odown master mymaster 127.0.0.1 6381 #quorum 2/2
95009:X 01 Feb 18:59:42.316 # +new-epoch 1
95009:X 01 Feb 18:59:42.316 # +try-failover master mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:42.325 # +vote-for-leader 7bc01d8db6cf93c5d1f9d0a146c9967c0c4b339b 1
95009:X 01 Feb 18:59:42.363 # 9044885c3cd355b168a93bc68ed4c2b7c78ebca6 voted for 7bc01d8db6cf93c5d1f9d0a146c9967c0c4b339b 1
95009:X 01 Feb 18:59:42.363 # 574de03c2a3e5ace4798af0df4945961398efdf1 voted for 7bc01d8db6cf93c5d1f9d0a146c9967c0c4b339b 1
95009:X 01 Feb 18:59:42.379 # +elected-leader master mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:42.379 # +failover-state-select-slave master mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:42.470 # +selected-slave slave 127.0.0.1:6383 127.0.0.1 6383 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:42.470 * +failover-state-send-slaveof-noone slave 127.0.0.1:6383 127.0.0.1 6383 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:42.522 * +failover-state-wait-promotion slave 127.0.0.1:6383 127.0.0.1 6383 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:43.315 # +promoted-slave slave 127.0.0.1:6383 127.0.0.1 6383 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:43.315 # +failover-state-reconf-slaves master mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:43.376 * +slave-reconf-sent slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:44.363 * +slave-reconf-inprog slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:44.363 * +slave-reconf-done slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:44.422 # +failover-end master mymaster 127.0.0.1 6381
95009:X 01 Feb 18:59:44.422 # +switch-master mymaster 127.0.0.1 6381 127.0.0.1 6383
95009:X 01 Feb 18:59:44.422 * +slave slave 127.0.0.1:6382 127.0.0.1 6382 @ mymaster 127.0.0.1 6383
95009:X 01 Feb 18:59:44.422 * +slave slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6383
95009:X 01 Feb 19:00:14.465 # +sdown slave 127.0.0.1:6381 127.0.0.1 6381 @ mymaster 127.0.0.1 6383
说明集群的主服务器已经切换到了6383上,即原来的从机6383变为了主机
我们去看一下6383的信息
127.0.0.1:6383> info replication
# Replication
role:master
connected_slaves:1
slave0:ip=127.0.0.1,port=6382,state=online,offset=19848,lag=1
master_repl_offset:19981
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:2
repl_backlog_histlen:19980
确实角色为master,它下面挂载一台从机6382
它现在升级为主机,所以可以读写,验证如下:
127.0.0.1:6383> set key5 v5
OK
127.0.0.1:6383> get key5
"v5"
再去看下一6382从机的信息
127.0.0.1:6382> info replication
# Replication
role:slave
master_host:127.0.0.1
master_port:6383
master_link_status:up
master_last_io_seconds_ago:0
master_sync_in_progress:0
slave_repl_offset:17440
slave_priority:100
slave_read_only:1
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
说明它确实从属的主机为6383
它还是只能读,数据只能从主机同步
127.0.0.1:6382> set key6 v6
(error) READONLY You can't write against a read only slave.
127.0.0.1:6382> get key5
"v5"
假设原来的6381服务器已经恢复正常了,我们来启动它
可以看到,6381启动后,自动变为了从机,主机还是后来的6383,说明原来的主机恢复后,不会恢复它原来的地位。
因为它已经是从机,所以只能读,不能写
再看主服务器6383的信息,它已经有两个从机了,6381是后来恢复后自动挂载上的
哨兵服务器26381上看到对应的自动挂载信息
哨兵服务器26382上看到对应的自动挂载信息
哨兵服务器26383上看到对应的自动挂载信息