redis集群高可用集群
功能：1：数据分部署存储
2：解决单点故障
3：实现数据的自动备份
环境准备：
| ip | 服务 |
| 192.168.4.51 | redis |
| 192.168.4.52 | redis |
| 192.168.4.53 | redis |
| 192.168.4.54 | redis |
| 192.168.4.55 | redis |
| 192.168.4.56 | redis |
每台机器启用集群功能
cluster-enabled yes //启用汲取功能
cluster-config-file nodes-6379.conf //存储集群信息文件
cluster-node-timeout 15000 //通信超时时间
查看：
~]# netstat -utnlp | grep redis-server tcp 0 0 192.168.4.52:6379 0.0.0.0:* LISTEN 13094/redis-server tcp 0 0 192.168.4.52:16379 0.0.0.0:* LISTEN 13094/redis-server
]# redis-cli -h 192.168.4.52
192.168.4.52:6379> cluster info 查看集群信息
cluster_state:fail
cluster_slots_assigned:0
cluster_slots_ok:0
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:1
cluster_size:0
cluster_current_epoch:0
cluster_my_epoch:0
cluster_stats_messages_sent:0
准备管理主机(管理redis集群)ip 192.168.4.57
配置如下:
1 准备ruby脚本的运行环境
]# yum -y install ruby
]# which gem || yum -y install rubygems
]# gem install redis-3.2.1.gem
2 创建ruby脚本
[root@host57 ~]# tar -zxvf redis-4.0.8.tar.gz [root@host57 ~]# cd redis-4.0.8/ ]# tar -zxvf redis-4.0.8.tar.gz ]# mkdir /root/bin ]# cp src/redis-trib.rb /root/bin/ ]# redis-trib.rb help 可以查看到命令的帮助信息
3 创建集群
***务必在51-56 做如下检查 1 必须保证内存里没有数据 2 必须没有连接密码 3 必须关闭firewalld 4 必须启用了集群功能 [root@host57 ~]# redis-trib.rb create --replicas 1 192.168.4.51:6379 192.168.4.52:6379 192.168.4.53:6379 192.168.4.54:6379 192.168.4.55:6379 192.168.4.56:6379 Adding replica 192.168.4.55:6379 to 192.168.4.51:6379 Adding replica 192.168.4.56:6379 to 192.168.4.52:6379 Adding replica 192.168.4.54:6379 to 192.168.4.53:6379 M: 25b9c9fd991b859c23935bedba3e7b22565aefe0 192.168.4.51:6379 slots:0-5460 (5461 slots) master M: eed58010ef711bdb313a8ceadcbc6bff5d8b8278 192.168.4.52:6379 slots:5461-10922 (5462 slots) master M: 196a0e98868d7b6fb08c548730f28f6a729eef62 192.168.4.53:6379 slots:10923-16383 (5461 slots) master S: 8feff919774986515289bb656831ee9e5d4d8d93 192.168.4.54:6379 replicates 196a0e98868d7b6fb08c548730f28f6a729eef62 S: d6e866731303ffe8cf7eb48ff924e3493bceb8f2 192.168.4.55:6379 replicates 25b9c9fd991b859c23935bedba3e7b22565aefe0 S: a48a379acebf34b736eb5f2b1f67c55180b96d95 192.168.4.56:6379 replicates eed58010ef711bdb313a8ceadcbc6bff5d8b8278 Can I set the above configuration? (type 'yes' to accept): yes 同意 >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join... >>> Performing Cluster Check (using node 192.168.4.51:6379) M: 25b9c9fd991b859c23935bedba3e7b22565aefe0 192.168.4.51:6379 slots:0-5460 (5461 slots) master 1 additional replica(s) S: a48a379acebf34b736eb5f2b1f67c55180b96d95 192.168.4.56:6379 slots: (0 slots) slave replicates eed58010ef711bdb313a8ceadcbc6bff5d8b8278 S: d6e866731303ffe8cf7eb48ff924e3493bceb8f2 192.168.4.55:6379 slots: (0 slots) slave replicates 25b9c9fd991b859c23935bedba3e7b22565aefe0 S: 8feff919774986515289bb656831ee9e5d4d8d93 192.168.4.54:6379 slots: (0 slots) slave replicates 196a0e98868d7b6fb08c548730f28f6a729eef62 M: eed58010ef711bdb313a8ceadcbc6bff5d8b8278 192.168.4.52:6379 slots:5461-10922 (5462 slots) master 1 additional replica(s) M: 196a0e98868d7b6fb08c548730f28f6a729eef62 192.168.4.53:6379 slots:10923-16383 (5461 slots) master 1 additional replica(s) [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. 成功的提示 [root@host57 ~]#
创建集群失败的统一解决办法，在6台主机分别执行如下操作
1 停止redis服务
2 rm -rf /var/lib/redis/6379/*
3 启动redis服务
4 在管理主机再次执行创建集群的命令
4 查看集群信息
4.1 在管理主机查看
#查看统计信息
[root@host57 ~]# redis-trib.rb info 192.168.4.54:6379 192.168.4.52:6379 (eed58010...) -> 0 keys | 5462 slots | 1 slaves. 192.168.4.53:6379 (196a0e98...) -> 0 keys | 5461 slots | 1 slaves. 192.168.4.51:6379 (25b9c9fd...) -> 0 keys | 5461 slots | 1 slaves. [OK] 0 keys in 3 masters. 0.00 keys per slot on average.
理论：
存储数据的工作过程
16384 slots 0 ~ 16383 平均的分配给3台master服务器存储数据的机会
测试服务的高可用
master 角色的服务器宕机后 对应的slave 会自动升级为master并接收hash slot
宕机的master恢复后，自动加入集群 自动做当前master的slvae服务器同步数据
二 :管理集群 准备主机 运行Redis服务并启用集群功能
准备两台新的主机：
192.168.4.58
192.168.4.59
2.1：向集群添加新主机
1：添加master角色主机（场景：提高集群处理性能，扩大内存空间）
把58主机添加到集群内做master服务器
58主机必须运行redis服务并启用集群功能
1 添加主机到集群 [root@host57 ~]# redis-trib.rb add-node 192.168.4.58:6379 192.168.4.52:6379
添加节点机 新主机ip端口 访问的原始主机ip：端口
>>> Send CLUSTER MEET to node 192.168.4.58:6379 to make it join the cluster. [OK] New node added correctly.
2 重新分片 (给新的master服务器分配hash slot)
[root@host57 ~]# redis-trib.rb reshard 192.168.4.52:6379
回答问题
1
How many slots do you want to move (from 1 to 16384)?4096
2
What is the receiving node ID? 1cb19e8b2746996da3c3eca7eb7f4246699b2f24
3
all
4
Do you want to proceed with the proposed reshard plan (yes/no)? yes
3 查看集群信息
[root@host57 ~]# redis-trib.rb info 192.168.4.52:6379 192.168.4.52:6379 (eed58010...) -> 2 keys | 4096 slots | 1 slaves. 192.168.4.53:6379 (196a0e98...) -> 1 keys | 4096 slots | 1 slaves. 192.168.4.58:6379 (1cb19e8b...) -> 3 keys | 4096 slots | 0 slaves. 192.168.4.55:6379 (d6e86673...) -> 2 keys | 4096 slots | 1 slaves. [OK] 8 keys in 4 masters. 0.00 keys per slot on average.
检查得到的槽位
~]# redis-trib.rb check 192.168.4.52:6379
M: 1cb19e8b2746996da3c3eca7eb7f4246699b2f24 192.168.4.58:6379
slots:0-1364,5461-6826,10923-12287 (4096 slots) master
0 additional replica(s)
2 添加slave角色主机
把59主机添加到集群里做 58 的 slave
59主机必须运行Redis服务并启用集群功能
在管理主机57 做如下操作：
1 添加slave角色主机（因为slave角色主机的数据是在主服务器
同步的，所以不需要分配hash slost）
[root@host57 ~]# redis-trib.rb add-node --slave 192.168.4.59:6379 192.168.4.55:6379
>>> Configure node as replica of 192.168.4.58:6379.
[OK] New node added correctly.
查看信息：
~]# redis-trib.rb info 192.168.4.58:6379
192.168.4.58:6379 (1cb19e8b...) -> 3 keys | 4096 slots | 1 slaves.
192.168.4.55:6379 (d6e86673...) -> 2 keys | 4096 slots | 1 slaves.
192.168.4.52:6379 (eed58010...) -> 3 keys | 4096 slots | 1 slaves.
192.168.4.53:6379 (196a0e98...) -> 2 keys | 4096 slots | 1 slaves.
[OK] 10 keys in 4 masters.
0.00 keys per slot on average.
2.2 从集群中移除主机
1 移除master角色主机
2 移除slave角色主机(不占用hash slot 直接执行命令删除就可以)
在管理主机执行删除命令
#查看被移除主机的id
~]# redis-trib.rb check 192.168.4.58:6379 | grep 192.168.4.59
S: 524f73686be65ffb3422a36c5f3566d150ed9ba9 192.168.4.59:6379
#删除主机
~]# redis-trib.rb del-node 192.168.4.55:6379 524f73686be65ffb3422a36c5f3566d150ed9ba9
>>> Removing node 524f73686be65ffb3422a36c5f3566d150ed9ba9 from cluster 192.168.4.55:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
]# redis-trib.rb info 192.168.4.58:6379 //查看信息
192.168.4.58:6379 (1cb19e8b...) -> 3 keys | 4096 slots | 0 slaves.
192.168.4.55:6379 (d6e86673...) -> 2 keys | 4096 slots | 1 slaves.
192.168.4.52:6379 (eed58010...) -> 3 keys | 4096 slots | 1 slaves.
192.168.4.53:6379 (196a0e98...) -> 2 keys | 4096 slots | 1 slaves.
[OK] 10 keys in 4 masters.
0.00 keys per slot on average.
清楚之前的信息
[root@host59 ~]# redis-cli -h 192.168.4.59 192.168.4.59:6379> cluster reset 清除前世的记忆 192.168.4.59:6379> exit
添加做指定master 服务器的 slave
~]# redis-trib.rb add-node --slave --master-id d6e866731303ffe8cf7eb48ff924e3493bceb8f2 192.168.4.59:6379 192.168.4.55:6379
[OK] New node added correctly.
把master角色的主机移除集群
在管理主机做如下操作：
1 释放hash slot host57~]# redis-trib.rb reshard 192.168.4.51:6379 问题1 释放槽的个数 How many slots do you want to move (from 1 to 16384)? 4096 问题2 接收槽主机的id What is the receiving node ID? 196a0e98868d7b6fb08c548730f28f6a729eef62 问题3 释放槽主机的id Source node #1:1cb19e8b2746996da3c3eca7eb7f4246699b2f24 Source node #2:done 槽的个数已经够了 结束ID指定 Do you want to proceed with the proposed reshard plan (yes/no)? yes 同意
2 移除集群 #查看主机ID [root@host57 ~]# redis-trib.rb check 192.168.4.51:6379 | grep 192.168.4.58 M: 1cb19e8b2746996da3c3eca7eb7f4246699b2f24 192.168.4.58:6379 [root@host57 ~]# #删除主机 [root@host57 ~]# redis-trib.rb del-node 192.168.4.51:6379 1cb19e8b2746996da3c3eca7eb7f4246699b2f24 >>> Removing node 1cb19e8b2746996da3c3eca7eb7f4246699b2f24 from cluster 192.168.4.51:6379 >>> Sending CLUSTER FORGET messages to the cluster... >>> SHUTDOWN the node. [root@host57 ~]# 3 查看集群状态 [root@host57 ~]# redis-trib.rb info 192.168.4.51:6379 192.168.4.53:6379 (196a0e98...) -> 5 keys | 8192 slots | 1 slaves. 192.168.4.52:6379 (eed58010...) -> 3 keys | 4096 slots | 1 slaves. 192.168.4.55:6379 (d6e86673...) -> 2 keys | 4096 slots | 2 slaves. [OK] 10 keys in 3 masters. 0.00 keys per slot on average.
4 清除redis服务器的集群信息 [root@host58 ~]# /etc/init.d/redis_6379 start [root@host58 ~]# redis-cli -h 192.168.4.58 192.168.4.58:6379> cluster reset 调试所有的主平均占用hash slot [root@host57 ~]# redis-trib.rb rebalance 192.168.4.51:6379 把集群中的主机恢复为独立的Redis服务器 [root@host51 ~]# redis-cli -h 192.168.4.51 shutdown [root@host51 ~]# rm -rf /var/lib/redis/6379/* [root@host51 ~]# sed -i '815s/^/#/' /etc/redis/6379.conf [root@host51 ~]# sed -i '823s/^/#/' /etc/redis/6379.conf [root@host51 ~]# sed -i '829s/^/#/' /etc/redis/6379.conf [root@host51 ~]# sed -n '815p;823p;829p' /etc/redis/6379.conf #cluster-enabled yes #cluster-config-file nodes-6379.conf #cluster-node-timeout 15000 [root@host51 ~]# [root@host51 ~]# /etc/init.d/redis_6379 start Starting Redis server... [root@host51 ~]# netstat -utnlp | grep redis-server tcp 0 0 192.168.4.51:6379 0.0.0.0:* LISTEN 26170/redis-server
总结：
在做redis集群时，最少要准备六台服务器
装环境的时候，注意都要开启集群功能：
如果是以前做过的服务器。记得清楚主配置文件中的信息，否则会出现报错。
在删除或者添加新主机的时候，根据计算好各个服务器的各种硬件性能，合理的分配槽位。使集群发挥高标准的性能。
常用命令：
查看集群信息 ]# redis-trib.rb info 192.168.4.52:6379 重新分片 (给新的master服务器分配hash slot) ]# redis-trib.rb reshard 192.168.4.52:6379 添加主机到集群 ]# redis-trib.rb add-node 192.168.4.58:6379 192.168.4.52:6379 检查 ]# redis-trib.rb check 192.168.4.52:6379 删除主机 ~]# redis-trib.rb del-node 192.168.4.55:6379 524f73686be65ffb3422a36c5f3566d150ed9ba9 清楚之前的信息 > cluster reset