版本：4.0.8
环境准备 准备 6台(51-56) redis服务器 以默认配置运行redis服务即可
一.创建Redis集群
1.启用集群功能( 51-56 都要配置)
]# netstat -antupl |grep :6379（要有东西）
]# /etc/init.d/redis_6379 stop
*一一对应，以52为例*
]# vim /etc/redis/6379.conf
70 bind 192.168.4.52
93 port 6352
815 cluster-enabled yes
823 cluster-config-file nodes-6352.conf
829 cluster-node-timeout 5000
:wq
]# /etc/init.d/redis_6379 start
]# netstat -utnlp | grep redis-server
tcp 0 0 192.168.4.52:6352 0.0.0.0:* LISTEN 7469/redis-server 1
tcp 0 0 192.168.4.52:16352 0.0.0.0:* LISTEN 7469/redis-server 1
]# cat /var/lib/redis/6379/nodes-6352.conf 查看集群信息文件
47b98f4c7d16486e5f8032e7a420675a025dd090 :0@0 myself,master - 0 0 0 connected
vars currentEpoch 0 lastVoteEpoch 0
2 创建集群
2.1 部署ruby脚本执行环境 (192.168.4.51)
]# yum -y install ruby rubygems uby-devel
]# gem install redis-3.2.1.gem
]# gem install redis-3.2.1.gem
Successfully installed redis-3.2.1
Parsing documentation for redis-3.2.1
Installing ri documentation for redis-3.2.1
1 gem installed
2.2 创建管理集群的ruby脚本
]# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
]# mkdir /root/bin
]# cp redis-4.0.8/src/redis-trib.rb /root/bin/
]# chmod +x /root/bin/redis-trib.rb
]# redis-trib.rb help
Usage: redis-trib <command> <options> <arguments ...>
create host1:port1 ... hostN:portN
--replicas <arg>
check host:port
info host:port
fix host:port
--timeout <arg>
reshard host:port
--from <arg>
--to <arg>
--slots <arg>
--yes
--timeout <arg>
--pipeline <arg>
rebalance host:port
--weight <arg>
--auto-weights
--use-empty-masters
--timeout <arg>
--simulate
--pipeline <arg>
--threshold <arg>
add-node new_host:new_port existing_host:existing_port
--slave
--master-id <arg>
del-node host:port node_id
set-timeout host:port milliseconds
call host:port command arg arg .. arg
import host:port
--from <arg>
--copy
--replace
help (show this help)
2.3 创建集群
]# redis-trib.rb create --replicas 1
> 192.168.4.51:6351 192.168.4.52:6352
> 192.168.4.53:6353 192.168.4.54:6354
> 192.168.4.55:6355 192.168.4.56:6356
...
Can I set the above configuration? (type 'yes' to accept): yes(输入yes)
...
[OK] All 16384 slots covered.
3 查看集群中主机状态信息
]# cat /var/lib/redis/6379/nodes-6351.conf
]# redis-cli -h 192.168.4.51 -p 6351
> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:189
cluster_stats_messages_pong_sent:175
cluster_stats_messages_sent:364
cluster_stats_messages_ping_received:170
cluster_stats_messages_pong_received:189
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:364
> cluster nodes
47b98f4c7d16486e5f8032e7a420675a025dd090 192.168.4.52:6352@16352 master - 0 1580813019504 2 connected 5461-10922
3680239417226f1e0cc1b744e822de5c85b84890 192.168.4.53:6353@16353 master - 0 1580813018602 3 connected 10923-16383
e528e231af6cb62ade68202e9fee2b5b090c9025 192.168.4.55:6355@16355 slave d9e8b7d81f4dff3d3157747e92b521220774da8b 0 1580813019104 5 connected
2384a45084fec6db31e3331912c92379c45bd4bd 192.168.4.56:6356@16356 slave 47b98f4c7d16486e5f8032e7a420675a025dd090 0 1580813019605 6 connected
43eb87ced708e2845e5b58618c817e2e1535e882 192.168.4.54:6354@16354 slave 3680239417226f1e0cc1b744e822de5c85b84890 0 1580813019104 4 connected
d9e8b7d81f4dff3d3157747e92b521220774da8b 192.168.4.51:6351@16351 myself,master - 0 1580813018000 1 connected 0-5460
集群创建失败的解决办法
]# redis-cli -h x.x.x.x -p xxxx shutdown
]# rm -rf /var/lib/redis/6379/*
]# /etc/init.d/redis_6379 start
]# netstat -utnlp | grep redis_server
再次执行创建集群命令
4 测试集群 在192.168.4.50 访问集群存取数据(可以连接任意一台主库的ip地址)
]# redis-cli -c -h 192.168.4.51 -p 6351
192.168.4.51:6351> keys *
(empty list or set)
192.168.4.51:6351> set x 101
-> Redirected to slot [16287] located at 192.168.4.53:6353
OK
192.168.4.53:6353> keys * (现在自动切换到6353库)
1) "x"
主库对应的从库，会自动把数据同步到本机。
在管理主机51上 执行如下操作
]# redis-trib.rb info 192.168.4.51:6351 查看数据存储情况
192.168.4.51:6351 (d9e8b7d8...) -> 0 keys | 5461 slots | 1 slaves.
192.168.4.52:6352 (47b98f4c...) -> 0 keys | 5462 slots | 1 slaves.
192.168.4.53:6353 (36802394...) -> 1 keys | 5461 slots | 1 slaves.
[OK] 1 keys in 3 masters.
0.00 keys per slot on average.
]# redis-trib.rb check 192.168.4.51:6351 查看集群信息
>>> Performing Cluster Check (using node 192.168.4.51:6351)
M: d9e8b7d81f4dff3d3157747e92b521220774da8b 192.168.4.51:6351
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 47b98f4c7d16486e5f8032e7a420675a025dd090 192.168.4.52:6352
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: 3680239417226f1e0cc1b744e822de5c85b84890 192.168.4.53:6353
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: e528e231af6cb62ade68202e9fee2b5b090c9025 192.168.4.55:6355
slots: (0 slots) slave
replicates d9e8b7d81f4dff3d3157747e92b521220774da8b
S: 2384a45084fec6db31e3331912c92379c45bd4bd 192.168.4.56:6356
slots: (0 slots) slave
replicates 47b98f4c7d16486e5f8032e7a420675a025dd090
S: 43eb87ced708e2845e5b58618c817e2e1535e882 192.168.4.54:6354
slots: (0 slots) slave
replicates 3680239417226f1e0cc1b744e822de5c85b84890
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
主库 从库
51 -- 55
52 -- 56
53 -- 54
主库 宕机后，对应的从库会自动升级为主库(52宕机，56主库）
]# redis-cli -h 192.168.4.52 -p 6352 shutdown
]# redis-trib.rb check 192.168.4.51:6351
>>> Performing Cluster Check (using node 192.168.4.51:6351)
M: d9e8b7d81f4dff3d3157747e92b521220774da8b 192.168.4.51:6351
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 3680239417226f1e0cc1b744e822de5c85b84890 192.168.4.53:6353
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: e528e231af6cb62ade68202e9fee2b5b090c9025 192.168.4.55:6355
slots: (0 slots) slave
replicates d9e8b7d81f4dff3d3157747e92b521220774da8b
M: 2384a45084fec6db31e3331912c92379c45bd4bd 192.168.4.56:6356
slots:5461-10922 (5462 slots) master
0 additional replica(s)
S: 43eb87ced708e2845e5b58618c817e2e1535e882 192.168.4.54:6354
slots: (0 slots) slave
replicates 3680239417226f1e0cc1b744e822de5c85b84890
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
开启52,变为56的从库
mysql52]# /etc/init.d/redis_6379 start
]# redis-trib.rb check 192.168.4.51:6351
>>> Performing Cluster Check (using node 192.168.4.51:6351)
M: d9e8b7d81f4dff3d3157747e92b521220774da8b 192.168.4.51:6351
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 47b98f4c7d16486e5f8032e7a420675a025dd090 192.168.4.52:6352
slots: (0 slots) slave
replicates 2384a45084fec6db31e3331912c92379c45bd4bd
M: 3680239417226f1e0cc1b744e822de5c85b84890 192.168.4.53:6353
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: e528e231af6cb62ade68202e9fee2b5b090c9025 192.168.4.55:6355
slots: (0 slots) slave
replicates d9e8b7d81f4dff3d3157747e92b521220774da8b
M: 2384a45084fec6db31e3331912c92379c45bd4bd 192.168.4.56:6356
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: 43eb87ced708e2845e5b58618c817e2e1535e882 192.168.4.54:6354
slots: (0 slots) slave
replicates 3680239417226f1e0cc1b744e822de5c85b84890
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
二.管理集群
准备2台新的虚拟机ip 地址分别为 192.168.4.57 192.168.4.58
运行redis服务并启用的集群配置
]# /etc/init.d/redis_6379 stop
]# vim /etc/redis/6379.conf(52为例子）
70 bind 192.168.4.52
93 port 6352
815 cluster-enabled yes
823 cluster-config-file nodes-6352.conf
829 cluster-node-timeout 5000
:wq
]# /etc/init.d/redis_6379 start
]# netstat -utnlp | grep redis-server
2.1 向集群里添加新主机
2.1.1 添加master角色主机 (添加 192.168.4.57，在51操作)
]# redis-trib.rb add-node 192.168.4.57:6357 192.168.4.51:6351
]# redis-trib.rb check 192.168.4.51:6351 查看
>>> Performing Cluster Check (using node 192.168.4.51:6351)
M: d9e8b7d81f4dff3d3157747e92b521220774da8b 192.168.4.51:6351
slots:0-5460 (5461 slots) master
1 additional replica(s)
S: 47b98f4c7d16486e5f8032e7a420675a025dd090 192.168.4.52:6352
slots: (0 slots) slave
replicates 2384a45084fec6db31e3331912c92379c45bd4bd
M: 3680239417226f1e0cc1b744e822de5c85b84890 192.168.4.53:6353
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: e528e231af6cb62ade68202e9fee2b5b090c9025 192.168.4.55:6355
slots: (0 slots) slave
replicates d9e8b7d81f4dff3d3157747e92b521220774da8b
M: 2384a45084fec6db31e3331912c92379c45bd4bd 192.168.4.56:6356
slots:5461-10922 (5462 slots) master
1 additional replica(s)
M: 44077cd747b4b32b9026a8f8ae2c0a1a4cca7441 192.168.4.57:6357
slots: (0 slots) master
0 additional replica(s)
S: 43eb87ced708e2845e5b58618c817e2e1535e882 192.168.4.54:6354
slots: (0 slots) slave
replicates 3680239417226f1e0cc1b744e822de5c85b84890
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
]# redis-trib.rb reshard 192.168.4.51:6351 重新分片
指定移出hash槽个数 4096
指定接收hash槽主机ID 57主机的ID值 440...
指定移出hash槽主机ID all
同意这样分片 yes
查看分片
]# redis-trib.rb check 192.168.4.51:6351
>>> Performing Cluster Check (using node 192.168.4.51:6351)
M: d9e8b7d81f4dff3d3157747e92b521220774da8b 192.168.4.51:6351
slots:1365-5460 (4096 slots) master
1 additional replica(s)
S: 47b98f4c7d16486e5f8032e7a420675a025dd090 192.168.4.52:6352
slots: (0 slots) slave
replicates 2384a45084fec6db31e3331912c92379c45bd4bd
M: 3680239417226f1e0cc1b744e822de5c85b84890 192.168.4.53:6353
slots:12288-16383 (4096 slots) master
1 additional replica(s)
S: e528e231af6cb62ade68202e9fee2b5b090c9025 192.168.4.55:6355
slots: (0 slots) slave
replicates d9e8b7d81f4dff3d3157747e92b521220774da8b
M: 2384a45084fec6db31e3331912c92379c45bd4bd 192.168.4.56:6356
slots:6827-10922 (4096 slots) master
1 additional replica(s)
M: 44077cd747b4b32b9026a8f8ae2c0a1a4cca7441 192.168.4.57:6357
slots:0-1364,5461-6826,10923-12287 (4096 slots) master
0 additional replica(s)
S: 43eb87ced708e2845e5b58618c817e2e1535e882 192.168.4.54:6354
slots: (0 slots) slave
replicates 3680239417226f1e0cc1b744e822de5c85b84890
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
客户端连接主库57 可以存取数据
]# redis-cli -c -h 192.168.4.57 -p 6357
192.168.4.57:6357> ping
PONG
192.168.4.57:6357> set key 111
-> Redirected to slot [12539] located at 192.168.4.53:6353
OK
192.168.4.53:6353> get key
"111"
192.168.4.53:6353> keys *
1) "key"
2) "x"
2.1.2 添加slave角色主机 (添加slave 192.168.4.58 51上操作）
]# redis-trib.rb add-node --slave 192.168.4.58:6358 192.168.4.51:6351
]# redis-trib.rb check 192.168.4.51:6351
>>> Performing Cluster Check (using node 192.168.4.51:6351)
M: d9e8b7d81f4dff3d3157747e92b521220774da8b 192.168.4.51:6351
slots:1365-5460 (4096 slots) master
1 additional replica(s)
S: 47b98f4c7d16486e5f8032e7a420675a025dd090 192.168.4.52:6352
slots: (0 slots) slave
replicates 2384a45084fec6db31e3331912c92379c45bd4bd
M: 3680239417226f1e0cc1b744e822de5c85b84890 192.168.4.53:6353
slots:12288-16383 (4096 slots) master
1 additional replica(s)
S: e528e231af6cb62ade68202e9fee2b5b090c9025 192.168.4.55:6355
slots: (0 slots) slave
replicates d9e8b7d81f4dff3d3157747e92b521220774da8b
M: 2384a45084fec6db31e3331912c92379c45bd4bd 192.168.4.56:6356
slots:6827-10922 (4096 slots) master
1 additional replica(s)
S: a02f23b9cd8410c03d047433d48ad4b73e6dea19 192.168.4.58:6358
slots: (0 slots) slave
replicates 44077cd747b4b32b9026a8f8ae2c0a1a4cca7441
M: 44077cd747b4b32b9026a8f8ae2c0a1a4cca7441 192.168.4.57:6357
slots:0-1364,5461-6826,10923-12287 (4096 slots) master
1 additional replica(s)
S: 43eb87ced708e2845e5b58618c817e2e1535e882 192.168.4.54:6354
slots: (0 slots) slave
replicates 3680239417226f1e0cc1b744e822de5c85b84890
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
自动分配给57
客户端连接主库58 可以查看到数据
]# redis-cli -c -h 192.168.4.58 -p 6358
192.168.4.58:6358> set t2 333
-> Redirected to slot [4748] located at 192.168.4.51:6351
OK
192.168.4.51:6351> keys *
1) "t2"
192.168.4.51:6351> set t3 444
-> Redirected to slot [685] located at 192.168.4.57:6357
OK
192.168.4.57:6357> keys *
1) "t3"
192.168.4.57:6357> set t4 444
-> Redirected to slot [12874] located at 192.168.4.53:6353
OK
192.168.4.53:6353> keys *
1) "t4"
2) "key"
3) "x"
2.2 从集群里移除主机
2.2.1 移除slave角色主机 (移除192.168.4.58，51操作)
redis-trib.rb del-node 任意主机 被移除主机ID(slave主机没有haxi槽，可直接移除）
]# redis-trib.rb del-node 192.168.4.51:6351 a02f23b9cd8410c03d047433d48ad4b73e6dea19
>>> Removing node a02f23b9cd8410c03d047433d48ad4b73e6dea19 from cluster 192.168.4.51:6351
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
]# redis-trib.rb check 192.168.4.51:6351
此时没有58的slave信息
2.2.2 移除master角色主机 (移除 192.168.4.57 51操作)
释放占用的hash槽
]# redis-trib.rb reshard 192.168.4.51:6351
指定移出hash槽个数 4096
指定接收hash槽主机ID 写任意一个主库ID值
Source node #1: 写57主机的ID值
Source node #2: done
同意这样分片 yes
移除master主机
]# redis-trib.rb del-node 192.168.4.51:6351 44077cd747b4b32b9026a8f8ae2c0a1a4cca7441
检测集群
]# redis-trib.rb check 192.168.4.51:6351
>>> Performing Cluster Check (using node 192.168.4.51:6351)
M: d9e8b7d81f4dff3d3157747e92b521220774da8b 192.168.4.51:6351
slots:1365-5460 (4096 slots) master
1 additional replica(s)
S: 47b98f4c7d16486e5f8032e7a420675a025dd090 192.168.4.52:6352
slots: (0 slots) slave
replicates 2384a45084fec6db31e3331912c92379c45bd4bd
M: 3680239417226f1e0cc1b744e822de5c85b84890 192.168.4.53:6353
slots:12288-16383 (4096 slots) master
1 additional replica(s)
S: e528e231af6cb62ade68202e9fee2b5b090c9025 192.168.4.55:6355
slots: (0 slots) slave
replicates d9e8b7d81f4dff3d3157747e92b521220774da8b
M: 2384a45084fec6db31e3331912c92379c45bd4bd 192.168.4.56:6356
slots:0-1364,5461-12287 (8192 slots) master
1 additional replica(s)
S: 43eb87ced708e2845e5b58618c817e2e1535e882 192.168.4.54:6354
slots: (0 slots) slave
replicates 3680239417226f1e0cc1b744e822de5c85b84890
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
重新平均分配hash槽
]# redis-trib.rb rebalance 192.168.4.51:6351
]# redis-trib.rb check 192.168.4.51:6351 查看
>>> Performing Cluster Check (using node 192.168.4.51:6351)
M: d9e8b7d81f4dff3d3157747e92b521220774da8b 192.168.4.51:6351
slots:0-5461 (5462 slots) master
1 additional replica(s)
S: 47b98f4c7d16486e5f8032e7a420675a025dd090 192.168.4.52:6352
slots: (0 slots) slave
replicates 2384a45084fec6db31e3331912c92379c45bd4bd
M: 3680239417226f1e0cc1b744e822de5c85b84890 192.168.4.53:6353
slots:5462-6826,12288-16383 (5461 slots) master
1 additional replica(s)
S: e528e231af6cb62ade68202e9fee2b5b090c9025 192.168.4.55:6355
slots: (0 slots) slave
replicates d9e8b7d81f4dff3d3157747e92b521220774da8b
M: 2384a45084fec6db31e3331912c92379c45bd4bd 192.168.4.56:6356
slots:6827-12287 (5461 slots) master
1 additional replica(s)
S: 43eb87ced708e2845e5b58618c817e2e1535e882 192.168.4.54:6354
slots: (0 slots) slave
replicates 3680239417226f1e0cc1b744e822de5c85b84890
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
把redis集群里的主机 恢复为独立的redis服务器(51-58)
]# redis-cli -h 192.168.4.51 -p 6351 shutdown
]# rm -rf /var/lib/redis/6379/*
]# vim /etc/redis/6379.conf (把与集群配置相关的 3项 注释)
]# /etc/init.d/redis_6379 start
]# netstat -utnlp | grep redis-server