redis cluster和小哨兵组合 高可用方案是一种全自动的办法
cluster 主要编辑3个参数
cluster-enabled yes #开启cluster，去掉注释
cluster-config-file nodes.conf
cluster-node-timeout 15000
启动server
[redis@redis02 ~]$ cat s.sh pkill -9 redis redis-server /u01/redis/redis-5.0.5/conf/7001/7001.conf redis-server /u01/redis/redis-5.0.5/conf/7003/7003.conf redis-server /u01/redis/redis-5.0.5/conf/7004/7004.conf ps -ef|grep redis-server|grep -v grep [redis@redis02 ~]$ sh s.sh redis 15089 1 0 22:07 ? 00:00:00 redis-server 127.0.0.1:7001 [cluster] redis 15094 1 0 22:07 ? 00:00:00 redis-server 127.0.0.1:7003 [cluster] redis 15099 1 0 22:07 ? 00:00:00 redis-server 127.0.0.1:7004 [cluster]
创建cluster，redis5.0开始不再使用ruby搭建集群
[redis@redis01 ~]$ redis-cli --cluster create 192.168.20.201:7001 192.168.20.201:7002 192.168.20.202:7001 >>> Performing hash slots allocation on 3 nodes... Master[0] -> Slots 0 - 5460 Master[1] -> Slots 5461 - 10922 Master[2] -> Slots 10923 - 16383 M: 797d123e6f8fb94260490b543cb8a5adce0d90db 192.168.20.201:7001 slots:[0-5460] (5461 slots) master M: 4a3aff53bd451c3df2c497d576dad674fbf2ff26 192.168.20.201:7002 slots:[10923-16383] (5461 slots) master M: fe3f033c4a899c032ac9c0696b89e7be7e4975b2 192.168.20.202:7001 slots:[5461-10922] (5462 slots) master Can I set the above configuration? (type 'yes' to accept): yes >>> Nodes configuration updated >>> Assign a different config epoch to each node >>> Sending CLUSTER MEET messages to join the cluster Waiting for the cluster to join ... >>> Performing Cluster Check (using node 192.168.20.201:7001) M: 797d123e6f8fb94260490b543cb8a5adce0d90db 192.168.20.201:7001 slots:[0-5460] (5461 slots) master M: fe3f033c4a899c032ac9c0696b89e7be7e4975b2 192.168.20.202:7001 slots:[5461-10922] (5462 slots) master M: 4a3aff53bd451c3df2c497d576dad674fbf2ff26 192.168.20.201:7002 slots:[10923-16383] (5461 slots) master [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
查看cluster信息
[redis@redis01 ~]$ redis-cli -h 192.168.20.201 -p 7001 cluster info cluster_state:ok cluster_slots_assigned:16384 cluster_slots_ok:16384 cluster_slots_pfail:0 cluster_slots_fail:0 cluster_known_nodes:3 cluster_size:3 cluster_current_epoch:3 cluster_my_epoch:1 cluster_stats_messages_ping_sent:256 cluster_stats_messages_pong_sent:281 cluster_stats_messages_sent:537 cluster_stats_messages_ping_received:279 cluster_stats_messages_pong_received:256 cluster_stats_messages_meet_received:2 cluster_stats_messages_received:537 [redis@redis01 ~]$ redis-cli -h 192.168.20.201 -p 7001 cluster nodes fe3f033c4a899c032ac9c0696b89e7be7e4975b2 192.168.20.202:7001@17001 master - 0 1568682616539 3 connected 5461-10922 4a3aff53bd451c3df2c497d576dad674fbf2ff26 192.168.20.201:7002@17002 master - 0 1568682618047 2 connected 10923-16383 797d123e6f8fb94260490b543cb8a5adce0d90db 192.168.20.201:7001@17001 myself,master - 0 1568682615000 1 connected 0-5460
添加从节点
[redis@redis01 ~]$ redis-cli --cluster add-node 192.168.20.201:7003 192.168.20.202:7001 --cluster-slave >>> Adding node 192.168.20.201:7003 to cluster 192.168.20.202:7001 >>> Performing Cluster Check (using node 192.168.20.202:7001) M: fe3f033c4a899c032ac9c0696b89e7be7e4975b2 192.168.20.202:7001 slots:[5461-10922] (5462 slots) master M: 4a3aff53bd451c3df2c497d576dad674fbf2ff26 192.168.20.201:7002 slots:[10923-16383] (5461 slots) master M: 797d123e6f8fb94260490b543cb8a5adce0d90db 192.168.20.201:7001 slots:[0-5460] (5461 slots) master [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. Automatically selected master 192.168.20.202:7001 >>> Send CLUSTER MEET to node 192.168.20.201:7003 to make it join the cluster. Waiting for the cluster to join
查看nodes
[redis@redis01 ~]$ redis-cli -h 192.168.20.201 -p 7001 cluster nodes 4a3aff53bd451c3df2c497d576dad674fbf2ff26 192.168.20.201:7002@17002 master - 0 1568712829454 2 connected 10923-16383 4464a46e8f73931feea8991c6ce9bde7277c41aa 192.168.20.202:7003@17003 slave 797d123e6f8fb94260490b543cb8a5adce0d90db 0 1568712831825 1 connected fe3f033c4a899c032ac9c0696b89e7be7e4975b2 192.168.20.202:7001@17001 master - 0 1568712832857 3 connected 5461-10922 7fa3dc96bf77976e0612aa55f872ddb3c9f271f1 192.168.20.202:7004@17004 slave 4a3aff53bd451c3df2c497d576dad674fbf2ff26 0 1568712831000 2 connected 797d123e6f8fb94260490b543cb8a5adce0d90db 192.168.20.201:7001@17001 myself,master - 0 1568712829000 1 connected 0-5460 39336bfdf1b0c35435ecebe66afb6f79e78339ba 192.168.20.201:7003@17003 slave fe3f033c4a899c032ac9c0696b89e7be7e4975b2 0 1568712830537 3 connected
添加master
[redis@redis01 ~]$ redis-cli --cluster add-node 192.168.20.201:7003 192.168.20.202:7001 >>> Adding node 192.168.20.201:7003 to cluster 192.168.20.202:7001 >>> Performing Cluster Check (using node 192.168.20.202:7001) M: fe3f033c4a899c032ac9c0696b89e7be7e4975b2 192.168.20.202:7001 slots:[5461-10922] (5462 slots) master M: 4a3aff53bd451c3df2c497d576dad674fbf2ff26 192.168.20.201:7002 slots:[10923-16383] (5461 slots) master M: 797d123e6f8fb94260490b543cb8a5adce0d90db 192.168.20.201:7001 slots:[0-5460] (5461 slots) master [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered. >>> Send CLUSTER MEET to node 192.168.20.201:7003 to make it join the cluster. [OK] New node added correctly. [redis@redis01 ~]$ redis-cli -h 192.168.20.201 -p 7002 cluster nodes fe3f033c4a899c032ac9c0696b89e7be7e4975b2 192.168.20.202:7001@17001 master - 0 1568711371227 3 connected 5461-10922 046582928a7b3d21aa2a344d01f182c489e7f26d 192.168.20.201:7003@17003 master - 0 1568711370000 0 connected 797d123e6f8fb94260490b543cb8a5adce0d90db 192.168.20.201:7001@17001 master - 0 1568711372564 1 connected 0-5460 4a3aff53bd451c3df2c497d576dad674fbf2ff26 192.168.20.201:7002@17002 myself,master - 0 1568711371000 2 connected 10923-16383
删除节点 和 删除复制节点命令是一样的
[redis@redis02 ~]$ redis-cli --cluster del-node 192.168.20.201:7003 100a5f359104a7e13e1bccd02677032cfab667ea >>> Removing node 100a5f359104a7e13e1bccd02677032cfab667ea from cluster 192.168.20.201:7003 >>> Sending CLUSTER FORGET messages to the cluster... >>> SHUTDOWN the node.
检查节点
[redis@redis01 ~]$ redis-cli --cluster check 192.168.20.201:7002 Could not connect to Redis at 192.168.20.201:7001: Connection refused 192.168.20.201:7002 (4a3aff53...) -> 1 keys | 5461 slots | 1 slaves. 192.168.20.202:7001 (fe3f033c...) -> 1 keys | 5462 slots | 1 slaves. 192.168.20.202:7003 (4464a46e...) -> 0 keys | 5461 slots | 0 slaves. [OK] 2 keys in 3 masters. 0.00 keys per slot on average. >>> Performing Cluster Check (using node 192.168.20.201:7002) M: 4a3aff53bd451c3df2c497d576dad674fbf2ff26 192.168.20.201:7002 slots:[10923-16383] (5461 slots) master 1 additional replica(s) M: fe3f033c4a899c032ac9c0696b89e7be7e4975b2 192.168.20.202:7001 slots:[5461-10922] (5462 slots) master 1 additional replica(s) S: 39336bfdf1b0c35435ecebe66afb6f79e78339ba 192.168.20.201:7003 slots: (0 slots) slave replicates fe3f033c4a899c032ac9c0696b89e7be7e4975b2 S: 7fa3dc96bf77976e0612aa55f872ddb3c9f271f1 192.168.20.202:7004 slots: (0 slots) slave replicates 4a3aff53bd451c3df2c497d576dad674fbf2ff26 M: 4464a46e8f73931feea8991c6ce9bde7277c41aa 192.168.20.202:7003 slots:[0-5460] (5461 slots) master [OK] All nodes agree about slots configuration. >>> Check for open slots... >>> Check slots coverage... [OK] All 16384 slots covered.
搭建cluster过程当中遇到的错误
错误1
[redis@redis01 redis-5.0.5]$ make install
cd src && make install
make[1]: Entering directory `/u01/redis/redis-5.0.5/src'
Hint: It's a good idea to run 'make test' ;)
INSTALL install
install: cannot create regular file `/usr/local/bin/redis-server': Permission denied
make[1]: *** [install] Error 1
make[1]: Leaving directory `/u01/redis/redis-5.0.5/src'
make: *** [install] Error 2
make install 使用root用户执行
错误2
[redis@redis01 ~]$ redis-cli --cluster create 192.168.20.201:7001 192.168.20.201:7002 192.168.20.202:7001
[ERR] Node 192.168.20.201:7001 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.
[redis@redis01 ~]$ redis-cli -h 192.168.20.201 -p 7001 CLUSTER NODES
571fc5a63b009d201c50a90d45fa23452d43d596 127.0.0.1:7001@17001 myself,master - 0 1568642020827 1 connected 0-4095 [7365-<-f3f3d854bce9e15955bfcfe78d71c943025d7b30] [11298-<-0f5ee2036d799381612b76f423072d3760463697] [15363-<-70ff508977ef8bd4422d073dd5f79cefc164d193] [15495-<-70ff508977ef8bd4422d073dd5f79cefc164d193]
0f5ee2036d799381612b76f423072d3760463697 :0@0 master,fail?,noaddr - 1568642020835 1568642020827 3 disconnected 8192-12287
70ff508977ef8bd4422d073dd5f79cefc164d193 127.0.0.1:7004@17004 master,fail? - 1568642020835 1568642020827 4 disconnected 12288-16383
f3f3d854bce9e15955bfcfe78d71c943025d7b30 :0@0 master,fail?,noaddr - 1568642020835 1568642020827 2 disconnected 4096-8191
[redis@redis01 ~]$ sh s.sh
redis 14997 1 0 22:16 ? 00:00:00 redis-server /u01/redis/redis-5.0.5/conf/7001/7001.conf
redis 14999 1 0 22:16 ? 00:00:00 redis-server /u01/redis/redis-5.0.5/conf/7002/7002.conf
redis 15001 1 0 22:16 ? 00:00:00 redis-server /u01/redis/redis-5.0.5/conf/7003/7003.conf
[redis@redis01 ~]$ redis-cli -h 192.168.20.201 -p 7001 CLUSTER NODES
13f63e9b5a7696181f7ae06f7ffc08b3afbb5cc3 :7001@17001 myself,master - 0 0 0 connected 3168 3300 7365 11298 15363 15495
将每个节点下aof、rdb、nodes.conf本地备份文件删除
flushdb
错误3
Waiting for the cluster to join hang
修改conf文件中 bind 修改成真实的ip