redis集群节点重启后恢复

服务器重启后，集群报错：

[root@SHH-HQ-NHS11S nhsuser]# redis-cli -c -h ip -p 7000
ip:7000> set cc dd
(error) CLUSTERDOWN Hash slot not served

检查集群节点情况

> cluster nodes
c0767666bee76e5e0dc67f24031a3e1b574235cc :7000@17000 myself,master - 0 0 0 connected

发现集群的节点只剩一个，其余节点都不见了。

将集群节点加入：

# redis-cli --cluster add-node ip:7005 ip:7000
>>> Adding node ip:7005 to cluster ip:7000
>>> Performing Cluster Check (using node ip:7000)
M: c0767666bee76e5e0dc67f24031a3e1b574235cc ip:7000
   slots: (0 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[ERR] Not all 16384 slots are covered by nodes.

这个往往是由于主node移除了，但是并没有移除node上面的slot，从而导致了slot总数没有达到16384，其实也就是slots分布不正确。所以在删除节点的时候一定要注意删除的是否是Master主节点。现在这样情况，我们先修复集群：

# redis-cli --cluster fix IP:7000

>>> Covering slot 9675 with ip:7000

有点久,稍等下

然后检查修复情况:

# redis-cli --cluster check ip:7000
ip:7000 (c0767666...) -> 0 keys | 16384 slots | 0 slaves.
[OK] 0 keys in 1 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node ip:7000)
M: c0767666bee76e5e0dc67f24031a3e1b574235cc IP:7000
   slots:[0-16383] (16384 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

将节点加入集群:

[root@SHH-HQ-NHS11S nhsuser]# redis-cli --cluster add-node ip:7001 ip:7000
>>> Adding node ip:7001 to cluster ip:7000
>>> Performing Cluster Check (using node ip:7000)
M: c0767666bee76e5e0dc67f24031a3e1b574235cc ip:7000
   slots:[0-16383] (16384 slots) master
M: a98432e520af41117f5d8a81e2c0e2a430940c7e ip:7003
   slots: (0 slots) master
M: 2ff0c7ee051e26f484b6826d6aa8a5a3fbc93d17 ip:7002
   slots: (0 slots) master
M: 26bf8ce786d7c9d0bbb9d9d78978497f86fbbe0a ip:7005
   slots: (0 slots) master
M: 0a7718d2c53a16debdbfb391a7acd23a50a05faa ip:7004
   slots: (0 slots) master
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node ip:7001 to make it join the cluster.
[OK] New node added correctly.

测试一把：

[root@SHH-HQ-NHS11S nhsuser]# redis-cli -c -h IP -p 7000
IP:7000> set aa dd
OK
IP:7000> get aa
"dd"
IP:7000>