Redis 学习之集群

该文使用centos6.5 64位  redis3.2.8

一、  redis-cluster架构图

集群通信：所有redis节点之间通过PING-PONG机制彼此互联，内部使用二进制鞋子优化传输速度和带宽。

集群数据存储（哈希槽）：

（1）redis-cluster把所有的物理节点映射到[0-16383]slot上,cluster 负责维护node<->slot<->value
（2）Redis 集群中内置了 16384 个哈希槽，当需要在 Redis 集群中放置一个 key-value 时，redis 先对 key 使用 crc16 算法算出一个结果，然后把结果对 16384 求余数，这样每个 key 都会对应一个编号在 0-16383 之间的哈希槽，redis 会根据节点数量大致均等的将哈希槽映射到不同的节点。

举例说明数据如何选择存储在哪台redis服务器上：

现在有3台redis服务器的集群：redis Server1(0-5000)  redis Server1(5001-10000)  redis Server1(10001-16384)

存储数据：set key A

A存储在哪个节点上：

计算A的hash值，例如值为98，98这个槽在redis Server1上，所以A应该放到redis Server1。

节点之间如何确定某个节点挂掉:容错机制

判断节点是否挂掉：某个节点（黄色节点）给红色节点发送PING命令，但未得带PONG回应，于是该节点怀疑红色节点已经挂掉，于是他把这个猜想通知给其他节点，其他节点就会向红色节点发送PING命令，如果没有PONG回应，则也认为红色节点已经挂掉，如果半数以上master节点与master节点通信超过(cluster-node-timeout),认为当前master节点挂掉。

什么时候整个集群不可用(cluster_state:fail):

  a:如果集群任意master挂掉,且当前master没有slave.集群进入fail状态,也可以理解成集群的slot映射[0-16383]不完整时进入fail状态，所以一般我们会为每一个master节点做从节点slave 。ps : redis-3.0.0.rc1加入cluster-require-full-coverage参数,默认关闭,打开集群兼容部分失败.

  b:如果集群超过半数以上master挂掉，无论是否有slave,集群进入fail状态.

  ps:当集群不可用时,所有对集群的操作做都不可用，收到((error) CLUSTERDOWN The cluster is down)错误

 二、搭建redis集群

集群中有三个节点的集群，每个节点有一主一备。需要6台虚拟机。

在这里我们搭建一个伪分布式的集群，使用6个redis实例来模拟。

1、集群需要ruby环境：

安装ruby
yum install ruby
yum install rubygems

2.查看redis集群管理工具redis-trib.rb是否存在

[root@localhost src]# pwd
/tools/redis-3.2.8/src
[root@localhost src]# ll *.rb
-rwxrwxr-x. 1 root root 60852 2月  12 23:14 redis-trib.rb
[root@localhost src]# 

3、安装redis-trib.rb脚本运行依赖的环境 redis-3.2.2.gem

   安装ruby的包：gem install redis-3.2.2.gem

 4、创建6个redis实例

[root@localhost local]# pwd
/usr/local
[root@localhost local]# mkdir redis-cluster
[root@localhost local]# cp -r redis redis-cluster/redis01  //复制我们安装的单机版redis实例

 //删除复制后的 dump.rdb 文件

  [root@localhost bin]# pwd
  /usr/local/redis-cluster/redis01/bin

 [root@localhost bin]# rm -rf dump.rdb

//修改配置文件 port    与 cluster-enabled

# Accept connections on the specified port, default is 6379 (IANA #815344).
# If port 0 is specified Redis will not listen on a TCP socket.
port 7001

################################ REDIS CLUSTER  ###############################
#
# ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
# WARNING EXPERIMENTAL: Redis Cluster is considered to be stable code, however
# in order to mark it as "mature" we need to wait for a non trivial percentage
# of users to deploy it in production.
# ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
#
# Normal Redis instances can't be part of a Redis Cluster; only nodes that are
# started as cluster nodes can. In order to start a Redis instance as a
# cluster node enable the cluster support uncommenting the following:
#
 cluster-enabled yes  //开启集群功能

将redis01 复制5份

[root@localhost redis-cluster]# cp -r redis01/ redis02
[root@localhost redis-cluster]# cp -r redis01/ redis03
[root@localhost redis-cluster]# cp -r redis01/ redis04
[root@localhost redis-cluster]# cp -r redis01/ redis05
[root@localhost redis-cluster]# cp -r redis01/ redis06

修改他们的端口为 redis02:7002    redis03:7003    redis04:7004    redis05:7005    redis06:7006

5、将集群依赖的ruby脚本redis-trib.rb 移动到  /usr/local/redis-cluster 目录下便于使用

  [root@localhost src]# pwd
  /tools/redis-3.2.8/src
  [root@localhost src]# cp *.rb /usr/local/redis-cluster

6、启动6各redis实例 使用脚本启动

[root@localhost redis-cluster]# vim startRedis.sh

cd redis01/bin
./redis-server ../etc/redis.conf
cd ../../
cd redis02/bin
./redis-server ../etc/redis.conf
cd ../../
cd redis03/bin
./redis-server ../etc/redis.conf
cd ../../
cd redis04/bin
./redis-server ../etc/redis.conf
cd ../../
cd redis05/bin
./redis-server ../etc/redis.conf
cd ../../
cd redis06/bin
./redis-server ../etc/redis.conf
cd ../../

赋予该文件可执行权限 ：[root@localhost redis-cluster]# chmod +x startRedis.sh

7、启动脚本并查看6各redis实例是否启动成功

[root@localhost redis-cluster]# ./startRedis.sh 
[root@localhost redis-cluster]# ps -ef|grep redis
root      1634     1  0 17:11 ?        00:00:00 ./redis-server 0.0.0.0:7001 [cluster]
root      1637     1  0 17:11 ?        00:00:00 ./redis-server 0.0.0.0:7003 [cluster]
root      1644     1  0 17:11 ?        00:00:00 ./redis-server 0.0.0.0:7005 [cluster]
root      1939     1  0 17:15 ?        00:00:00 ./redis-server 0.0.0.0:7002 [cluster]
root      1943     1  0 17:15 ?        00:00:00 ./redis-server 0.0.0.0:7004 [cluster]
root      1951     1  0 17:15 ?        00:00:00 ./redis-server 0.0.0.0:7006 [cluster]
root      1955  1463  0 17:15 pts/0    00:00:00 grep redis

8、创建集群

[root@localhost redis-cluster]# ./redis-trib.rb create --replicas 1 192.168.6.190:7001 192.168.6.190:7002 192.168.6.190:7003 192.168.6.190:7004 192.168.6.190:7005 192.168.6.190:7006

[root@localhost redis-cluster]# ./redis-trib.rb create --replicas 1 192.168.6.190:7001 192.168.6.190:7002 192.168.6.190:7003 192.168.6.190:7004 192.168.6.190:7005 192.168.6.190:7006
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.6.190:7001
192.168.6.190:7002
192.168.6.190:7003
Adding replica 192.168.6.190:7004 to 192.168.6.190:7001
Adding replica 192.168.6.190:7005 to 192.168.6.190:7002
Adding replica 192.168.6.190:7006 to 192.168.6.190:7003
M: 6e8c35d55d3699afe0b10317cf9c199e7df1ae50 192.168.6.190:7001
   slots:0-5460 (5461 slots) master     //slots槽的分配
M: 4cc06e1060369a16f70d4f8097c065bf886162a3 192.168.6.190:7002
   slots:5461-10922 (5462 slots) master
M: 532355022ea3473834726e7512270025b7eddae3 192.168.6.190:7003
   slots:10923-16383 (5461 slots) master
S: de789ed503705e7a5a2e5fc41adb7ed3290de08c 192.168.6.190:7004
   replicates 6e8c35d55d3699afe0b10317cf9c199e7df1ae50
S: c1261800c3a7e74ff3b2fda22b07ef994f7db4a7 192.168.6.190:7005
   replicates 4cc06e1060369a16f70d4f8097c065bf886162a3
S: 07910c0d839e4fcd6139598edd4953fcd32adb29 192.168.6.190:7006
   replicates 532355022ea3473834726e7512270025b7eddae3
Can I set the above configuration? (type 'yes' to accept): yes  输入yes创建集群
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join....
>>> Performing Cluster Check (using node 192.168.6.190:7001)
M: 6e8c35d55d3699afe0b10317cf9c199e7df1ae50 192.168.6.190:7001   M:主节点  S:从节点
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 532355022ea3473834726e7512270025b7eddae3 192.168.6.190:7003
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
S: de789ed503705e7a5a2e5fc41adb7ed3290de08c 192.168.6.190:7004
   slots: (0 slots) slave
   replicates 6e8c35d55d3699afe0b10317cf9c199e7df1ae50
S: c1261800c3a7e74ff3b2fda22b07ef994f7db4a7 192.168.6.190:7005
   slots: (0 slots) slave
   replicates 4cc06e1060369a16f70d4f8097c065bf886162a3
M: 4cc06e1060369a16f70d4f8097c065bf886162a3 192.168.6.190:7002
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: 07910c0d839e4fcd6139598edd4953fcd32adb29 192.168.6.190:7006
   slots: (0 slots) slave
   replicates 532355022ea3473834726e7512270025b7eddae3
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

9、测试集群

使用6各redis实例中任意一个客户端链接集群：

[root@localhost redis-cluster]# redis02/bin/redis-cli -h 192.168.6.190 -p 7002 -c

[root@localhost redis-cluster]# redis02/bin/redis-cli -h 192.168.6.190 -p 7002 -c
192.168.6.190:7002> set age 100
-> Redirected to slot [741] located at 192.168.6.190:7001  
OK

在这里我们登陆了集群的7002redis服务器，age的hash值是741于是就把该数据存储在了7001节点上，应为该节点的槽范围是 （slots:0-5460）。

10、关闭集群

我们创建一个脚本文件redisShutdown.sh 

redis01/bin/redis-cli -p 7001 shutdown
redis02/bin/redis-cli -p 7002 shutdown
redis03/bin/redis-cli -p 7003 shutdown
redis04/bin/redis-cli -p 7004 shutdown
redis05/bin/redis-cli -p 7005 shutdown
redis06/bin/redis-cli -p 7006 shutdown

到此集群搭建完毕。 ps:单机版redis默认有16个数据库，集群中只有一个数据库即0号库