Redislabs的简单使用与benchmark测试结果

Redislabs的简单使用与benchmark测试结果

---

自己的理解

• 仅是理解. 没有看过源码

哨兵可以实现高可用, 但是对高吞吐是没有太大帮助的. 
虽然可以实现主写从读, 但是在高并发的场景下延迟肯定会增加.

集群模式可以实现hash分片, 虽有有了一个根据key值进行hash的动作
增加了延时, 但是也增加系统容量, 在降低一定响应时间的情况下提高了系统容量.

能够扩展出更大的吞吐量. 降低redis单节点连接数过多的问题. 

---

背景

周六一早和晚上安装了下两个节点的Redislabs 
睡觉前简单整理了下redislabs 在openEuler上面的安装.
今天白天继续进行redislabs的相关工作处理

第一步 是让产品能够连的上redislabs 至少可以正常启动
第二步 可以跑一下一些基本的benchmark的方式验证一下性能
第三步 是需要进行功能和性能测试进行简单验证.

第三步我自己做不了,但是前两步应该还行. 然后基于此水这一篇文章.

---

第一部分: Spingboot连接redis

springboot 连接redis有很多种方式
我们产品复杂, 使用了jedis和redisson两种连接池. 
所以需要在两个地方进行设置.
# 不过我感觉太过复杂并不是好事情. 容易产生误解. 

jedis可以直接写. 在配置文件中: 
caching-configuration:
  enableRedis: true
  redisManagers:
  - name: default
    mode: cluster
    nodes: 10.110.139.191:14085,10.110.139.192:14085
    max-redirects: 8
    password: Yourpassword?!
# 注意 mode 是 cluster
# 注意 不是host 是 ndoes
# 注意 没有port了 是在nodes 里面直接添加上了. 

---

第一部分: Spingboot连接redis

redisson的稍微复杂一点,需要单独处理一个文档. 
yaml 文件里面需要修改一下配置:
yaml 内部的配置:
  redis:
    redisson:
      config: classpath:redisson.yaml

然后在runtime/libs 目录下面增加
redisson.yaml

---

第一部分: Spingboot连接redis

• redisson.yaml 的相关配置为:

clusterServersConfig:
  # 连接空闲超时，单位：毫秒 默认10000
  idleConnectionTimeout: 10000
  pingTimeout: 1000
  # 同任何节点建立连接时的等待超时。时间单位是毫秒 默认10000
  connectTimeout: 10000
  # 等待节点回复命令的时间。该时间从命令发送成功时开始计时。默认3000
  timeout: 3000
  # 命令失败重试次数
  retryAttempts: 3
  # 命令重试发送时间间隔，单位：毫秒
  retryInterval: 6000
  # 重新连接时间间隔，单位：毫秒
  reconnectionTimeout: 6000
  # 执行失败最大次数
  failedAttempts: 3
  # 密码
  password: Yourpasswords?!
  # 单个连接最大订阅数量
  subscriptionsPerConnection: 5
  clientName: null
  # loadBalancer 负载均衡算法类的选择
  loadBalancer: !<org.redisson.connection.balancer.RoundRobinLoadBalancer> {}
  #从节点发布和订阅连接的最小空闲连接数
  slaveSubscriptionConnectionMinimumIdleSize: 1
  #从节点发布和订阅连接池大小 默认值50
  slaveSubscriptionConnectionPoolSize: 50
  # 从节点最小空闲连接数 默认值32
  slaveConnectionMinimumIdleSize: 32
  # 从节点连接池大小 默认64
  # 感觉需要同步修改
  slaveConnectionPoolSize: 512
  # 主节点最小空闲连接数 默认32
  masterConnectionMinimumIdleSize: 32
  # 主节点连接池大小 默认64
  # 赵本帅增加默认值
  masterConnectionPoolSize: 512
  # 订阅操作的负载均衡模式
  subscriptionMode: SLAVE
  # 只在从服务器读取
  readMode: SLAVE
  # 集群地址
  nodeAddresses:
    - "redis://10.110.139.191:14085"
    - "redis://10.110.139.192:14085" 
  # 对Redis集群节点状态扫描的时间间隔。单位是毫秒。默认1000
  scanInterval: 1000
#这个线程池数量被所有RTopic对象监听器，RRemoteService调用者和RExecutorService任务共同共享。默认2
threads: 400
#这个线程池数量是在一个Redisson实例内，被其创建的所有分布式数据类型和服务，以及底层客户端所一同共享的线程池里保存的线程数量。默认2
nettyThreads: 400
# 编码方式 默认org.redisson.codec.JsonJacksonCodec
codec: !<org.redisson.codec.JsonJacksonCodec> {}
#传输模式
transportMode: NIO
# 分布式锁自动过期时间，防止死锁，默认30000
lockWatchdogTimeout: 30000
# 通过该参数来修改是否按订阅发布消息的接收顺序出来消息，如果选否将对消息实行并行处理，该参数只适用于订阅发布消息的情况, 默认true
keepPubSubOrder: true

---

第一部分: Spingboot连接redis

配置一定准确, 不要有错别字..
配置完成后应该就可以启动了
还可以通过 redislabs 的 8443
端口进行查询性能

---

第二部分: 性能验证

• 先说结论部分

1. redislabs的吞吐量跟同配置的开源版本并无大的提升. 
   集群模式下的损耗和shard导致吞吐量有下降. 
   但是企业版的着重点不在于吞吐量 而在于其他稳定性和监控等. 
2. 多线程能够大幅提高吞吐量, 并且能够规避掉仅能够使用一个CPU核心的问题
   今天测试了四线程的操作. PING命令的性能提升了接近3倍.MSET的性能提升2倍+
   认为有较大的优化空间.
3. OpenEuler2203的5.10的内存的同配置机器的性能表现要优于CentOS7 3.10内核的虚拟机. 
   5.10内核的机器 最高的吞吐量能到10万. 但是 3.10内核的基本上只有7万左右. 
   差距比较明显主要说的是 MSET (10 keys) 的性能数据. 
4. Redis5.0.4的性能明显比6.2.6的性能表现要好. 原因不明. 

---

第二部分: 性能测试

测试方法 redis-benchmark -h -p -a 导出数据 
转储到具体文件 ,然后执行如下语句进行分析
cat xxxx .log |grep -E "requests per second|====="

具体结果可以做成excel  但是大周末的 我只放原始数据算了..

既然是水 就要水到底...

---

第三部分: 性能测试原始数据

机器配置信息大部分为:
Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
CPU 为8C-16C
内存为 48-64G的虚拟机
磁盘为 HDD RAID6 12块 4T的盘
虚拟机操作系统分别为:
OpenEuler 2203 5.10 
CentOS7 2009   3.10

---

第三部分: 性能测试原始数据

这里准备将元数据放出. 
不进行完整的对比了
章节名字就是测试的场景.

---

redis 5.0.4 单线程 OpenEuler without swap

====== PING_INLINE ======
108813.92 requests per second
====== PING_BULK ======
104931.80 requests per second
====== SET ======
107874.87 requests per second
====== GET ======
105932.20 requests per second
====== INCR ======
108342.37 requests per second
====== LPUSH ======
107642.62 requests per second
====== RPUSH ======
108225.10 requests per second
====== LPOP ======
108342.37 requests per second
====== RPOP ======
105708.25 requests per second
====== SADD ======
105932.20 requests per second
====== HSET ======
109170.30 requests per second
====== SPOP ======
105708.25 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
109170.30 requests per second
====== LRANGE_100 (first 100 elements) ======
58582.31 requests per second
====== LRANGE_300 (first 300 elements) ======
23110.70 requests per second
====== LRANGE_500 (first 450 elements) ======
16641.71 requests per second
====== LRANGE_600 (first 600 elements) ======
12479.72 requests per second
====== MSET (10 keys) ======
110864.74 requests per second

---

6.2.6 OpenEuler with SWAP

====== PING_INLINE ======
107066.38 requests per second
====== PING_BULK ======
97560.98 requests per second
====== SET ======
106609.80 requests per second
====== GET ======
108459.87 requests per second
====== INCR ======
108342.37 requests per second
====== LPUSH ======
107296.14 requests per second
====== RPUSH ======
107411.38 requests per second
====== LPOP ======
110375.27 requests per second
====== RPOP ======
109649.12 requests per second
====== SADD ======
106382.98 requests per second
====== HSET ======
103092.78 requests per second
====== SPOP ======
105152.48 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
107642.62 requests per second
====== LRANGE_100 (first 100 elements) ======
51948.05 requests per second
====== LRANGE_300 (first 300 elements) ======
22691.17 requests per second
====== LRANGE_500 (first 450 elements) ======
17111.57 requests per second
====== LRANGE_600 (first 600 elements) ======
12597.63 requests per second
====== MSET (10 keys) ======
107066.38 requests per second

---

redis 6.2.6 OpenEuler 单线程 without Swap

====== PING_INLINE ======
104493.20 requests per second
====== PING_BULK ======
100200.40 requests per second
====== SET ======
105152.48 requests per second
====== GET ======
108108.11 requests per second
====== INCR ======
104058.27 requests per second
====== LPUSH ======
110253.59 requests per second
====== RPUSH ======
106269.93 requests per second
====== LPOP ======
110497.24 requests per second
====== RPOP ======
106496.27 requests per second
====== SADD ======
106496.27 requests per second
====== HSET ======
108695.65 requests per second
====== SPOP ======
106382.98 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
109289.62 requests per second
====== LRANGE_100 (first 100 elements) ======
48875.86 requests per second
====== LRANGE_300 (first 300 elements) ======
21910.60 requests per second
====== LRANGE_500 (first 450 elements) ======
16450.07 requests per second
====== LRANGE_600 (first 600 elements) ======
12280.49 requests per second
====== MSET (10 keys) ======
88417.33 requests per second

---

redislabs 企业版 OpenEuler 主节点测试主节点

====== PING_INLINE ======
67659.00 requests per second
====== PING_BULK ======
67980.97 requests per second
====== SET ======
63572.79 requests per second
====== GET ======
62814.07 requests per second
====== INCR ======
63331.22 requests per second
====== LPUSH ======
67385.45 requests per second
====== RPUSH ======
67842.61 requests per second
====== LPOP ======
68446.27 requests per second
====== RPOP ======
67704.80 requests per second
====== SADD ======
67704.80 requests per second
====== HSET ======
68073.52 requests per second
====== SPOP ======
67796.61 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
67842.61 requests per second
====== LRANGE_100 (first 100 elements) ======
43497.17 requests per second
====== LRANGE_300 (first 300 elements) ======
18559.76 requests per second
====== LRANGE_500 (first 450 elements) ======
13627.69 requests per second
====== LRANGE_600 (first 600 elements) ======
10636.03 requests per second
====== MSET (10 keys) ======
64641.24 requests per second

---

Redislabs OpenEuler 从节点测试主节点

====== PING_INLINE ======
70921.98 requests per second
====== PING_BULK ======
75357.95 requests per second
====== SET ======
83056.48 requests per second
====== GET ======
80321.28 requests per second
====== INCR ======
83056.48 requests per second
====== LPUSH ======
80321.28 requests per second
====== RPUSH ======
80710.25 requests per second
====== LPOP ======
76863.95 requests per second
====== RPOP ======
77821.02 requests per second
====== SADD ======
77519.38 requests per second
====== HSET ======
78125.00 requests per second
====== SPOP ======
74794.31 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
79113.92 requests per second
====== LRANGE_100 (first 100 elements) ======
45578.85 requests per second
====== LRANGE_300 (first 300 elements) ======
18037.52 requests per second
====== LRANGE_500 (first 450 elements) ======
13059.95 requests per second
====== LRANGE_600 (first 600 elements) ======
10339.12 requests per second
====== MSET (10 keys) ======
77220.08 requests per second

---

redislabs OpenEuler 从节点测试从节点

====== PING_INLINE ======
67024.13 requests per second
====== PING_BULK ======
66225.16 requests per second
====== SET ======
65530.80 requests per second
====== GET ======
68212.83 requests per second
====== INCR ======
67934.78 requests per second
====== LPUSH ======
63572.79 requests per second
====== RPUSH ======
62539.09 requests per second
====== LPOP ======
63091.48 requests per second
====== RPOP ======
61881.19 requests per second
====== SADD ======
62893.08 requests per second
====== HSET ======
62344.14 requests per second
====== SPOP ======
63091.48 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
61996.28 requests per second
====== LRANGE_100 (first 100 elements) ======
63051.70 requests per second
====== LRANGE_300 (first 300 elements) ======
62893.08 requests per second
====== LRANGE_500 (first 450 elements) ======
63331.22 requests per second
====== LRANGE_600 (first 600 elements) ======
62617.41 requests per second
====== MSET (10 keys) ======
68634.18 requests per second

6.2.6 centos7 3.10 保护模式 单节点

====== PING_INLINE ======
71073.21 requests per second
====== PING_BULK ======
69979.01 requests per second
====== SET ======
69013.11 requests per second
====== GET ======
71225.07 requests per second
====== INCR ======
72411.30 requests per second
====== LPUSH ======
74019.25 requests per second
====== RPUSH ======
74019.25 requests per second
====== LPOP ======
74183.98 requests per second
====== RPOP ======
74074.07 requests per second
====== SADD ======
71428.57 requests per second
====== HSET ======
73260.07 requests per second
====== SPOP ======
73475.39 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
74074.07 requests per second
====== LRANGE_100 (first 100 elements) ======
43802.02 requests per second
====== LRANGE_300 (first 300 elements) ======
19992.00 requests per second
====== LRANGE_500 (first 450 elements) ======
14723.20 requests per second
====== LRANGE_600 (first 600 elements) ======
11544.68 requests per second
====== MSET (10 keys) ======
71942.45 requests per second

---

6.2.6 centos7 3.10 非保护模式

====== PING_INLINE ======
71787.51 requests per second
====== PING_BULK ======
71073.21 requests per second
====== SET ======
71994.23 requests per second
====== GET ======
72150.07 requests per second
====== INCR ======
72202.16 requests per second
====== LPUSH ======
72098.05 requests per second
====== RPUSH ======
73099.41 requests per second
====== LPOP ======
73367.57 requests per second
====== RPOP ======
73313.78 requests per second
====== SADD ======
72516.32 requests per second
====== HSET ======
72992.70 requests per second
====== SPOP ======
71994.23 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
73529.41 requests per second
====== LRANGE_100 (first 100 elements) ======
43591.98 requests per second
====== LRANGE_300 (first 300 elements) ======
20036.06 requests per second
====== LRANGE_500 (first 450 elements) ======
14795.09 requests per second
====== LRANGE_600 (first 600 elements) ======
11498.22 requests per second
====== MSET (10 keys) ======
75815.01 requests per second

---

redis 6.2.6CentOS7 with out password without protected CentOS7 3.10

====== PING_INLINE ======
72621.64 requests per second
====== PING_BULK ======
72833.21 requests per second
====== SET ======
73046.02 requests per second
====== GET ======
72411.30 requests per second
====== INCR ======
74019.25 requests per second
====== LPUSH ======
73421.44 requests per second
====== RPUSH ======
73313.78 requests per second
====== LPOP ======
74294.21 requests per second
====== RPOP ======
73800.73 requests per second
====== SADD ======
72886.30 requests per second
====== HSET ======
72780.20 requests per second
====== SPOP ======
72150.07 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
73909.83 requests per second
====== LRANGE_100 (first 100 elements) ======
44091.71 requests per second
====== LRANGE_300 (first 300 elements) ======
20695.37 requests per second
====== LRANGE_500 (first 450 elements) ======
14779.78 requests per second
====== LRANGE_600 (first 600 elements) ======
11739.85 requests per second
====== MSET (10 keys) ======
75987.84 requests per second

---

Redis 6.2.6 centos 7 3.10 内核 四线程测试数据

====== PING_INLINE ======
  throughput summary: 199203.20 requests per second
====== PING_MBULK ======
  throughput summary: 199203.20 requests per second
====== SET ======
  throughput summary: 199600.80 requests per second
====== GET ======
  throughput summary: 199203.20 requests per second
====== INCR ======
  throughput summary: 199600.80 requests per second
====== LPUSH ======
  throughput summary: 199203.20 requests per second
====== RPUSH ======
  throughput summary: 199203.20 requests per second
====== LPOP ======
  throughput summary: 199203.20 requests per second
====== RPOP ======
  throughput summary: 199203.20 requests per second
====== SADD ======
  throughput summary: 199203.20 requests per second
====== HSET ======
  throughput summary: 199203.20 requests per second
====== SPOP ======
  throughput summary: 199203.20 requests per second
====== ZADD ======
  throughput summary: 199203.20 requests per second
====== ZPOPMIN ======
  throughput summary: 199203.20 requests per second
====== LPUSH (needed to benchmark LRANGE) ======
  throughput summary: 199203.20 requests per second
====== LRANGE_100 (first 100 elements) ======
  throughput summary: 79744.82 requests per second
====== LRANGE_300 (first 300 elements) ======
  throughput summary: 44286.98 requests per second
====== LRANGE_500 (first 500 elements) ======
  throughput summary: 28457.60 requests per second
====== LRANGE_600 (first 600 elements) ======
  throughput summary: 24888.00 requests per second
====== MSET (10 keys) ======
  throughput summary: 132802.12 requests per second