环境准备
| 主机名 | sht-sgmhadoopdn-01 | sht-sgmhadoopdn-02 | sht-sgmhadoopdn-03 |
| OS | CentOS Linux release 7.6.1810 (Core) | ||
| IP地址 | 172.16.101.58 | 172.16.101.59 | 172.16.101.60 |
| redis安装路径 | /usr/local/redis | ||
| redis角色 | master | slave | slave |
| sentinel | yes | yes | yes |
redis各版本下载地址
http://download.redis.io/releases/
此处我们选择3.2.12版本
http://download.redis.io/releases/redis-3.2.12.tar.gz
一. 各节点系统环境配置
1. 新建系统用户redis
# groupadd -r dba # useradd -r -g dba -G root redis
# cat /home/redis/.bash_profile
................
PATH=/usr/local/redis/src:$PATH:$HOME/.local/bin:$HOME/bin
................
2. 内核参数优化
# cat /etc/sysctl.conf ....................... vm.overcommit_memory=1 vm.swappiness=1 fs.aio-max-nr=1048576 fs.file-max= 7672460 net.ipv4.ip_local_port_range=9000 65500 net.core.rmem_default=262144 net.core.rmem_max=4194304 net.core.wmem_default=262144 net.core.wmem_max=1048586 net.core.somaxconn = 2048 kernel.sem= 50100 64128000 50100 1280 kernel.shmall=5242880 kernel.shmmax=12884901888 .......................
3. 系统资源使用优化
redis soft nproc 65536 redis hard nproc 65536 redis soft nofile 65536 redis hard nofile 65536
4. 大内存页优化
# echo "echo never > /sys/kernel/mm/transparent_hugepage/enabled" >> /etc/rc.local
二. 各节点redis安装
$ cd /usr/local/redis
$ tar -zxf redis-3.2.12.tar.gz $ mv redis-3.2.12/* . $ rm -rf redis-3.2.12* $ make
$ mkdir {data,log,conf}
$ mv redis.conf sentinel.conf conf
三.修改redis主配置文件redis.conf，这里仅列出需要的参数
bind 0.0.0.0 protected-mode no tcp-backlog 1024 tcp-keepalive 0 supervised systemd daemonize yes logfile "/usr/local/redis/log/redis.log" dir "/usr/local/redis/data" maxmemory 4gb maxmemory-policy allkeys-lru maxclients 65000 appendonly yes
appendfilename "appendonly.aof" rename-command flushall "" rename-command flushdb "" rename-command save "" rename-command keys "" repl-ping-slave-period 3 repl-timeout 15 min-slaves-to-write 1 min-slaves-max-lag 10
注意：
- maxmemory内存为系统总内存的2/3左右
- 在slave节点配置文件需要添加“slaveof 172.16.101.58 6379”
四. 在各节点将redis配置为systemd服务，并启动redis服务
[Unit] Description=Redis In-Memory Data Store Documentation=https://redis.io/documentation After=syslog.target After=network.target [Service] Type=notify User=redis Group=dba Restart=always # Disable OOM kill on the Redis OOMScoreAdjust=-1000 Environment=PG_OOM_ADJUST_FILE=/proc/self/oom_score_adj Environment=PG_OOM_ADJUST_VALUE=0 ExecStart=/usr/local/redis/src/redis-server /usr/local/redis/conf/redis.conf ExecStop=/bin/kill -15 $MAINPID LimitNOFILE=65536 [Install] WantedBy=multi-user.target
# systemctl start redis
# systemctl enable redis
# systemctl status redis
● redis.service - Redis In-Memory Data Store
Loaded: loaded (/usr/lib/systemd/system/redis.service; enabled; vendor preset: disabled)
Active: active (running) since Sat 2020-03-28 21:15:36 CST; 1s ago
Docs: https://redis.io/documentation
Process: 12897 ExecStop=/bin/kill -15 $MAINPID (code=exited, status=1/FAILURE)
Main PID: 12945 (redis-server)
CGroup: /system.slice/redis.service
└─12945 /usr/local/redis/src/redis-server 0.0.0.0:6379
Mar 28 21:15:36 sht-sgmhadoopdn-01 systemd[1]: Starting Redis In-Memory Data Store...
Mar 28 21:15:36 sht-sgmhadoopdn-01 systemd[1]: Started Redis In-Memory Data Store.
redis master启动成功的log如下
12945:C 28 Mar 21:15:36.238 * supervised by systemd, will signal readiness
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 3.2.12 (00000000/0) 64 bit
.-`` .-```. ```/ _.,_ ''-._
( ' , .-` | `, ) Running in standalone mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 6379
| `-._ `._ / _.-' | PID: 12945
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
12945:M 28 Mar 21:15:36.241 # Server started, Redis version 3.2.12
12945:M 28 Mar 21:15:36.241 * DB loaded from disk: 0.000 seconds
12945:M 28 Mar 21:15:36.242 * The server is now ready to accept connections on port 6379
redis slave节点启动成功的log如下
5116:C 28 Mar 21:45:36.336 * supervised by systemd, will signal readiness
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 3.2.12 (00000000/0) 64 bit
.-`` .-```. ```/ _.,_ ''-._
( ' , .-` | `, ) Running in standalone mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 6379
| `-._ `._ / _.-' | PID: 5116
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
5116:S 28 Mar 21:45:36.342 # Server started, Redis version 3.2.12
5116:S 28 Mar 21:45:36.342 * DB loaded from disk: 0.000 seconds
5116:S 28 Mar 21:45:36.342 * The server is now ready to accept connections on port 6379
5116:S 28 Mar 21:45:36.342 * Connecting to MASTER 172.16.101.58:6379
5116:S 28 Mar 21:45:36.342 * MASTER <-> SLAVE sync started
5116:S 28 Mar 21:45:36.343 * Non blocking connect for SYNC fired the event.
5116:S 28 Mar 21:45:36.344 * Master replied to PING, replication can continue...
5116:S 28 Mar 21:45:36.344 * Partial resynchronization not possible (no cached master)
5116:S 28 Mar 21:45:36.345 * Full resync from master: e6eb0ae9701e18de4080032853185db5c748267a:1093
5116:S 28 Mar 21:45:36.406 * MASTER <-> SLAVE sync: receiving 77 bytes from master
5116:S 28 Mar 21:45:36.406 * MASTER <-> SLAVE sync: Flushing old data
5116:S 28 Mar 21:45:36.406 * MASTER <-> SLAVE sync: Loading DB in memory
5116:S 28 Mar 21:45:36.406 * MASTER <-> SLAVE sync: Finished with success
五，验证master、slave节点复制情况
1. 查看主从状态
在master节点查看
$ redis-cli info replication
# Replication role:master connected_slaves:2 min_slaves_good_slaves:2 slave0:ip=172.16.101.60,port=6379,state=online,offset=1877,lag=1 slave1:ip=172.16.101.59,port=6379,state=online,offset=1877,lag=1 master_repl_offset:1877 repl_backlog_active:1 repl_backlog_size:1048576 repl_backlog_first_byte_offset:2 repl_backlog_histlen:1876
在slave节点1查看
# Replication role:slave master_host:172.16.101.58 master_port:6379 master_link_status:up master_last_io_seconds_ago:0 master_sync_in_progress:0 slave_repl_offset:2185 slave_priority:100 slave_read_only:1 connected_slaves:0 min_slaves_good_slaves:0 master_repl_offset:0 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0
在slave2节点查看
# Replication role:slave master_host:172.16.101.58 master_port:6379 master_link_status:up master_last_io_seconds_ago:1 master_sync_in_progress:0 slave_repl_offset:2451 slave_priority:100 slave_read_only:1 connected_slaves:0 min_slaves_good_slaves:0 master_repl_offset:0 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0
2. 测试复制情况
在主节点写入数据
172.16.101.58:6379> set string "This is Redis Replication." OK 172.16.101.58:6379> get string "This is Redis Replication."
分别在从节点查看
172.16.101.59:6379> scan 10 1) "0" 2) 1) "string" 172.16.101.59:6379> get string "This is Redis Replication."
172.16.101.60:6379> scan 10 1) "0" 2) 1) "string" 172.16.101.60:6379> get string "This is Redis Replication."
六.哨兵部署
1. 修改各节点sentinel.conf文件，内容如下，并将该文件复制到所有节点
protected-mode no port 26379 supervised systemd dir "/usr/local/redis/data" logfile "/usr/local/redis/log/sentinel.log" sentinel monitor mymaster 172.16.101.58 6379 2 sentinel down-after-milliseconds mymaster 5000 sentinel failover-timeout mymaster 60000
2. 将sentinel配置为systemd服务并启动
[Unit] Description=Redis Sentinel provides high availability for Redis. Documentation=https://redis.io/topics/sentinel After=syslog.target After=network.target After=redis.target [Service] Type=fork User=redis Group=dba Restart=always # Disable OOM kill on the Redis OOMScoreAdjust=-1000 Environment=PG_OOM_ADJUST_FILE=/proc/self/oom_score_adj Environment=PG_OOM_ADJUST_VALUE=0 ExecStart=/usr/local/redis/src/redis-sentinel /usr/local/redis/conf/sentinel.conf ExecStop=/bin/kill -15 $MAINPID LimitNOFILE=65536 [Install] WantedBy=multi-user.target
# systemctl daemon-reload
# systemctl start sentinel
# systemctl status sentinel
● sentinel.service - Redis Sentinel provides high availability for Redis.
Loaded: loaded (/usr/lib/systemd/system/sentinel.service; disabled; vendor preset: disabled)
Active: active (running) since Sat 2020-03-28 23:43:02 CST; 4s ago
Docs: https://redis.io/topics/sentinel
Main PID: 14942 (redis-sentinel)
CGroup: /system.slice/sentinel.service
└─14942 /usr/local/redis/src/redis-sentinel 0.0.0.0:26379 [sentinel]
Mar 28 23:43:02 sht-sgmhadoopdn-03 systemd[1]: Starting Redis Sentinel provides high availability for Redis....
Mar 28 23:43:02 sht-sgmhadoopdn-03 systemd[1]: Started Redis Sentinel provides high availability for Redis..
3. 在各节点查看sentinel运行情况
$ redis-cli -h 172.16.101.58 -p 26379 info sentinel # Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 master0:name=mymaster,status=ok,address=172.16.101.58:6379,slaves=2,sentinels=1
$ redis-cli -h 172.16.101.59 -p 26379 info sentinel # Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 master0:name=mymaster,status=ok,address=172.16.101.58:6379,slaves=2,sentinels=1
$ redis-cli -h 172.16.101.60 -p 26379 info sentinel # Sentinel sentinel_masters:1 sentinel_tilt:0 sentinel_running_scripts:0 sentinel_scripts_queue_length:0 sentinel_simulate_failure_flags:0 master0:name=mymaster,status=ok,address=172.16.101.58:6379,slaves=2,sentinels=1