今天在用ansilbe命令 ansible noredis -m shell -a "netstat -nltp|grep redis"查看集群redis的启动状态发现:
其中有一台是启动不了的,非常奇怪都是一样的配置。
于是我ssh 10.25.115.125过去查看了配置,还有命令,发现一切正常! netstat -nltp|grep 6379也没发现端口已经被占用,那问题出在哪呢?
于是,我在/etc/redis.conf配置了 logfile "/var/log/redis.log" (汗,发现配置log的重要性了,默认redis配置是不开log的):结果找到问题了
[16560] 14 Sep 17:32:37.724 # Server started, Redis version 2.8.21
[16560] 14 Sep 17:32:37.724 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
[16560] 14 Sep 17:32:37.724 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
[16560] 14 Sep 17:32:37.724 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
[16560] 14 Sep 17:32:37.724 # Short read or OOM loading DB. Unrecoverable error, aborting now.
原来问题出现内存不足。