100并发 压测 后台web管理系统 查询用户反馈记录,报错Action.c(28): Error -27796: Failed to connect to server "xxxx": [10060] Connection timed out,错误率超过0.1%
排查:/var/log路径 命令 tailf messages,看到网卡丢包的问题
日志:
ar 4 10:38:29 localhost kernel: __ratelimit: 38 callbacks suppressed
Mar 4 10:38:29 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:29 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:29 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:29 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:29 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:29 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:29 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:29 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:29 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:29 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:34 localhost kernel: __ratelimit: 193 callbacks suppressed
Mar 4 10:38:34 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:34 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:35 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:35 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:35 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:35 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:35 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:36 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:36 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:36 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:40 localhost kernel: __ratelimit: 93 callbacks suppressed
Mar 4 10:38:40 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:41 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:41 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:42 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:42 localhost kernel: nf_conntrack: table full, dropping packet.
Mar 4 10:38:42 localhost kernel: nf_conntrack: table full, dropping packet.
原因
服务器访问量大,内核netfilter模块conntrack相关参数配置不合理,导致新连接被drop掉。
解决:
关闭linux服务器防火墙
命令:
1. 永久性生效
开启:chkconfig iptables on
关闭:chkconfig iptables off
2. 即时生效,重启后失效
开启:service iptables start
关闭:service iptables stop