• rancher server偶发504问题排查


    问题

    rancher/server 部署到gce莫名其妙地出现504,导致访问异常,看containers logs访问互联网都是失败的

    #docker logs
    [root@rancher 037c3949be815f9311700d771678a3d7c875627287a62fd3f029b6ee38857d93]# journalctl -u docker -f 
    -- Logs begin at Tue 2020-10-13 12:24:20 UTC. --
    Nov 13 10:38:05 rancher dockerd[12162]: time="2020-11-13T10:38:05.335827262Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.17.0.0/16. Daemon option --bip can be used to set a preferred IP address"
    Nov 13 10:38:06 rancher dockerd[12162]: time="2020-11-13T10:38:06.158533175Z" level=info msg="Loading containers: done."
    Nov 13 10:38:06 rancher dockerd[12162]: time="2020-11-13T10:38:06.244847016Z" level=info msg="Docker daemon" commit=4484c46d9d graphdriver(s)=overlay2 version=19.03.13
    Nov 13 10:38:06 rancher dockerd[12162]: time="2020-11-13T10:38:06.244930649Z" level=info msg="Daemon has completed initialization"
    Nov 13 10:38:06 rancher systemd[1]: Started Docker Application Container Engine.
    Nov 13 10:38:06 rancher dockerd[12162]: time="2020-11-13T10:38:06.297384575Z" level=info msg="API listen on /var/run/docker.sock"
    Nov 13 10:39:18 rancher dockerd[12162]: time="2020-11-13T10:39:18.271599362Z" level=info msg="Container 632ebda68cfbb44909669307112bb6e5e158ecaeb5677667be18320e948bf290 failed to exit within 10 seconds of signal 15 - using the force"
    Nov 13 10:39:18 rancher dockerd[12162]: time="2020-11-13T10:39:18.544146121Z" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
    Nov 19 03:54:02 rancher dockerd[12162]: time="2020-11-19T03:54:02.985747420Z" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
    Nov 19 03:54:28 rancher dockerd[12162]: time="2020-11-19T03:54:28.748323212Z" level=warning msg="IPv4 forwarding is disabled. Networking will not work."
    ^C
    [root@rancher 037c3949be815f9311700d771678a3d7c875627287a62fd3f029b6ee38857d93]# ^C
    [root@rancher 037c3949be815f9311700d771678a3d7c875627287a62fd3f029b6ee38857d93]# cat /proc/sys/net/ipv4/ip_forward
    0
    

    解决

    cat > /etc/sysctl.d/docker.conf <<-'EOF'
    net.bridge.bridge-nf-call-iptables = 1
    net.bridge.bridge-nf-call-ip6tables = 1
    net.ipv4.ip_forward = 1
    EOF
    
    sysctl  --system
    

    开启网络转发

  • 相关阅读:
    浅析BIO、NIO、AIO
    equals()和hashCode()区别
    mysql分区
    java反射
    设计模式
    两个线程一个生产者个一个消费者
    Redis事务
    常用面试题
    springboot整合redis(注解形式)
    ElasticSearch6更新与重大变化
  • 原文地址:https://www.cnblogs.com/xiaochina/p/14015633.html
Copyright © 2020-2023  润新知