kube-proxy ipvs calico

1. 关闭防火墙

#/etc/selinux/config

SELINUX=disabled

#sestatus

SELinux status:                 disabled

#systemctl stop firewalld
#systemctl disable firewalld

　　

2. 修改系统参数

# /etc/sysctl.conf

net.ipv4.ip_forward=1
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1

# sysctl -p

　　

3. 安装Ipvsadm

yum -y install ipvsadm ipset

# lsmod | grep ip_vs

ip_vs_sh               12688  0
ip_vs_wrr              12697  0
ip_vs_rr               12600  27
ip_vs                 141092  33 ip_vs_rr,ip_vs_sh,ip_vs_wrr
nf_conntrack          133387  9 ip_vs,nf_nat,nf_nat_ipv4,nf_nat_ipv6,xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_netlink,nf_conntrack_ipv4,nf_conntrack_ipv6
libcrc32c              12644  4 xfs,ip_vs,nf_nat,nf_conntrack

如果模块没有加载

cat > /etc/sysconfig/modules/ipvs.modules <<EOF

#!/bin/bash

modprobe -- ip_vs

modprobe -- ip_vs_rr

modprobe -- ip_vs_wrr

modprobe -- ip_vs_sh

modprobe -- nf_conntrack_ipv4

EOF

# chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules && lsmod | grep -e ip_vs -e nf_conntrack_ipv4

上面脚本创建了的/etc/sysconfig/modules/ipvs.modules文件，保证在节点重启后能自动加载所需模块。 使用lsmod | grep -e ip_vs -e nf_conntrack_ipv4命令查看是否已经正确加载所需的内核模块。

4. 修改kubelet配置

KUBE_PROXY_ARGS="--bind-address=10.10.1.8 
                 --hostname-override=docker4.node 
                 --masquerade-all 
                 --feature-gates=SupportIPVSProxyMode=true 
                 --proxy-mode=ipvs 
                 --ipvs-min-sync-period=5s 
                 --ipvs-sync-period=5s 
                 --ipvs-scheduler=rr 
                 --kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig 
                 --cluster-cidr=10.254.0.0/16"

　

5. calico 配置 port range (没有指定的可以跳过)

  calico.yaml
          
            # The default IPv4 pool to create on startup if none exists. Pod IPs will be
            # chosen from this range. Changing this value after installation will have
            # no effect. This should fall within `--cluster-cidr`.
            - name: CALICO_IPV4POOL_CIDR
              value: "172.18.0.0/16"
            # Disable file logging so `kubectl logs` works.
            - name: CALICO_DISABLE_FILE_LOGGING
              value: "true"
            # Set Felix endpoint to host default action to ACCEPT.
            - name: FELIX_DEFAULTENDPOINTTOHOSTACTION
              value: "ACCEPT"
            # Disable IPv6 on Kubernetes.
            - name: FELIX_IPV6SUPPORT
              value: "false"
            # Set Felix logging to "info"
            - name: FELIX_LOGSEVERITYSCREEN
              value: "info"
            - name: FELIX_HEALTHENABLED
              value: "true"
            # Node port range while kube-proxy in ipvs mode
            - name: FELIX_KUBENODEPORTRANGES
              value: "10000:11000"

　　

　注意事项 ：

1.calico会自动选择网卡，这里要看calico-node的日志确定是否是需要的网卡。

2.calico的 CIDR不要与clusterip和hostip重叠，否则网络不通。

参考资料

https://docs.projectcalico.org/networking/use-ipvs

https://docs.projectcalico.org/getting-started/kubernetes/self-managed-onprem/onpremises

https://blog.51cto.com/1000682/2362853

https://www.kubernetes.org.cn/3025.html