Centos7二进制部署k8s-v1.20.2 ipvs版本(kube-proxy、calico)

一、部署kube-proxy

获取最新更新以及文章用到的软件包，请移步点击：查看更新

1、创建csr请求文件

cat > kube-proxy-csr.json << EOF 
{
  "CN": "system:kube-proxy",
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Sichuan",
      "L": "Chengdu",
      "O": "k8s",
      "OU": "system"
    }
  ]
}
EOF

生成证书
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy

2、创建kubeconfig文件

kubectl config set-cluster kubernetes --certificate-authority=ca.pem --embed-certs=true --server=https://192.168.112.130:7443 --kubeconfig=kube-proxy.kubeconfig

kubectl config set-credentials kube-proxy --client-certificate=kube-proxy.pem --client-key=kube-proxy-key.pem --embed-certs=true --kubeconfig=kube-proxy.kubeconfig

kubectl config set-context default --cluster=kubernetes --user=kube-proxy --kubeconfig=kube-proxy.kubeconfig

kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig

3、创建kube-proxy配置文件

cat > /opt/kubernetes/cfg/kube-proxy.yaml << EOF 
apiVersion: kubeproxy.config.k8s.io/v1alpha1
bindAddress: 192.168.112.130
clientConnection:
  kubeconfig: /opt/kubernetes/cfg/kube-proxy.kubeconfig
clusterCIDR: 192.168.0.0/16                           #此处网段必须与网络组件网段保持一致，否则部署网络组件时会报错
healthzBindAddress: 192.168.112.130:10256
kind: KubeProxyConfiguration
metricsBindAddress: 192.168.112.130:10249
mode: "ipvs"
EOF 

注：配置文件kube-proxy.yaml中address修改为各节点的实际IP

4、创建启动文件

cat > /usr/lib/systemd/system/kube-proxy.service << EOF 
[Unit]
Description=Kubernetes Kube-Proxy Server
Documentation=https://github.com/kubernetes/kubernetes
After=network.target

[Service]
WorkingDirectory=/opt/kubernetes/kube-proxy
ExecStart=/opt/kubernetes/bin/kube-proxy 
  --config=/opt/kubernetes/cfg/kube-proxy.yaml 
  --alsologtostderr=true 
  --logtostderr=false 
  --log-dir=/opt/kubernetes/logs 
  --v=2
Restart=on-failure
RestartSec=5
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target
EOF

5、同步文件到各个节点

cp kube-proxy*.pem /opt/kubernetes/ssl/
cp kube-proxy.kubeconfig /opt/kubernetes/cfg
scp /usr/lib/systemd/system/kube-proxy.service root@192.168.112.132:/usr/lib/systemd/system/

cd /root/TLS/k8s/kubernetes/server/bin
cp kube-proxy /opt/kubernetes/bin/
scp kube-proxy root@192.168.112.132:/opt/kubernetes/bin/

6、启动服务

mkdir -p /opt/kubernetes/kube-proxy
systemctl daemon-reload
systemctl enable kube-proxy
systemctl restart kube-proxy
systemctl status kube-proxy

二、安装网络插件

1、配置网络组件

#下载yaml文件
wget https://docs.projectcalico.org/v3.14/manifests/calico.yaml

#搜索这行关键字
- name: IP
  value: "autodetect"
在它之前添加两行　　
# Auto-detect the BGP IP address.
- name: IP_AUTODETECTION_METHOD         
  value: "interface=eth0"                           #你得具体网卡名字
         
#创建网络插件容器
kubectl apply -f calico.yaml

2、查看各个节点，均为Ready状态

kubectl get pods -A -o wide
kubectl get nodes

三、部署coredns

1、部署coredns

#下载yaml文件
wget https://raw.githubusercontent.com/coredns/deployment/master/kubernetes/coredns.yaml.sed

mv coredns.yaml.sed coredns.yaml

2、修改文件里面内容

修改yaml文件
kubernetes CLUSTER_DOMAIN REVERSE_CIDRS
改为
kubernetes cluster.local  in-addr.arpa ip6.arpa

forward . UPSTREAMNAMESERVER
改为
forward . /etc/resolv.conf

clusterIP: CLUSTER_DNS_IP 
改为
clusterIP: 10.255.0.2

去掉STUBDOMAINS

3、创建容器

kubectl apply -f coredns.yaml 

4、DNS解析测试

kubectl run -it --rm dns-test --image=busybox:1.28.4 sh

If you don't see a command prompt, try pressing enter. 
 
/ # nslookup kubernetes 
Server:    10.0.0.2 
Address 1: 10.0.0.2 kube-dns.kube-system.svc.cluster.local 
 
Name:      kubernetes 
Address 1: 10.0.0.1 kubernetes.default.svc.cluster.local

5、如果报以下错误，请安装服务，之后重启kube-proxy

systemctl status kube-proxy
● kube-proxy.service - Kubernetes Kube-Proxy Server
   Loaded: loaded (/usr/lib/systemd/system/kube-proxy.service; enabled; vendor preset: disabled)
   Active: active (running) since 一 2021-08-09 17:51:04 CST; 5min ago
     Docs: https://github.com/kubernetes/kubernetes
 Main PID: 903 (kube-proxy)
    Tasks: 8
   Memory: 44.3M
   CGroup: /system.slice/kube-proxy.service
           └─903 /opt/kubernetes/bin/kube-proxy --config=/opt/kubernetes/cfg/kube-proxy.yaml --alsologtostderr=true --logtostderr=false --log-dir=/opt/kubernetes/logs --v=2

8月 09 17:51:50 k8s-master kube-proxy[903]: I0809 17:51:50.478753     903 service.go:275] Service default/kubernetes updated: 1 ports
8月 09 17:51:50 k8s-master kube-proxy[903]: I0809 17:51:50.478802     903 service.go:275] Service kube-system/kube-dns updated: 3 ports
8月 09 17:51:50 k8s-master kube-proxy[903]: I0809 17:51:50.555611     903 shared_informer.go:247] Caches are synced for service config
8月 09 17:51:50 k8s-master kube-proxy[903]: I0809 17:51:50.555962     903 service.go:390] Adding new service port "kube-system/kube-dns:dns" at 10.255.0.2:53/UDP
8月 09 17:51:50 k8s-master kube-proxy[903]: I0809 17:51:50.555993     903 service.go:390] Adding new service port "kube-system/kube-dns:dns-tcp" at 10.255.0.2:53/TCP
8月 09 17:51:50 k8s-master kube-proxy[903]: I0809 17:51:50.556009     903 service.go:390] Adding new service port "kube-system/kube-dns:metrics" at 10.255.0.2:9153/TCP
8月 09 17:51:50 k8s-master kube-proxy[903]: I0809 17:51:50.556024     903 service.go:390] Adding new service port "default/kubernetes:https" at 10.255.0.1:443/TCP
8月 09 17:51:50 k8s-master kube-proxy[903]: I0809 17:51:50.556134     903 proxier.go:1067] Stale udp service kube-system/kube-dns:dns -> 10.255.0.2
8月 09 17:51:50 k8s-master kube-proxy[903]: E0809 17:51:50.677871     903 proxier.go:1673] Failed to delete stale service IP 10.255.0.2 connections, error: error deleting connection tracking state for UDP servic...t found in $PATH
8月 09 17:52:19 k8s-master kube-proxy[903]: E0809 17:52:19.609789     903 proxier.go:1952] Failed to delete kube-system/kube-dns:dns endpoint connections, error: error deleting conntrack entries for udp peer {10...t found in $PATH
Hint: Some lines were ellipsized, use -l to show in full.

执行下面命令：
yum -y install conntrack-tools

四、安装k8s插件

1、配置kubectl子命令补全

[root@master1 work]# yum install -y bash-completion
[root@master1 work]# source /usr/share/bash-completion/bash_completion
[root@master1 work]# source <(kubectl completion bash)
[root@master1 work]# kubectl completion bash > ~/.kube/completion.bash.inc
[root@master1 work]# source '/root/.kube/completion.bash.inc'  
[root@master1 work]# source $HOME/.bash_profile

2、配置切换命名空间

mv kubens /usr/local/bin/

3、配置命令行颜色化

mv kubecolor/usr/local/bin/
vim  /etc/profile
添加
source <(kubectl completion bash)
command -v kubecolor >/dev/null 2>&1 && alias kubectl="kubecolor"