• k8s网络之calico学习


    一、知识准备

    1.calico主要通过ipip协议与bgp协议来实现通信。前者通过ipip隧道作为通信基础,后者则是纯三层的路由交换
    2.bgp协议主要由两种方式:BGP Speaker 全互联模式(node-to-node mesh)与BGP Speaker RR模式
    3.本文主要探索一下calico bgp的两种模式


    二、环境准备

    组件 版本
    OS Ubuntu 18.04.1 LTS
    docker 18.06.0-ce
    k8s 1.10.1
    calico 3.1.3

    ip hostname
    192.168.56.101 k8s-master
    192.168.56.102 k8s-node1
    192.168.56.103 k8s-node2
    192.168.56.104 k8s-node3

    三、安装

    k8s安装

    参考官网安装以及社区诸多大神的安装帖子,这里就不班门弄斧了

    本文的k8s的环境打开了rbac,etcd加入了证书

    calico安装

    主要参考官方文档 https://docs.projectcalico.org/v3.1/getting-started/kubernetes/installation/calico

    1.calico rbac

    kubectl apply -f 
    https://docs.projectcalico.org/v3.1/getting-started/kubernetes/installation/rbac.yaml
    
    

    2.下载calico.yaml

    curl 
    https://docs.projectcalico.org/v3.1/getting-started/kubernetes/installation/hosted/calico.yaml 
    -O
    
    

    3.填入etcd地址

    ETCD_ENDPOINTS="https://192.168.56.101:2379"
    sed -i "s#.*etcd_endpoints:.*#  etcd_endpoints: "${ETCD_ENDPOINTS}"#g" calico.yaml
    
    sed -i "s#__ETCD_ENDPOINTS__#${ETCD_ENDPOINTS}#g" calico.yaml
    
    

    4.将etcd证书信息填入。我的etcd证书在/etc/etcd/ssl下

    ETCD_CERT=`cat /etc/etcd/ssl/etcd.pem | base64 | tr -d '
    '`
    ETCD_KEY=`cat /etc/etcd/ssl/etcd-key.pem | base64 | tr -d '
    '`
    ETCD_CA=`cat /etc/etcd/ssl/etcd-root-ca.pem | base64 | tr -d '
    '`
    
    sed -i "s#.*etcd-cert:.*#  etcd-cert: ${ETCD_CERT}#g" calico.yaml
    sed -i "s#.*etcd-key:.*#  etcd-key: ${ETCD_KEY}#g" calico.yaml
    sed -i "s#.*etcd-ca:.*#  etcd-ca: ${ETCD_CA}#g" calico.yaml
    
    sed -i 's#.*etcd_ca:.*#  etcd_ca: "/calico-secrets/etcd-ca"#g' calico.yaml
    sed -i 's#.*etcd_cert:.*#  etcd_cert: "/calico-secrets/etcd-cert"#g' calico.yaml
    sed -i 's#.*etcd_key:.*#  etcd_key: "/calico-secrets/etcd-key"#g' calico.yaml
    
    sed -i "s#__ETCD_KEY_FILE__#/etc/etcd/ssl/etcd-key.pem#g" calico.yaml
    sed -i "s#__ETCD_CERT_FILE__#/etc/etcd/ssl/etcd.pem#g" calico.yaml
    sed -i "s#__ETCD_CA_CERT_FILE__#/etc/etcd/ssl/etcd-root-ca.pem#g" calico.yaml
    sed -i "s#__KUBECONFIG_FILEPATH__#/etc/cni/net.d/calico-kubeconfig#g" calico.yaml
    
    

    5.配置calico bgp 并且修改ip cidr:10.10.0.0/16

    sed -i '/CALICO_IPV4POOL_IPIP/{n;s/Always/off/g}' calico.yaml
    sed -i '/CALICO_IPV4POOL_CIDR/{n;s/192.168.0.0/10.10.0.0/g}' calico.yaml
    
    

    6.kubectl安装calico

    kubectl apply -f calico.yaml
    

    注意:因为calico-node需要获取操作系统的权限运行,所以要在apiserver、kubelet中加入--allow-privileged=true

    查看一下状态:

    root@k8s-master:/tmp# kubectl get pods -n kube-system -owide
    NAME                                        READY     STATUS    RESTARTS   AGE       IP               NODE
    calico-kube-controllers-98989846-b4n72      1/1       Running   0          18d       192.168.56.102   k8s-node1
    calico-node-58pck                           2/2       Running   0          18d       192.168.56.103   k8s-node2
    calico-node-s2txw                           2/2       Running   0          18d       192.168.56.101   k8s-master
    calico-node-svmbp                           2/2       Running   0          18d       192.168.56.102   k8s-node1
    ...
    

    7.kubelet配置calico

    找到kubelet的配置文件(我的环境在/etc/kubernetes/kubelet),加入
    --network-plugin=cni
    
    重启kubelet
    

    8.测试一个pod

    cat << EOF | kubectl create -f -
    apiVersion: v1
    kind: Pod
    metadata:
      name: network-test
      namespace: test
    spec:
      containers:
      - image: busybox:latest
        command:
          - sleep
          - "3600"
        name: network-test
    EOF
    
    root@k8s-master:~# kubectl -n test get pods -owide
    NAME           READY     STATUS    RESTARTS   AGE       IP              NODE
    network-test   1/1       Running   0          41s       10.10.169.139   k8s-node2
    

    至此:calico安装已经完成


    四、calicoctl使用

    1.下载calicoctl

    https://github.com/projectcalico/calicoctl/releases/download/v3.1.3/calicoctl-linux-amd64

    2.查看当前的calico-node

    root@k8s-master:/tmp# calicoctl get node
    NAME
    k8s-master
    k8s-node1
    k8s-node2
    
    calicoctl get node -o yaml 查看详细信息
    

    3.查看当前的ippool

    root@k8s-master:/tmp# calicoctl get ippool
    NAME                  CIDR
    default-ipv4-ippool   10.10.0.0/16
    default-ipv6-ippool   fdc6:1a69:2b39::/48
    

    4.查看当前模式

    root@k8s-master:/tmp# calicoctl node status
    Calico process is running.
    
    IPv4 BGP status
    +----------------+-------------------+-------+----------+-------------+
    |  PEER ADDRESS  |     PEER TYPE     | STATE |  SINCE   |    INFO     |
    +----------------+-------------------+-------+----------+-------------+
    | 192.168.56.102 | node-to-node mesh | up    | 07:39:02 | Established |
    | 192.168.56.103 | node-to-node mesh | up    | 07:39:02 | Established |
    +----------------+-------------------+-------+----------+-------------+
    
    IPv6 BGP status
    No IPv6 peers found.
    
    root@k8s-master:/tmp# netstat -anp | grep ESTABLISH | grep bird
    tcp        0      0 192.168.56.101:33029    192.168.56.102:179      ESTABLISHED 26558/bird
    tcp        0      0 192.168.56.101:58055    192.168.56.103:179      ESTABLISHED 26558/bird
    

    当前运行在BGP Speaker 全互联模式(node-to-node mesh)模式,calico集群中的节点之间都会相互建立连接,用于路由交换。适合规模不大的集群中运行,一旦集群节点增大,mesh模式将形成一个巨大服务网格,连接数暴增

    5.修改BGP Speaker RR模式

    禁止mesh模式,配置bgpPeer

    cat << EOF | calicoctl create -f -
    apiVersion: projectcalico.org/v3
    kind: BGPConfiguration
    metadata:
      name: default
    spec:
      logSeverityScreen: Info
      nodeToNodeMeshEnabled: false
      asNumber: 61234
    EOF
    
    cat << EOF | calicoctl create -f -
    apiVersion: projectcalico.org/v3
    kind: BGPPeer
    metadata:
      name: bgppeer-global
    spec:
      peerIP: 192.168.56.103
      asNumber: 61234
    EOF
    

    查看RR模式配置:

    root@k8s-master:~# calicoctl get bgpconfig
    NAME      LOGSEVERITY   MESHENABLED   ASNUMBER
    default   Info          false         61234
    
    root@k8s-master:~# calicoctl get bgppeer
    NAME             PEERIP           NODE       ASN
    bgppeer-global   192.168.56.103   (global)   61234
    

    安装routereflector

    docker run --privileged --net=host -d                             
               --name=calico-rr                                       
               -e IP=192.168.56.104                                   
               -e ETCD_ENDPOINTS=https://192.168.56.101:2379          
               -v /etc/etcd/ssl:/etc/calico/ssl                     
               -e ETCD_CA_CERT_FILE=/etc/calico/ssl/etcd-root-ca.pem  
               -e ETCD_CERT_FILE=/etc/calico/ssl/etcd.pem             
               -e ETCD_KEY_FILE=/etc/calico/ssl/etcd-key.pem          
               calico/routereflector:v0.6.1
    

    查看效果:

    root@k8s-master:~# calicoctl node status
    Calico process is running.
    
    IPv4 BGP status
    +----------------+-----------+-------+----------+-------------+
    |  PEER ADDRESS  | PEER TYPE | STATE |  SINCE   |    INFO     |
    +----------------+-----------+-------+----------+-------------+
    | 192.168.56.103 | global    | up    | 09:13:23 | Established |
    +----------------+-----------+-------+----------+-------------+
    
    IPv6 BGP status
    No IPv6 peers found.
    
    root@k8s-master:~# netstat -anp | grep ESTABLISH | grep bird
    tcp        0      0 192.168.56.101:179      192.168.56.103:54903    ESTABLISHED 26558/bird
    

    每台机器都只会与rr建立一条连接,并且与rr通信即可拿到所有路由,大大减少了连接数量



    至此,本文结束
    在下才疏学浅,有撒汤漏水的,请各位不吝赐教...

  • 相关阅读:
    curl 文件上传
    地区选择三级联动
    水平居中
    z-index
    css垂直居中
    Enrolment API
    阿里云OSS存储
    java中的vo、dto 、dao
    Spring mvc 中使用 kaptcha 验证码
    将下载的本地的jar手动添加到maven仓库
  • 原文地址:https://www.cnblogs.com/MrVolleyball/p/9920964.html
Copyright © 2020-2023  润新知