Kubernetes基本操作篇

Kubernetes基本操作篇

k8s 是经典的一对多模型，有一个主要的管理节点master和许多的工作节点node。当然，k8s 也可以配置多个管理节点，拥有两个以上的管理节点被称为 高可用。k8s 包括了许多的组件，每个组件都是单运行在一个docker容器中，然后通过自己规划的虚拟网络相互访问。

集群基本操作

• Cluster 集群，一个集群里有一个 Master 和数个 Node
• Node 通常拿一台物理机座位图一个 Node，也可以用虚拟机
• Pod 是一个 docker 实例，一个 Node 里有一个或多个 Pod
• Deployment 一个发布，可以包含一个或多个 Pod
• Service 暴露出来的服务，内置了负载分发

• 查看集群节点

  [root@kubernetes-master-01 ~]# kubectl get nodes
  NAME                   STATUS   ROLES    AGE     VERSION
  kubernetes-master-01   Ready    master   3m44s   v1.19.0
  kubernetes-node-01     Ready    <none>   2m21s   v1.19.0
  kubernetes-node-02     Ready    <none>   2m15s   v1.19.0Copy to clipboardErrorCopied
  

• 查看系统组件

  [root@kubernetes-master-01 ~]# kubectl get pods -n kube-system
  NAME                                           READY   STATUS    RESTARTS   AGE
  coredns-7df6d7c9f9-w5b9r                       1/1     Running   0          3m10s
  coredns-7df6d7c9f9-zxlx4                       1/1     Running   0          3m10s
  etcd-kubernetes-master-01                      1/1     Running   0          3m20s
  kube-apiserver-kubernetes-master-01            1/1     Running   0          3m20s
  kube-controller-manager-kubernetes-master-01   1/1     Running   0          3m20s
  kube-flannel-ds-amd64-9vqc2                    1/1     Running   0          2m7s
  kube-flannel-ds-amd64-d5v74                    1/1     Running   0          2m1s
  kube-flannel-ds-amd64-f424g                    1/1     Running   0          2m45s
  kube-proxy-5g5vs                               1/1     Running   0          3m10s
  kube-proxy-h67k7                               1/1     Running   0          2m1s
  kube-proxy-thb9g                               1/1     Running   0          2m7s
  kube-scheduler-kubernetes-master-01            1/1     Running   0          3m20sCopy to clipboardErrorCopied
  

• 加入Master节点

  kubeadm join 172.16.0.55:8443 --token abcdef.0123456789abcdef     --discovery-token-ca-cert-hash sha256:777cdc282d7988dccb109dc95d781bc2f6aa3fb145b6adc54b8499d49c12ebb9     --control-plane --certificate-key 4f1c28f81b1ba8a7518002f47aa9fb99c233092a7cc9d468d7cfd436bdb44c0fCopy to clipboardErrorCopied
  

• 加入Node节点

   kubeadm join 172.16.0.55:8443 --token abcdef.0123456789abcdef     --discovery-token-ca-cert-hash sha256:777cdc282d7988dccb109dc95d781bc2f6aa3fb145b6adc54b8499d49c12ebb9Copy to clipboardErrorCopied
  

k8s最小单元Pod

Pod是Kubernetes创建或部署的最小/最简单的基本单位，一个Pod代表集群上正在运行的一个进程。

一个Pod封装一个应用容器（也可以有多个容器），存储资源、一个独立的网络IP以及管理控制容器运行方式的策略选项。Pod代表部署的一个单位：Kubernetes中单个应用的实例，它可能由单个容器或多个容器共享组成的资源。

POD 基本概念

每个Pod都是运行应用的单个实例，如果需要水平扩展应用（例如，运行多个实例），则应该使用多个Pods，每个实例一个Pod。

Pod中的容器在集群中Node上被自动分配，容器之间可以共享资源、网络和相互依赖关系，并同时被调度使用。

• 单容器Pod，最常见的应用方式。
• Pod中不同容器之间的端口是不能冲突的
• 多容器Pod，对于多容器Pod，Kubernetes会保证所有的容器都在同一台物理主机或虚拟主机中运行。多容器Pod是相对高阶的使用方式，除非应用耦合特别严重，一般不推荐使用这种方式。一个Pod内的容器共享IP地址和端口范围，容器之间可以通过 localhost 互相访问。

Pod并不提供保证正常运行的能力，因为可能遭受Node节点的物理故障、网络分区等等的影响，整体的高可用是Kubernetes集群通过在集群内调度Node来实现的。通常情况下我们不要直接创建Pod，一般都是通过Controller来进行管理，但是了解Pod对于我们熟悉控制器非常有好处。

网络

每个Pod被分配一个独立的IP地址，Pod中的每个容器共享网络命名空间，包括IP地址和网络端口。Pod内的容器可以使用localhost相互通信。当Pod中的容器与Pod 外部通信时，他们必须协调如何使用共享网络资源（如端口）。

存储

Pod可以指定一组共享存储volumes。Pod中的所有容器都可以访问共享volumes，允许这些容器共享数据。volumes 还用于Pod中的数据持久化，以防其中一个容器需要重新启动而丢失数据。

Pod带来的好处

• Pod做为一个可以独立运行的服务单元，简化了应用部署的难度，以更高的抽象层次为应用部署管提供了极大的方便。
• Pod做为最小的应用实例可以独立运行，因此可以方便的进行部署、水平扩展和收缩、方便进行调度管理与资源的分配。
• Pod中的容器共享相同的数据和网络地址空间，Pod之间也进行了统一的资源管理与分配。

POD基本操作

• 创建pod

  [root@kubernetes-master-01 ~]# kubectl run django --image=alvinos/django:v1
  pod/django createdCopy to clipboardErrorCopied
  

• 查看pod

  [root@kubernetes-master-01 ~]# kubectl get pods
  NAME     READY   STATUS    RESTARTS   AGE
  django   1/1     Running   0          2m9sCopy to clipboardErrorCopied
  

• 测试pod

  [root@kubernetes-master-01 ~]# curl 10.244.2.2/index
  主机名：django，版本：v1Copy to clipboardErrorCopied
  

• 查看pod IP

  [root@kubernetes-master-01 ~]# kubectl get pods -o wide
  NAME     READY   STATUS    RESTARTS   AGE     IP           NODE                 NOMINATED NODE   READINESS GATES
  django   1/1     Running   0          3m19s   10.244.2.2   kubernetes-node-02   <none>           <none>Copy to clipboardErrorCopied
  

通过配置文件创建POD

Pod的配置信息中有几个重要部分，apiVersion、kind、metadata(元数据)、spec以及status。其中apiVersion和kind是比较固定的，status是运行时的状态，所以最重要的就是metadata和spec两个部分。

• Pod经典案例

  apiVersion: v1
  kind: Pod
  metadata:
    name: first-pod
    labels:
      app: bash
      tir: backend
  spec:
    containers:
      - name: bash-container
        image: busybox
        command: ['sh', '-c', 'echo Hello Kubernetes! && sleep 10']Copy to clipboardErrorCopied
  

• 操作结果

  [root@kubernetes-master-01 ~]# vim pod.yaml
  [root@kubernetes-master-01 ~]# kubectl apply -f pod.yaml
  pod/first-pod created
  [root@kubernetes-master-01 ~]# kubectl get pods
  NAME        READY   STATUS    RESTARTS   AGE
  django      1/1     Running   0          21m
  first-pod   1/1     Running   0          32s
  [root@kubernetes-master-01 ~]# kubectl logs first-pod
  Hello Kubernetes!Copy to clipboardErrorCopied
  

• 查看Pod运行信息

  [root@kubernetes-master-01 ~]# kubectl describe pod first-pod
  Name:         first-pod
  Namespace:    default
  Priority:     0
  Node:         kubernetes-node-01/172.16.0.53
  Start Time:   Sat, 05 Sep 2020 17:39:17 +0800
  Labels:       app=bash
                tir=backend
  Annotations:  <none>
  Status:       Running
  IP:           10.244.1.2
  IPs:
    IP:  10.244.1.2
  Containers:
    bash-container:
      Container ID:  docker://de1374c6bb241df68a8e983cdf3acf4093d354da1fe8e6a819e886a27d43edaa
      Image:         busybox
      Image ID:      docker-pullable://busybox@sha256:c3dbcbbf6261c620d133312aee9e858b45e1b686efbcead7b34d9aae58a37378
      Port:          <none>
      Host Port:     <none>
      Command:
        sh
        -c
        echo Hello Kubernetes! && sleep 10
      State:          Running
        Started:      Sat, 05 Sep 2020 17:40:59 +0800
      Last State:     Terminated
        Reason:       Completed
        Exit Code:    0
        Started:      Sat, 05 Sep 2020 17:40:11 +0800
        Finished:     Sat, 05 Sep 2020 17:40:21 +0800
      Ready:          True
      Restart Count:  3
      Environment:    <none>
      Mounts:
        /var/run/secrets/kubernetes.io/serviceaccount from default-token-ldlxt (ro)
  Conditions:
    Type              Status
    Initialized       True
    Ready             True
    ContainersReady   True
    PodScheduled      True
  Volumes:
    default-token-ldlxt:
      Type:        Secret (a volume populated by a Secret)
      SecretName:  default-token-ldlxt
      Optional:    false
  QoS Class:       BestEffort
  Node-Selectors:  <none>
  Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                   node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
  Events:
    Type     Reason     Age                 From                         Message
    ----     ------     ----                ----                         -------
    Normal   Scheduled  108s                default-scheduler            Successfully assigned default/first-pod to kubernetes-node-01
    Normal   Pulled     89s                 kubelet, kubernetes-node-01  Successfully pulled image "busybox" in 17.675119792s
    Normal   Pulled     77s                 kubelet, kubernetes-node-01  Successfully pulled image "busybox" in 1.107122896s
    Normal   Pulled     54s                 kubelet, kubernetes-node-01  Successfully pulled image "busybox" in 1.690204423s
    Warning  BackOff    33s (x3 over 66s)   kubelet, kubernetes-node-01  Back-off restarting failed container
    Normal   Pulling    22s (x4 over 106s)  kubelet, kubernetes-node-01  Pulling image "busybox"
    Normal   Pulled     7s                  kubelet, kubernetes-node-01  Successfully pulled image "busybox" in 15.376057413s
    Normal   Created    6s (x4 over 89s)    kubelet, kubernetes-node-01  Created container bash-container
    Normal   Started    6s (x4 over 89s)    kubelet, kubernetes-node-01  Started container bash-containerCopy to clipboardErrorCopied
  

• 查看Pod配置项信息

  [root@kubernetes-master-01 ~]# kubectl explain pod
  KIND:     Pod
  VERSION:  v1
  
  DESCRIPTION:
       Pod is a collection of containers that can run on a host. This resource is
       created by clients and scheduled onto hosts.
  
  FIELDS:
     apiVersion    <string>
       APIVersion defines the versioned schema of this representation of an
       object. Servers should convert recognized schemas to the latest internal
       value, and may reject unrecognized values. More info:
       https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
  
     kind    <string>
       Kind is a string value representing the REST resource this object
       represents. Servers may infer this from the endpoint the client submits
       requests to. Cannot be updated. In CamelCase. More info:
       https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
  
     metadata    <Object>
       Standard object's metadata. More info:
       https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata
  
     spec    <Object>
       Specification of the desired behavior of the pod. More info:
       https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
  
     status    <Object>
       Most recently observed status of the pod. This data may not be up to date.
       Populated by the system. Read-only. More info:
       https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
  Copy to clipboardErrorCopied
  

Pod配置标签

Label是Kubernetes系统中另外一个核心概念。一个Label是一个key=value的键值对，其中key与vaue由用户自己指定。Label可以附加到各种资源对象上，例如Node、Pod、Service、RC等，一个资源对象可以定义任意数量的Label，同一个Label也可以被添加到任意数量的资源对象上去，Label通常在资源对象定义时确定，也可以在对象创建后动态添加或者删除。

我们可以通过指定的资源对象捆绑一个或多个不同的Label来实现多维度的资源分组管理功能，以便于灵活、方便地进行资源分配、调度、配置、部署等管理工作。例如：部署不同版本的应用到不同的环境中；或者监控和分析应用（日志记录、监控、告警）等。一些常用等label示例如下。

• 版本标签："release" : "stable" , "release" : "canary"
• 环境标签："environment" : "dev" , "environment" : "production"
• 架构标签："tier" : "frontend" , "tier" : "backend" , "tier" : "middleware"
• 分区标签："partition" : "customerA" , "partition" : "customerB"
• 质量管控标签："track" : "daily" , "track" : "weekly"

Label相当于我们熟悉的“标签”，给某个资源对象定义一个Label，就相当于给它打了一个标签，随后可以通过Label Selector（标签选择器）查询和筛选拥有某些Label的资源对象，Kubernetes通过这种方式实现了类似SQL的简单又通用的对象查询机制。

• 查看标签

  [root@kubernetes-master-01 ~]# kubectl get pods --show-labels
  NAME        READY   STATUS             RESTARTS   AGE     LABELS
  django      1/1     Running            0          27m     run=djangoCopy to clipboardErrorCopied
  

• 新增标签

  [root@kubernetes-master-01 ~]# kubectl label pod django env=test
  pod/django labeled
  [root@kubernetes-master-01 ~]# kubectl get pods --show-labels
  NAME        READY   STATUS      RESTARTS   AGE     LABELS
  django      1/1     Running     0          28m     env=test,run=djangoCopy to clipboardErrorCopied
  

• 删除标签

  [root@kubernetes-master-01 ~]# kubectl label pod django env-
  pod/django labeled
  [root@kubernetes-master-01 ~]# kubectl get pods --show-labels
  NAME        READY   STATUS             RESTARTS   AGE     LABELS
  django      1/1     Running            0          29m     run=djangoCopy to clipboardErrorCopied
  

• 通过标签筛选Pod

  [root@kubernetes-master-01 ~]# kubectl get pods -l run=django --show-labels
  NAME     READY   STATUS    RESTARTS   AGE   LABELS
  django   1/1     Running   0          30m   run=djangoCopy to clipboardErrorCopied
  

• 通过标签删除Pod

  [root@kubernetes-master-01 ~]# kubectl delete pod -l run=django
  pod "django" deleted
  [root@kubernetes-master-01 ~]# kubectl get pods -l run=django
  No resources found in default namespace.Copy to clipboardErrorCopied
  

Replication Controller

Replication Controller控制器简称：rc。Replication Controller 确保在任何时候都有特定数量的 pod 副本处于正常运行状态。则Replication Controller会终止额外的pod，如果减少，RC会创建新的pod，始终保持在定义范围。

• 保证Pod数量
• 跨节点监控Pod数量

• 创建RC

  [root@kubernetes-master-01 ~]# vim rc.yaml
  [root@kubernetes-master-01 ~]# cat rc.yaml
  apiVersion: v1
  kind: ReplicationController
  metadata:
    name: nginx
  spec:
    replicas: 3
    selector:
      app: nginx
    template:
      metadata:
        name: nginx
        labels:
          app: nginx
      spec:
        containers:
        - name: nginx
          image: nginx
          ports:
          - containerPort: 80
  
  [root@kubernetes-master-01 ~]# kubectl apply -f rc.yaml
  replicationcontroller/nginx created
  [root@kubernetes-master-01 ~]# kubectl get rc
  NAME    DESIRED   CURRENT   READY   AGE
  nginx   3         3         0       5s
  [root@kubernetes-master-01 ~]# kubectl get pods
  NAME          READY   STATUS              RESTARTS   AGE
  nginx-cfp5l   0/1     ContainerCreating   0          10s
  nginx-p546l   0/1     ContainerCreating   0          10s
  nginx-vn9ng   0/1     ContainerCreating   0          10sCopy to clipboardErrorCopied
  

• 测试RC自动创建POD

  [root@kubernetes-master-01 ~]# kubectl get pods
  NAME          READY   STATUS    RESTARTS   AGE
  nginx-hhnw4   1/1     Running   0          16s
  nginx-p546l   1/1     Running   0          3m31s
  nginx-vn9ng   1/1     Running   0          3m31s
  [root@kubernetes-master-01 ~]# kubectl delete pod nginx-p546l
  pod "nginx-p546l" deleted
  [root@kubernetes-master-01 ~]# kubectl get pods
  NAME          READY   STATUS              RESTARTS   AGE
  nginx-hhnw4   1/1     Running             0          35s
  nginx-phbh9   0/1     ContainerCreating   0          5s
  nginx-vn9ng   1/1     Running             0          3m50sCopy to clipboardErrorCopied
  

  可以看到删除一个Pod之后，RC自动创建了Pod。

Service

部署在容器中的服务想要被外界访问到，就得做端口映射才行。但是这种方式在K8s中是无法实现的，因为每一次Pod的创建和重启其IP都会改变，而且同一个节点上的Pod可能存在多个，这个时候做端口映射是不现实的。而Service相当于我们Pod的VIP

Service的三种模式

Service有三种运行模式，其实是Service升级的三个版本。

userspace

Client要访问Pod时,它先将请求发给本机内核空间中的service规则，由它再将请求,转给监听在指定套接字上的kube-proxy，kube-proxy处理完请求，并分发请求到指定Server Pod后,再将请求递交给内核空间中的service,由service将请求转给指定的Server Pod。

iptables

直接由内核的Iptables转发到Pod，不在进过kube-proxy。

ipvs

它是直接有内核中的ipvs规则来接受Client Pod请求，并处理该请求,再有内核封包后，直接发给指定的Server Pod。

Service的四种类型

NodePort

创建这种方式的Service，内部可以通过ClusterIP进行访问，外部用户可以通过NodeIP:NodePort的方式单独访问每个Node上的实例。

[root@kubernetes-master-01 ~]# vim svc.yaml
[root@kubernetes-master-01 ~]# cat svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  type: NodePort
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30001
  selector:
    app: nginx
---
apiVersion: v1
kind: ReplicationController
metadata:
  name: nginx
spec:
  replicas: 3
  selector:
    app: nginx
  template:
    metadata:
      name: nginx
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: alvinos/django:v1
        ports:
        - containerPort: 80

[root@kubernetes-master-01 ~]# kubectl apply -f svc.yaml
service/nginx created
replicationcontroller/nginx configured
[root@kubernetes-master-01 ~]# kubectl get all
NAME              READY   STATUS              RESTARTS   AGE
pod/nginx-47s6q   0/1     ContainerCreating   0          13s
pod/nginx-kd6d6   1/1     Running             0          13s
pod/nginx-t2vqz   1/1     Running             0          13s

NAME                          DESIRED   CURRENT   READY   AGE
replicationcontroller/nginx   3         3         2       13s

NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
service/kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP        106m
service/nginx        NodePort    10.102.211.15   <none>        80:30001/TCP   13sCopy to clipboardErrorCopied

• 测试连接

  [root@kubernetes-master-01 ~]# while true; do curl 10.102.211.15/index; echo ''; sleep 1; done
  主机名：nginx-47s6q，版本：v1
  主机名：nginx-47s6q，版本：v1
  主机名：nginx-t2vqz，版本：v1
  主机名：nginx-47s6q，版本：v1
  主机名：nginx-47s6q，版本：v1
  主机名：nginx-t2vqz，版本：v1
  主机名：nginx-47s6q，版本：v1Copy to clipboardErrorCopied
  

LoadBalancer

接入外部负载均衡器的IP，到我们的k8s上

• 创建LoadBalancer

[root@kubernetes-master-01 ~]# cat svc.yaml <<EOF
apiVersion: v1
kind: Service
metadata:
  name: loadbalancer
spec:
  type: LoadBalancer
  ports:
    - port: 80
      targetPort: 80
  selector:
    app: nginx
EOFCopy to clipboardErrorCopied

• 查看

[root@kubernetes-node-01 ~]# kubectl get svc
NAME           TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)        AGE
kubernetes     ClusterIP      10.0.0.1      <none>         443/TCP        110d
loadbalancer   LoadBalancer   10.0.129.18   81.71.12.240   80:30346/TCP   11sCopy to clipboardErrorCopied

HeadLess Service

创建一个没有IP的特殊的Cluster IP类型的Service。

• 创建

  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: nginx-deployment
    labels:
      app: nginx
  spec:
    replicas: 2
    selector:
      matchLabels:
        app: nginx
    template:
      metadata:
        labels:
          app: nginx
      spec:
        containers:
          - name: nginx
            image: nginx
            ports:
              - containerPort: 80
                name: http
  ---
  apiVersion: v1
  kind: Service
  metadata:
    name: headless-service
  spec:
    selector:
      app: nginx
    ports:
      - protocol: TCP
        port: 80
        targetPort: 80
    clusterIP: None    Copy to clipboardErrorCopied
  

• 查看

  [root@kubernetes-master-01 ~]# kubectl apply -f headless.yaml
  deployment.apps/nginx-deployment created
  service/headless-service created
  [root@kubernetes-master-01 ~]# kubectl get svc
  NAME               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
  headless-service   ClusterIP   None            <none>        80/TCP         4s
  kubernetes         ClusterIP   10.96.0.1       <none>        443/TCP        119m
  [root@kubernetes-master-01 ~]# kubectl describe svc headless-service
  Name:              headless-service
  Namespace:         default
  Labels:            <none>
  Annotations:       <none>
  Selector:          app=nginx
  Type:              ClusterIP
  IP:                None
  Port:              <unset>  80/TCP
  TargetPort:        80/TCP
  Endpoints:         10.244.1.5:80,10.244.1.6:80,10.244.2.6:80 + 2 more...
  Session Affinity:  None
  Events:            <none>Copy to clipboardErrorCopied
  
  

Nginx Ingress

将新加入的Ingress转化成Nginx的配置文件并使之生效。

• 组成
  • Ingress Controller：将新加入的Ingress配置转化成nginx配置，并使其生效。
  • Ingress服务：将Nginx配置抽象成Ingress配置，方便使用。
• 工作原理

ingress controller通过和kubernetes api交互，动态的去感知集群中ingress规则变化。然后读取它，按照自定义的规则，去集群当中寻找对应Service管理的Pod。再将生成的Nginx配置写到nginx-ingress-control的pod里，这个Ingress controller的pod里运行着一个Nginx服务，控制器会把生成的nginx配置写入/etc/nginx.conf文件中。然后reload一下使配置生效。以此达到域名分配置和动态更新的问题。

• 优点
  • 动态配置服务
  • 减少不必要的端口

• 安装

  kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.35.0/deploy/static/provider/baremetal/deploy.yamlCopy to clipboardErrorCopied
  
  

• 查看安装

  [root@kubernetes-master-01 ~]# kubectl get pods -n ingress-nginx
  NAME                                        READY   STATUS      RESTARTS   AGE
  ingress-nginx-admission-create-nn7mh        0/1     Completed   0          14s
  ingress-nginx-admission-patch-vwdxb         0/1     Completed   0          14s
  ingress-nginx-controller-77f4649599-dtrrm   1/1     Running     0          14sCopy to clipboardErrorCopied
  
  

• 测试

  [root@kubernetes-master-01 ~]# curl 10.96.191.89
  <html>
  <head><title>404 Not Found</title></head>
  <body>
  <center><h1>404 Not Found</h1></center>
  <hr><center>nginx/1.19.2</center>
  </body>
  </html>
  
  # 出现404是因为我们后台没有启动任何服务Copy to clipboardErrorCopied
  
  

Nginx Ingress 经典案例

• 创建Deployment

  kind: Deployment
  apiVersion: apps/v1
  metadata:
    name: ingress-deployment
    namespace: default
    labels:
      app: deployment
  spec:
    replicas: 3
    selector:
      matchLabels:
        app: pod
    template:
      metadata:
        labels:
          app: pod
      spec:
        containers:
          - name: ingress-pod
            image: nginx
            imagePullPolicy: IfNotPresent
            ports:
              - containerPort: 80
                name: httpCopy to clipboardErrorCopied
  
  

  查看部署结果

  [root@kubernetes-master-01 ~]# kubectl get deployments.apps -l app=deployment
  NAME                 READY   UP-TO-DATE   AVAILABLE   AGE
  ingress-deployment   3/3     3            3           4m37s
  [root@kubernetes-master-01 ~]# kubectl get pods -l app=pod
  NAME                                  READY   STATUS    RESTARTS   AGE
  ingress-deployment-5f6798968b-8zsn8   1/1     Running   0          4m46s
  ingress-deployment-5f6798968b-bls6q   1/1     Running   0          4m46s
  ingress-deployment-5f6798968b-h6hgg   1/1     Running   0          4m46sCopy to clipboardErrorCopied
  
  

• 创建Service

  kind: Service
  apiVersion: v1
  metadata:
    name: ingress-service
    namespace: default
    labels:
      app: svc
  
  spec:
    type: ClusterIP
    selector:
      app: pod
  
    ports:
      - port: 80
        targetPort: 80
        name: httpCopy to clipboardErrorCopied
  
  

  查看部署结果

  [root@kubernetes-master-01 ~]# kubectl get svc -l app=svc
  NAME              TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
  ingress-service   ClusterIP   10.111.19.36   <none>        80/TCP    5m38sCopy to clipboardErrorCopied
  
  

• 创建Ingress

  kind: Ingress
  apiVersion: extensions/v1beta1
  metadata:
    name: ingress-ingress
    namespace: default
    annotations:
      kubernetes.io/ingress.class: "nginx"
  spec:
    rules:
      - host: www.test.com
        http:
          paths:
            - path: /
              backend:
                serviceName: ingress-service
                servicePort: 80Copy to clipboardErrorCopied
  
  

  查看部署结果

  [root@kubernetes-master-01 ~]# kubectl get ingress
  NAME              CLASS    HOSTS          ADDRESS       PORTS   AGE
  ingress-ingress   <none>   www.test.com   172.16.0.53   80      6m36sCopy to clipboardErrorCopied
  
  

• 测试

  # 添加Host解析 127.0.0.1  www.test.com
  [root@kubernetes-master-01 ~]# curl www.test.com
  <!DOCTYPE html>
  <html>
  <head>
  <title>Welcome to nginx!</title>
  <style>
      body {
           35em;
          margin: 0 auto;
          font-family: Tahoma, Verdana, Arial, sans-serif;
      }
  </style>
  </head>
  <body>
  <h1>Welcome to nginx!</h1>
  <p>If you see this page, the nginx web server is successfully installed and
  working. Further configuration is required.</p>
  
  <p>For online documentation and support please refer to
  <a href="http://nginx.org/">nginx.org</a>.<br/>
  Commercial support is available at
  <a href="http://nginx.com/">nginx.com</a>.</p>
  
  <p><em>Thank you for using nginx.</em></p>
  </body>
  </html>Copy to clipboardErrorCopied
  
  

Deployment

前面使用rc和service是通过selector进行关联的，但是在rc的滚动升级过程中selector是可能发生改变的，所以升级之后service与rc可能失去关联关系导致无法访问。而Deployment升级是新创建一个rc，然后在创建Pod，但是对Service没有影响。

• 经典案例

  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: nginx-deployment
    labels:
      app: nginx
  spec:
    replicas: 3
    selector:
      matchLabels:
        app: nginx
    template:
      metadata:
        labels:
          app: nginx
      spec:
        containers:
          - name: nginx
            image: nginx
            ports:
              - containerPort: 80Copy to clipboardErrorCopied
  
  

  查看部署结果

  [root@kubernetes-master-01 ~]# vim deployment.yaml
  [root@kubernetes-master-01 ~]# kubectl apply -f deployment.yaml
  deployment.apps/nginx-deployment configured
  [root@kubernetes-master-01 ~]# kubectl get deployments.apps -l app=nginx
  NAME               READY   UP-TO-DATE   AVAILABLE   AGE
  nginx-deployment   2/3     1            2           71m
  [root@kubernetes-master-01 ~]# kubectl get pods -l app=nginx
  NAME                                READY   STATUS              RESTARTS   AGE
  nginx-deployment-6b5fc5c798-452cf   0/1     ContainerCreating   0          12s
  nginx-deployment-6b5fc5c798-7f4nc   0/1     ContainerCreating   0          12s
  nginx-deployment-6b5fc5c798-jnqhc   0/1     ContainerCreating   0          12sCopy to clipboardErrorCopied
  
  

滚动升级

• 创建

  [root@kubernetes-master-01 ~]# kubectl apply -f de.yaml
  deployment.apps/nginx-deployment created
  [root@kubernetes-master-01 ~]# kubectl get deployments.apps -l app=nginx
  NAME               READY   UP-TO-DATE   AVAILABLE   AGE
  nginx-deployment   3/3     3            3           15sCopy to clipboardErrorCopied
  
  

• 升级

  [root@kubernetes-master-01 ~]# kubectl set image deployment/nginx-deployment nginx=alvinos/django:v2
  deployment.apps/nginx-deployment image updated
  [root@kubernetes-master-01 ~]# kubectl get pods -l app=nginx -w
  NAME                                READY   STATUS        RESTARTS   AGE
  nginx-deployment-566c49dbc7-47t6k   1/1     Terminating   0          6m47s
  nginx-deployment-566c49dbc7-54lmk   1/1     Terminating   0          6m47s
  nginx-deployment-566c49dbc7-zfrvg   1/1     Terminating   0          6m47s
  nginx-deployment-69d86bb564-8728c   1/1     Running       0          14s
  nginx-deployment-69d86bb564-d67ng   1/1     Running       0          32s
  nginx-deployment-69d86bb564-rn2r9   1/1     Running       0          9sCopy to clipboardErrorCopied
  
  

回滚

• 查看部署历史

  [root@kubernetes-master-01 ~]# kubectl rollout history deployment nginx-deployment
  deployment.apps/nginx-deployment
  REVISION  CHANGE-CAUSE
  1         kubectl apply --filename=de.yaml --record=true
  2         kubectl apply --filename=de.yaml --record=true
  3         kubectl apply --filename=de.yaml --record=trueCopy to clipboardErrorCopied
  
  

• 回滚上一个版本

  [root@kubernetes-master-01 ~]# kubectl rollout undo deployment nginx-deployment
  deployment.apps/nginx-deployment rolled back
  [root@kubernetes-master-01 ~]# kubectl rollout history deployment nginx-deployment
  deployment.apps/nginx-deployment
  REVISION  CHANGE-CAUSE
  1         kubectl apply --filename=de.yaml --record=true
  3         kubectl apply --filename=de.yaml --record=true
  4         kubectl apply --filename=de.yaml --record=trueCopy to clipboardErrorCopied
  
  

• 回滚指定版本

  [root@kubernetes-master-01 ~]# kubectl rollout undo deployment nginx-deployment --to-revision=3
  deployment.apps/nginx-deployment rolled back
  [root@kubernetes-master-01 ~]# kubectl rollout history deployment nginx-deployment
  deployment.apps/nginx-deployment
  REVISION  CHANGE-CAUSE
  1         kubectl apply --filename=de.yaml --record=true
  4         kubectl apply --filename=de.yaml --record=true
  5         kubectl apply --filename=de.yaml --record=trueCopy to clipboardErrorCopied
  
  

扩容与缩容

• patch

  [root@kubernetes-master-01 ~]# kubectl patch deployments.apps nginx-deployment -p '{"spec":{"replicas": 5}}'
  deployment.apps/nginx-deployment patched
  [root@kubernetes-master-01 ~]# kubectl get deployments.apps nginx-deployment
  NAME               READY   UP-TO-DATE   AVAILABLE   AGE
  nginx-deployment   5/5     5            5           5m21sCopy to clipboardErrorCopied
  
  

• scale

  [root@kubernetes-master-01 ~]# kubectl scale deployment nginx-deployment --replicas=10
  deployment.apps/nginx-deployment scaled
  [root@kubernetes-master-01 ~]# kubectl get deployments.apps nginx-deployment
  NAME               READY   UP-TO-DATE   AVAILABLE   AGE
  nginx-deployment   5/10    10           5           5m57sCopy to clipboardErrorCopied
  
  

部署dashboard

[root@kubernetes-master-01 ~]# wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.4/aio/deploy/recommended.yaml
[root@kubernetes-master-01 ~]# sed -i 's#kubernetesui/dashboard#registry.cn-hangzhou.aliyuncs.com/k8sos/dashboard#g' recommended.yaml
[root@kubernetes-master-01 ~]# sed -i 's#kubernetesui/metrics-scraper#registry.cn-hangzhou.aliyuncs.com/k8sos/metrics-scraper#g' recommended.yaml
[root@kubernetes-master-01 ~]# kubectl apply -f recommended.yaml
namespace/kubernetes-dashboard created
serviceaccount/kubernetes-dashboard created
service/kubernetes-dashboard created
secret/kubernetes-dashboard-certs created
secret/kubernetes-dashboard-csrf created
secret/kubernetes-dashboard-key-holder created
configmap/kubernetes-dashboard-settings created
role.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created
rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created
deployment.apps/kubernetes-dashboard created
service/dashboard-metrics-scraper created
deployment.apps/dashboard-metrics-scraper created
[root@kubernetes-master-01 ~]# kubectl get all -n kubernetes-dashboard
NAME                                             READY   STATUS              RESTARTS   AGE
pod/dashboard-metrics-scraper-55f5746456-j5thm   0/1     ContainerCreating   0          15s
pod/kubernetes-dashboard-68784c9d-rffmb          1/1     Running             0          16s

NAME                                TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/dashboard-metrics-scraper   ClusterIP   10.102.154.91   <none>        8000/TCP   16s
service/kubernetes-dashboard        ClusterIP   10.111.190.51   <none>        443/TCP    16s

NAME                                        READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/dashboard-metrics-scraper   0/1     1            0           15s
deployment.apps/kubernetes-dashboard        1/1     1            1           16s

NAME                                                   DESIRED   CURRENT   READY   AGE
replicaset.apps/dashboard-metrics-scraper-55f5746456   1         1         0       15s
replicaset.apps/kubernetes-dashboard-68784c9d          1         1         1       16sCopy to clipboardErrorCopied

查看部署结果

持久化存储

我们知道，Pod是由容器组成的，而容器宕机或停止之后，数据就随之丢了，那么这也就意味着我们在做Kubernetes集群的时候就不得不考虑存储的问题，而存储卷就是为了Pod保存数据而生的。存储卷的类型有很多，我们常用到一般有四种：emptyDir，hostPath，NFS以及云存储等。

emptyDir

emptyDir类型的volume在pod分配到node上时被创建，kubernetes会在node上自动分配 一个目录，因此无需指定宿主机node上对应的目录文件。这个目录的初始内容为空，当Pod从node上移除时，emptyDir中的数据会被永久删除。emptyDir Volume主要用于某些应用程序无需永久保存的临时目录。

• 经典案例

  kind: Deployment
  apiVersion: apps/v1
  metadata:
    name: test-volume-deployment
    namespace: default
    labels:
      app: test-volume-deployment
  spec:
    selector:
      matchLabels:
        app: test-volume-pod
    template:
      metadata:
        labels:
          app: test-volume-pod
      spec:
        containers:
          - name: nginx
            image: busybox
            imagePullPolicy: IfNotPresent
            ports:
              - containerPort: 80
                name: http
              - containerPort: 443
                name: https
            volumeMounts:
              - mountPath: /data/
                name: empty
            command: ['/bin/sh','-c','while true;do echo $(date) >> /data/index.html;sleep 2;done']
          - name: os
            imagePullPolicy: IfNotPresent
            image: busybox
            volumeMounts:
              - mountPath: /data/
                name: empty
            command: ['/bin/sh','-c','while true;do echo 'budybox' >> /data/index.html;sleep 2;done']
        volumes:
          - name: empty
            emptyDir: {}Copy to clipboardErrorCopied
  
  

  查看部署状态

  [root@kubernetes-master-01 ~]# kubectl get pods -l app=test-volume-pod
  NAME                                      READY   STATUS    RESTARTS   AGE
  test-volume-deployment-66ccd5586b-s7j26   2/2     Running   0          4m13sCopy to clipboardErrorCopied
  
  

hostPath

hostPath类型则是映射node文件系统中的文件或者目录到pod里。在使用hostPath类型的存储卷时，也可以设置type字段，支持的类型有文件、目录、File、Socket、CharDevice和BlockDevice。

• 经典案例

apiVersion: v1
kind: Pod
metadata:
  name: vol-hostpath
  namespace: default
spec:
  volumes:
  - name: html
    hostPath:
      path: /data/pod/volume1/
      type: DirectoryOrCreate
  containers:
  - name: myapp
    image: nginx
    volumeMounts:
    - name: html
      mountPath: /usr/share/nginx/html/Copy to clipboardErrorCopied

• 查看部署结果

  # 在node-02上执行
  [root@kubernetes-node-02 volume1]# echo "index" > index.html
  # 在master-01上执行
  [root@kubernetes-master-01 data]# kubectl get pods vol-hostpath -o wide
  NAME           READY   STATUS    RESTARTS   AGE   IP            NODE                 NOMINATED NODE   READINESS GATES
  vol-hostpath   1/1     Running   0          66s   10.244.2.34   kubernetes-node-02   <none>           <none>
  [root@kubernetes-master-01 data]#  curl 10.244.2.34
  indexCopy to clipboardErrorCopied
  
  

PV和PVC

PersistentVolume（PV）是集群中已由管理员配置的一段网络存储。 集群中的资源就像一个节点是一个集群资源。 PV是诸如卷之类的卷插件，但是具有独立于使用PV的任何单个pod的生命周期。 该API对象捕获存储的实现细节，即NFS，iSCSI或云提供商特定的存储系统。

PersistentVolumeClaim（PVC）是用户存储的请求。PVC的使用逻辑：在pod中定义一个存储卷（该存储卷类型为PVC），定义的时候直接指定大小，pvc必须与对应的pv建立关系，pvc会根据定义去pv申请，而pv是由存储空间创建出来的。pv和pvc是kubernetes抽象出来的一种存储资源。

nfs

nfs使得我们可以挂载已经存在的共享到我们的Pod中，和emptyDir不同的是，当Pod被删除时，emptyDir也会被删除。但是nfs不会被删除，仅仅是解除挂在状态而已，这就意味着NFS能够允许我们提前对数据进行处理，而且这些数据可以在Pod之间相互传递，并且nfs可以同时被多个pod挂在并进行读写。

• 在每个需要装nfs的节点上安装nfs

  yum install nfs-utils.x86_64 -yCopy to clipboardErrorCopied
  
  

• 在每一个节点上配置nfs

  [root@kubernetes-master-01 nfs]# mkdir -p /nfs/v{1..5}
  
  [root@kubernetes-master-01 nfs]# cat > /etc/exports <<EOF
  /nfs/v1  172.16.0.0/16(rw,no_root_squash)
  /nfs/v2  172.16.0.0/16(rw,no_root_squash)
  /nfs/v3  172.16.0.0/16(rw,no_root_squash)
  /nfs/v4  172.16.0.0/16(rw,no_root_squash)
  /nfs/v5  172.16.0.0/16(rw,no_root_squash)
  EOF
  [root@kubernetes-master-01 nfs]# exportfs -arv
  exporting 172.16.0.0/16:/nfs/v5
  exporting 172.16.0.0/16:/nfs/v4
  exporting 172.16.0.0/16:/nfs/v3
  exporting 172.16.0.0/16:/nfs/v2
  exporting 172.16.0.0/16:/nfs/v1
  [root@kubernetes-master-01 nfs]# showmount -e
  Export list for kubernetes-master-01:
  /nfs/v5 172.16.0.0/16
  /nfs/v4 172.16.0.0/16
  /nfs/v3 172.16.0.0/16
  /nfs/v2 172.16.0.0/16
  /nfs/v1 172.16.0.0/16Copy to clipboardErrorCopied
  
  

• 创建POD使用Nfs

  [root@kubernetes-master-01 ~]# kubectl get pods -l app=nfs
  NAME                   READY   STATUS    RESTARTS   AGE
  nfs-5f56db5995-9shkg   1/1     Running   0          24s
  nfs-5f56db5995-ht7ww   1/1     Running   0          24s
  [root@kubernetes-master-01 ~]# echo "index" > /nfs/v1/index.html
  [root@kubernetes-master-01 ~]# kubectl exec -it nfs-5f56db5995-ht7ww -- bash
  root@nfs-5f56db5995-ht7ww:/# cd /usr/share/nginx/html/
  root@nfs-5f56db5995-ht7ww:/usr/share/nginx/html# ls
  index.htmlCopy to clipboardErrorCopied
  
  

PV 的访问模式

ReadWriteOnce（RWO）	可读可写，但只支持被单个节点挂载。
ReadOnlyMany（ROX）	只读，可以被多个节点挂载。
ReadWriteMany（RWX）	多路可读可写。这种存储可以以读写的方式被多个节点共享。不是每一种存储都支持这三种方式，像共享方式，目前支持的还比较少，比较常用的是 NFS。在 PVC 绑定 PV 时通常根据两个条件来绑定，一个是存储的大小，另一个就是访问模式。

PV 的回收策略

Retain	不清理, 保留 Volume（需要手动清理）
Recycle	删除数据，即 rm -rf /thevolume/*（只有 NFS 和 HostPath 支持）
Delete	删除存储资源，比如删除 AWS EBS 卷（只有 AWS EBS, GCE PD, Azure Disk 和 Cinder 支持）

PV 的四种状态

Available	可用。
Bound	已经分配给 PVC。
Released	PVC 解绑但还未执行回收策略。
Failed	发生错误。

创建PV

kind: PersistentVolume
apiVersion: v1
metadata:
  name: pv001
  labels:
    app: pv001
spec:
  nfs:
    path: /nfs/v2
    server: 172.16.0.50
  accessModes:
    - "ReadWriteMany"
    - "ReadWriteOnce"
  capacity:
    storage: 2Gi
---
kind: PersistentVolume
apiVersion: v1
metadata:
  name: pv002
  labels:
    app: pv002
spec:
  nfs:
    path: /nfs/v3
    server: 172.16.0.50
  accessModes:
    - "ReadWriteMany"
    - "ReadWriteOnce"
  capacity:
    storage: 5Gi
  persistentVolumeReclaimPolicy: Delete
---
kind: PersistentVolume
apiVersion: v1
metadata:
  name: pv003
  labels:
    app: pv003
spec:
  nfs:
    path: /nfs/v4
    server: 172.16.0.50
  accessModes:
    - "ReadWriteMany"
    - "ReadWriteOnce"
  capacity:
    storage: 10Gi
---
kind: PersistentVolume
apiVersion: v1
metadata:
  name: pv004
  labels:
    app: pv004
spec:
  nfs:
    path: /nfs/v5
    server: 172.16.0.50
  accessModes:
    - "ReadWriteMany"
    - "ReadWriteOnce"
  capacity:
    storage: 20GiCopy to clipboardErrorCopied

通过PVC使用PV

[root@kubernetes-master-01 ~]# kubectl apply -f pv.yaml
persistentvolume/pv001 created
persistentvolume/pv002 created
persistentvolume/pv003 created
persistentvolume/pv004 created
[root@kubernetes-master-01 ~]# kubectl get pv
NAME    CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
pv001   2Gi        RWO,RWX        Retain           Available                                   10s
pv002   5Gi        RWO,RWX        Delete           Available                                   10s
pv003   10Gi       RWO,RWX        Retain           Available                                   10s
pv004   20Gi       RWO,RWX        Retain           Available                                   10s

[root@kubernetes-master-01 ~]# cat > pvc.yaml <<EOF
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc
  namespace: default

spec:
  accessModes:
    - "ReadWriteMany"
  resources:
    requests:
      storage: "6Gi"

---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nfs
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nfs
  template:
    metadata:
      labels:
        app: nfs
    spec:
      containers:
        - name: nginx
          imagePullPolicy: IfNotPresent
          image: nginx
          volumeMounts:
            - name: html
              mountPath: /usr/share/nginx/html/
      volumes:
        - name: html
          persistentVolumeClaim:
            claimName: pvc # PVC 名字
EOF
[root@kubernetes-master-01 ~]# kubectl apply -f pv.yaml
persistentvolumeclaim/pvc created
deployment.apps/nfs configured
[root@kubernetes-master-01 ~]# kubectl get pvc
NAME   STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
pvc    Bound    pv003    10Gi       RWO,RWX                       8s
[root@kubernetes-master-01 ~]# kubectl get pv
NAME    CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM         STORAGECLASS   REASON   AGE
pv001   2Gi        RWO,RWX        Retain           Available                                         4m18s
pv002   5Gi        RWO,RWX        Delete           Available                                         4m18s
pv003   10Gi       RWO,RWX        Retain           Bound       default/pvc                           4m18s
pv004   20Gi       RWO,RWX        Retain           Available                                         4m18sCopy to clipboardErrorCopied

StorageClass

在一个大规模的Kubernetes集群里,可能有成千上万个PVC,这就意味着运维人员必须实现创建出这个多个PV,此外,随着项目的需要,会有新的PVC不断被提交,那么运维人员就需要不断的添加新的,满足要求的PV,否则新的Pod就会因为PVC绑定不到PV而导致创建失败。而且通过 PVC 请求到一定的存储空间也很有可能不足以满足应用对于存储设备的各种需求，而且不同的应用程序对于存储性能的要求可能也不尽相同，比如读写速度、并发性能等，为了解决这一问题，Kubernetes 又为我们引入了一个新的资源对象：StorageClass，通过 StorageClass 的定义，管理员可以将存储资源定义为某种类型的资源，比如快速存储、慢速存储等，kubernetes根据 StorageClass 的描述就可以非常直观的知道各种存储资源的具体特性了，这样就可以根据应用的特性去申请合适的存储资源了。

定义StorageClass

每一个存储类都包含provisioner、parameters和reclaimPolicy这三个参数域，当一个属于某个类的PersistentVolume需要被动态提供时，将会使用上述的参数域。

• 创建StorageClass

[root@kubernetes-master-01 ~]# cat > sc.yaml <<EOF
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-storage-class
  annotations: 
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: kubernetes/nfs-storage-class
parameters:
  archiveOnDelete: "false" # pod删除后，不删除数据
EOF
[root@kubernetes-master-01 ~]# kubectl apply -f sc.yaml
storageclass.storage.k8s.io/nfs-storage-class created
[root@kubernetes-master-01 ~]# kubectl get sc
NAME                          PROVISIONER                    RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-storage-class (default)   kubernetes/nfs-storage-class   Delete          Immediate           false                  5sCopy to clipboardErrorCopied

• 创建PVC

[root@kubernetes-master-01 ~]# cat > sc.yaml <<EOF
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
  - apiGroups: [""]
    resources: ["services", "endpoints"]
    verbs: ["get", "create", "update", "delete"]
  - apiGroups: ["extensions"]
    resources: ["podsecuritypolicies"]
    resourceNames: ["nfs-provisioner"]
    verbs: ["use"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
     # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-provisioner
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-provisioner
  apiGroup: rbac.authorization.k8s.io
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nfs-client-provisioner
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: registry.cn-hangzhou.aliyuncs.com/kubeapps/quay-nfs-client-provisioner
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: pri/nfs
            - name: NFS_SERVER
              value: 172.16.0.50
            - name: NFS_PATH
              value: /nfs/v8
      volumes:
        - name: nfs-client-root
          nfs:
            server: 172.16.0.50
            path: /nfs/v8
---
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: nfs
provisioner: pri/nfs
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-sc
spec:
  accessModes:
    - ReadWriteOnce
    - ReadWriteMany
  storageClassName: "nfs"
  resources:
    requests:
      storage: 1Gi
EOF
[root@kubernetes-master-01 ~]# kubectl apply -f sc.yaml
storageclass.storage.k8s.io/nfs-storage-class created
deployment.apps/nfs-client-provisioner created
persistentvolumeclaim/claim1 created
[root@kubernetes-master-01 ~]# kubectl get pvc -o wide
NAME     STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE   VOLUMEMODE
pvc-sc   Bound    pvc-33a45e8b-e178-4c6b-b64c-fe28073e0a30   1Gi        RWO,RWX        nfs            24s   Filesystem
[root@kubernetes-master-01 ~]# kubectl get deployments.apps
NAME                     READY   UP-TO-DATE   AVAILABLE   AGE
nfs-client-provisioner   1/1     1            1           118sCopy to clipboardErrorCopied