k8s 资源管理

1. kubectl 命令行

1.1 管理k8s 核心资源的三种方法

• 陈述式管理方法

  依赖CLI 工具进行管理

• 声明式管理方法

  依赖资源配置清单 （manifest）进行管理

• GUI 式管理方法

  主要依赖图形化操作界面（web页面）进行管理

1.2 陈述式管理方法

使用 kubectl 命令进行管理

用于增删查较为方便

1.2.1 查看所有资源

kubectl get all -n kube-public

1.2.2 管理名称空间资源

• 查看某个名称空间下所有的资源

kubectl get all -n default

• 创建名称空间test

kubectl create namespace test

• 删除名称空间test

kubectl delete namespace test

1.2.3 管理Deployment 资源

• 创建deployment

kubectl create deploy nginx-dp --image harbor.od.com/public/nginx:1.7.9 -n kube-public

• 查看deployment

kubectl get deploy -n kube-public
kubectl describe deploy nginx-dp -n kube-public

1.2.4 管理pod资源

• 查看pod资源

kubectl get pods -n kube-public

• 进入pod资源

kubectl exec -it nginx-dp-656b87bf6d-bjt2c bash -n kube-public

1.2.5 管理service资源

通过名称空间和标签选择器关联service

• 创建deployment

kubectl create deploy nginx-dp --image harbor.od.com/public/nginx:1.7.9 -n kube-public

• 创建service

kubectl expose deployment nginx-dp --port=80 -n kube-public

不能够对daemonset 创建service

• 查看service

kubectl get service -n kube-public

• 扩容deployment

kubectl scale deployment nginx-dp --replicas=2 -n kube-public

• 查看代理的pod地址

ipvsadm -Ln

• 查看pod的ip地址

kubectl get pod -n kube-public -owide

• 查看service详情

kubectl describe svc nginx-dp -n kube-public

1.2.6 查看帮助信息

kubectl explain svc.metadata

1.3 声明式管理方法

依赖于YAML和JSON格式的资源配置清单进行管理, 用于修改较为方便

1.3.1 查看某个deployment的资源配置清单

kubectl get deploy nginx-dp -oyaml -n kube-public

1.3.2 查看某个service的资源配置清单

kubectl get svc nginx-dp -oyaml -n kube-public

1.3.3 创建资源配置清单

vim nginx-ds-svc.yaml

apiVersion: v1
kind: Service
metadata:
  labels: 
    app: nginx-ds
  name: nginx-ds
  namespace: default
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx-ds
  sessionAffinity: None
  type: ClusterIP

kubectl create -f nginx-ds-svc.yaml

kubectl get svc -n default

1.3.4 在线修改资源配置清单

kubectl edit svc nginx-ds

kubectl get svc

1.3.5 离线修改资源配置清单

vim nginx-ds-svc.yaml

apiVersion: v1
kind: Service
metadata:
  labels: 
    app: nginx-ds
  name: nginx-ds
  namespace: default
spec:
  ports:
  - port: 8088
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx-ds
  sessionAffinity: None
  type: ClusterIP

kubectl apply -f nginx-ds-svc.yaml

kubectl get svc

1.3.6 删除资源配置清单

kubectl delete -f nginx-ds-svc.yaml

kubectl get svc

2. kubernetes CNi网络插件 flannel

主要实现了跨宿主机网络通信

常见的CNI网络插件

• Flannel
• Calico
• Canal
• Contiv
• OpenContrail
• NSX-T
• Kube-router

主机名	角色	IP地址
hdss7-21.host.com	flannel	10.4.7.21
hdss7-22.host.com	flannel	10.4.7.22
hdss7-23.host.com	flannel	10.4.7.23
hdss7-24.host.com	flannel	10.4.7.24

2.1 下载软件，解压， 软链接

https://github.com/coreos/flannel/releases

cd /opt/src/
wget https://github.com/coreos/flannel/releases/download/v0.12.0/flannel-v0.12.0-linux-amd64.tar.gz
mkdir /opt/flannel-v0.12.0
tar xf flannel-v0.12.0-linux-amd64.tar.gz  -C /opt/flannel-v0.12.0/
ln -s /opt/flannel-v0.12.0/ /opt/flannel

2.2 创建目录并拷贝证书

mkdir /opt/flannel/cert
cd /opt/flannel/cert/
scp hdss7-200:/opt/certs/ca.pem .                   
scp hdss7-200:/opt/certs/client.pem .                                       
scp hdss7-200:/opt/certs/client-key.pem .

2.3 创建配置

cd /opt/flannel
vi subnet.env

• 10.4.7.21

FLANNEL_NETWORK=172.7.0.0/16
FLANNEL_SUBNET=172.7.21.1/24
FLANNEL_MTU=1500
FLANNEL_IPMASQ=false

• 10.4.7.22

FLANNEL_NETWORK=172.7.0.0/16
FLANNEL_SUBNET=172.7.22.1/24
FLANNEL_MTU=1500
FLANNEL_IPMASQ=false

2.4 创建启动脚本

vi flanneld.sh

• 10.4.7.21

#!/bin/sh
./flanneld 
  --public-ip=10.4.7.21 
  --etcd-endpoints=https://10.4.7.12:2379,https://10.4.7.21:2379,https://10.4.7.22:2379 
  --etcd-keyfile=./cert/client-key.pem 
  --etcd-certfile=./cert/client.pem 
  --etcd-cafile=./cert/ca.pem 
  --iface=eth0 
  --subnet-file=./subnet.env 
  --healthz-port=2401

• 10.4.7.22

#!/bin/sh
./flanneld 
  --public-ip=10.4.7.22 
  --etcd-endpoints=https://10.4.7.12:2379,https://10.4.7.21:2379,https://10.4.7.22:2379 
  --etcd-keyfile=./cert/client-key.pem 
  --etcd-certfile=./cert/client.pem 
  --etcd-cafile=./cert/ca.pem 
  --iface=eth0 
  --subnet-file=./subnet.env 
  --healthz-port=2401

• 10.4.7.23

#!/bin/sh
./flanneld 
  --public-ip=10.4.7.23 
  --etcd-endpoints=https://10.4.7.12:2379,https://10.4.7.21:2379,https://10.4.7.22:2379 
  --etcd-keyfile=./cert/client-key.pem 
  --etcd-certfile=./cert/client.pem 
  --etcd-cafile=./cert/ca.pem 
  --iface=eth0 
  --subnet-file=./subnet.env 
  --healthz-port=2401

• 10.4.7.24

#!/bin/sh
./flanneld 
  --public-ip=10.4.7.24 
  --etcd-endpoints=https://10.4.7.12:2379,https://10.4.7.21:2379,https://10.4.7.22:2379 
  --etcd-keyfile=./cert/client-key.pem 
  --etcd-certfile=./cert/client.pem 
  --etcd-cafile=./cert/ca.pem 
  --iface=eth0 
  --subnet-file=./subnet.env 
  --healthz-port=2401

2.5 创建日志文件

chmod +x flaneld.sh
mkdir -p /data/logs/flanneld

2.6 操作etcd, 创建host-gw

10.4.7.21上

cd /opt/etcd
./etcdctl member list

因为 10.4.7.21是etcd的leader角色， 所以在 10.4.7.21上操作

./etcdctl set /coreos.com/network/config '{"Network": "172.7.0.0/16", "Backend": {"Type": "host-gw"}}'

./etcdctl get /coreos.com/network/config

2.7 创建supervisor配置

vi /etc/supervisord.d/flannel.ini

• 10.4.7.21

[program:flanneld-7-21]
command=/opt/flannel/flanneld.sh                             ; the program (relative uses PATH, can take args)
numprocs=1                                                   ; number of processes copies to start (def 1)
directory=/opt/flannel                                       ; directory to cwd to before exec (def no cwd)
autostart=true                                               ; start at supervisord start (default: true)
autorestart=true                                             ; retstart at unexpected quit (default: true)
startsecs=30                                                 ; number of secs prog must stay running (def. 1)
startretries=3                                               ; max # of serial start failures (default 3)
exitcodes=0,2                                                ; 'expected' exit codes for process (default 0,2)
stopsignal=QUIT                                              ; signal used to kill process (default TERM)
stopwaitsecs=10                                              ; max num secs to wait b4 SIGKILL (default 10)
user=root                                                    ; setuid to this UNIX account to run the program
redirect_stderr=true                                         ; redirect proc stderr to stdout (default false)
stdout_logfile=/data/logs/flanneld/flanneld.stdout.log       ; stderr log path, NONE for none; default AUTO
stdout_logfile_maxbytes=64MB                                 ; max # logfile bytes b4 rotation (default 50MB)
stdout_logfile_backups=4                                     ; # of stdout logfile backups (default 10)
stdout_capture_maxbytes=1MB                                  ; number of bytes in 'capturemode' (default 0)
stdout_events_enabled=false                                  ; emit events on stdout writes (default false)

• 10.4.7.22

[program:flanneld-7-22]
command=/opt/flannel/flanneld.sh                             ; the program (relative uses PATH, can take args)
numprocs=1                                                   ; number of processes copies to start (def 1)
directory=/opt/flannel                                       ; directory to cwd to before exec (def no cwd)
autostart=true                                               ; start at supervisord start (default: true)
autorestart=true                                             ; retstart at unexpected quit (default: true)
startsecs=30                                                 ; number of secs prog must stay running (def. 1)
startretries=3                                               ; max # of serial start failures (default 3)
exitcodes=0,2                                                ; 'expected' exit codes for process (default 0,2)
stopsignal=QUIT                                              ; signal used to kill process (default TERM)
stopwaitsecs=10                                              ; max num secs to wait b4 SIGKILL (default 10)
user=root                                                    ; setuid to this UNIX account to run the program
redirect_stderr=true                                         ; redirect proc stderr to stdout (default false)
stdout_logfile=/data/logs/flanneld/flanneld.stdout.log       ; stderr log path, NONE for none; default AUTO
stdout_logfile_maxbytes=64MB                                 ; max # logfile bytes b4 rotation (default 50MB)
stdout_logfile_backups=4                                     ; # of stdout logfile backups (default 10)
stdout_capture_maxbytes=1MB                                  ; number of bytes in 'capturemode' (default 0)
stdout_events_enabled=false                                  ; emit events on stdout writes (default false)

• 10.4.7.23

[program:flanneld-7-23]
command=/opt/flannel/flanneld.sh                             ; the program (relative uses PATH, can take args)
numprocs=1                                                   ; number of processes copies to start (def 1)
directory=/opt/flannel                                       ; directory to cwd to before exec (def no cwd)
autostart=true                                               ; start at supervisord start (default: true)
autorestart=true                                             ; retstart at unexpected quit (default: true)
startsecs=30                                                 ; number of secs prog must stay running (def. 1)
startretries=3                                               ; max # of serial start failures (default 3)
exitcodes=0,2                                                ; 'expected' exit codes for process (default 0,2)
stopsignal=QUIT                                              ; signal used to kill process (default TERM)
stopwaitsecs=10                                              ; max num secs to wait b4 SIGKILL (default 10)
user=root                                                    ; setuid to this UNIX account to run the program
redirect_stderr=true                                         ; redirect proc stderr to stdout (default false)
stdout_logfile=/data/logs/flanneld/flanneld.stdout.log       ; stderr log path, NONE for none; default AUTO
stdout_logfile_maxbytes=64MB                                 ; max # logfile bytes b4 rotation (default 50MB)
stdout_logfile_backups=4                                     ; # of stdout logfile backups (default 10)
stdout_capture_maxbytes=1MB                                  ; number of bytes in 'capturemode' (default 0)
stdout_events_enabled=false                                  ; emit events on stdout writes (default false)

• 10.4.7.24

[program:flanneld-7-24]
command=/opt/flannel/flanneld.sh                             ; the program (relative uses PATH, can take args)
numprocs=1                                                   ; number of processes copies to start (def 1)
directory=/opt/flannel                                       ; directory to cwd to before exec (def no cwd)
autostart=true                                               ; start at supervisord start (default: true)
autorestart=true                                             ; retstart at unexpected quit (default: true)
startsecs=30                                                 ; number of secs prog must stay running (def. 1)
startretries=3                                               ; max # of serial start failures (default 3)
exitcodes=0,2                                                ; 'expected' exit codes for process (default 0,2)
stopsignal=QUIT                                              ; signal used to kill process (default TERM)
stopwaitsecs=10                                              ; max num secs to wait b4 SIGKILL (default 10)
user=root                                                    ; setuid to this UNIX account to run the program
redirect_stderr=true                                         ; redirect proc stderr to stdout (default false)
stdout_logfile=/data/logs/flanneld/flanneld.stdout.log       ; stderr log path, NONE for none; default AUTO
stdout_logfile_maxbytes=64MB                                 ; max # logfile bytes b4 rotation (default 50MB)
stdout_logfile_backups=4                                     ; # of stdout logfile backups (default 10)
stdout_capture_maxbytes=1MB                                  ; number of bytes in 'capturemode' (default 0)
stdout_events_enabled=false                                  ; emit events on stdout writes (default false)

2.9 更新supervisor配置

supervisorctl update
supervisorctl status

2.10 验证集群

kubectl get pods -owide
ping 172.7.22.2

2.11 flannel 原理

2.11.1 host-gw 模型

前提是宿主机之间是相通的，通过添加一条路由实现 跨宿主机容器间的网络通信

2.11.2 VxLAN 模型

通过 各自添加一块虚拟网卡 flannel.1 来进行通信实现 跨宿主机容器间的网络通信

2.12 SNAT规则优化

2.12.1 制作nginx镜像

mkdir -p /opt/dockerfile/nginx
vim /opt/dockerfile/nginx/dockerfile

FROM centos:7
RUN useradd  www -u 1200 -M -s /sbin/nologin
RUN mkdir -p /var/log/nginx
RUN yum install -y cmake pcre pcre-devel openssl openssl-devel gd-devel 
    zlib-devel gcc gcc-c++ net-tools iproute telnet wget curl &&
    yum clean all && 
    rm -rf /var/cache/yum/*
RUN wget https://www.chenleilei.net/soft/nginx-1.16.1.tar.gz
RUN tar xf nginx-1.16.1.tar.gz
WORKDIR nginx-1.16.1
RUN ./configure --prefix=/usr/local/nginx --with-http_image_filter_module --user=www --group=www 
    --with-http_ssl_module --with-http_v2_module --with-http_stub_status_module 
    --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log 
    --pid-path=/var/run/nginx/nginx.pid
RUN make -j 4 && make install && 
    rm -rf /usr/local/nginx/html/*  && 
    echo "leilei hello" >/usr/local/nginx/html/index.html  && 
    rm -rf nginx* && 
    ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime &&
    ln -sf /dev/stdout /var/log/nginx/access.log && 
    ln -sf /dev/stderr /var/log/nginx/error.log
RUN chown -R www.www /var/log/nginx
ENV LOG_DIR /var/log/nginx
ENV PATH $PATH:/usr/local/nginx/sbin
#COPY nginx.conf /usr/local/nginx/conf/nginx.conf
EXPOSE 80
WORKDIR /usr/local/nginx
CMD ["nginx","-g","daemon off;"]

docker image build -t nginx:curl /opt/dockerfile/nginx
docker tag nginx:curl harbor.od.com/public/nginx:curl
docker push !$

2.12.3 修改damonset的镜像

/root/nginx-ds.yaml

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: nginx-ds
spec:
  template:
    metadata:
      labels:
        app: nginx-ds
    spec:
      containers:
      - name: my-nginx
        image: harbor.od.com/public/nginx:curl
        ports:
        - containerPort: 80
      tolerations:
      - key: hdss7-21
        operator: Exists
        effect: NoSchedule
      - key: hdss7-22
        operator: Exists
        effect: NoSchedule

kubectl apply -f nginx-ds.yaml

重启所有的pod

2.12.4 容器内通过curl， 日志中会显示node的IP地址

• 10.4.7.21上操作

kubectl get pods -owide

kubectl exec -ti nginx-ds-xgpjp bash
curl -I 172.7.22.2

• 10.4.7.22上操作

kubectl logs -f nginx-ds-8g2lr

2.12.5 优化iptables规则

iptables-save |grep -i postrouting

• 安装iptables-services

yum -y install iptables-services
systemctl start iptables.service 
systemctl enable iptables.service

• 删除原来的规则

iptables -t nat -D POSTROUTING -s 172.7.21.0/24 ! -o docker0 -j MASQUERADE

• 添加规则

iptables -t nat -I POSTROUTING -s 172.7.21.0/24 ! -d 172.7.0.0/16 ! -o docker0 -j MASQUERADE

• 将iptables的reject规则清空

iptables-save |grep reject

iptables -t filter -D INPUT -j REJECT --reject-with icmp-host-prohibited
iptables -t filter -D FORWARD -j REJECT --reject-with icmp-host-prohibited

• 保存规则

iptables-save > /etc/sysconfig/iptables

• 10.4.7.21上操作

kubectl exec -ti nginx-ds-xgpjp bash
curl -I 172.7.22.2

• 10.4.7.22上操作

kubectl logs -f nginx-ds-8g2lr

2.12.6 问题

https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file

这里会出现问题， 如果机器重启或者重启docker, iptables的规则就会被docker重置， 解决方案是将docker的iptables规则禁止或者在.bashrc中添加这条iptables规则

docker进程禁用iptables

/etc/docker/daemon.json

{
"graph": "/data/docker",
"storage-driver": "overlay2",
"insecure-registries": ["registry.access.redhat.com", "quay.io", "harbor.od.com"],
"registry-mirrors": ["https://q2gr04ke.mirror.aliyuncs.com"],
"bip": "172.7.21.1/24",
"iptables": false,
"exec-opts": ["native.cgroupdriver=systemd"],
"live-restore": true
}

这种方案造成的问题是重启主机后所有关于docker的iptables规则都没有了

启动时候添加iptables规则

~/.bashrc

iptables -t nat -D POSTROUTING -s 172.7.21.0/24 ! -o docker0 -j MASQUERADE
iptables -t nat -I POSTROUTING -s 172.7.21.0/24 ! -d 172.7.0.0/16 ! -o docker0 -j MASQUERADE
iptables-save > /etc/sysconfig/iptables

3. kubernetes 服务发现插件 coredns

服务发现就是服务（应用）之间相互定位的过程

服务发现主要有以下场景

• 服务（应用）的动态性强
• 服务（应用）更新发布频繁
• 服务（应用）支持自动伸缩

coredns 关联service的名称和它的cluster ip

coredns 是k8s中默认的dns插件， 只负责维护 ervice的名称和它的cluster ip的关联关系

3.1.部署kubernetesubernetes的内网资源配置清单

在hdss7-200.host.com (10.4.7.200)上，配置一个NGINX，用以提供kubernetes统一的访问资源清单入口

vi /etc/nginx/conf.d/k8s-yaml.od.com.conf

server {
    listen       80;
    server_name  k8s-yaml.od.com;

    location / {
        autoindex on;
        default_type text/plain;
        root /data/k8s-yaml;
    }
}

nginx -s reload

mkdir -p /data/k8s-yaml/coredns

• 在 10.4.7.11 注册 k8s-yaml.od.com 的域名

  /var/named/od.com.zone

ORIGIN od.com.
$TTL 600	; 10 minutes
@	IN SOA	dns.od.com.  dnsadmin.od.com.  (
				 2020062204   ; serial
				 10800      ; refresh (3 hours)
				 900        ; retry (15 minutes)
				 604800     ; expire (1 week)
				 86400      ; minimun (1 day)
				 )
			NS	dns.od.com.
$TTL 60 ; 1 minute
dns		   A	  10.4.7.11
harbor             A      10.4.7.200
k8s-yaml           A      10.4.7.200

systemctl restart named

• 查看dns解析

dig -t A k8s-yaml.od.com +short

• 浏览器访问 k8s-yaml.od.com

3.2 部署coredns

3.2.1 准备 coredns镜像

docker pull coredns/coredns:1.7.0
docker tag coredns/coredns:1.7.0 harbor.od.com/public/coredns:1.7.0
docker push !$

3.2.2 准备资源配置清单

/data/k8s-yaml/coredns/rbac.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: coredns
  namespace: kube-system
  labels:
      kubernetes.io/cluster-service: "true"
      addonmanager.kubernetes.io/mode: Reconcile
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
    addonmanager.kubernetes.io/mode: Reconcile
  name: system:coredns
rules:
- apiGroups:
  - ""
  resources:
  - endpoints
  - services
  - pods
  - namespaces
  verbs:
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
    addonmanager.kubernetes.io/mode: EnsureExists
  name: system:coredns
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:coredns
subjects:
- kind: ServiceAccount
  name: coredns
  namespace: kube-system

/data/k8s-yaml/coredns/configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns
  namespace: kube-system
data:
  Corefile: |
    .:53 {
        errors
        log
        health
        ready
        kubernetes cluster.local 192.168.0.0/16
        forward . 10.4.7.11
        cache 30
        loop
        reload
        loadbalance
       }

/data/k8s-yaml/coredns/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: coredns
  namespace: kube-system
  labels:
    k8s-app: coredns
    kubernetes.io/name: "CoreDNS"
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: coredns
  template:
    metadata:
      labels:
        k8s-app: coredns
    spec:
      priorityClassName: system-cluster-critical
      serviceAccountName: coredns
      containers:
      - name: coredns
        image: harbor.od.com/public/coredns:1.7.0
        args:
        - -conf
        - /etc/coredns/Corefile
        volumeMounts:
        - name: config-volume
          mountPath: /etc/coredns
        ports:
        - containerPort: 53
          name: dns
          protocol: UDP
        - containerPort: 53
          name: dns-tcp
          protocol: TCP
        - containerPort: 9153
          name: metrics
          protocol: TCP
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 60
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 5
      dnsPolicy: Default
      volumes:
        - name: config-volume
          configMap:
            name: coredns
            items:
            - key: Corefile
              path: Corefile

/data/k8s-yaml/coredns/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: coredns
  namespace: kube-system
  labels:
    k8s-app: coredns
    kubernetes.io/cluster-service: "true"
    kubernetes.io/name: "CoreDNS"
spec:
  selector:
    k8s-app: coredns
  clusterIP: 192.168.0.2
  ports:
  - name: dns
    port: 53
    protocol: UDP
  - name: dns-tcp
    port: 53
  - name: metrics
    port: 9153
    protocol: TCP

3.2.3 应用资源配置

kubectl apply -f http://k8s-yaml.od.com/coredns/rbac.yaml
kubectl apply -f http://k8s-yaml.od.com/coredns/configmap.yaml
kubectl apply -f http://k8s-yaml.od.com/coredns/development.yaml
kubectl apply -f http://k8s-yaml.od.com/coredns/service.yaml

3.2.4 检查创建的资源

kubectl get all -n kube-system -owide

3.2.5 验证coreDNS

dig -t A hdss7-21.host.com @192.168.0.2 +short
dig -t A hdss7-22.host.com @192.168.0.2 +short
dig -t A hdss7-23.host.com @192.168.0.2 +short
dig -t A hdss7-24.host.com @192.168.0.2 +short
dig -t A hdss7-11.host.com  @192.168.0.2 +short
dig -t A hdss7-12.host.com  @192.168.0.2 +short
dig -t A www.baidu.com  @192.168.0.2 +short

3.2.6 验证之前建的nginx的service ip是否能被解析

kubectl get all -n kube-public

dig -t A nginx-dp.kube-public.svc.cluster.local @192.168.0.2 +short

• 进入容器中， 可以使用简写

curl nginx-dp.kube-public