Kubernetes部署CoreDNS

概述

Coredns是用Go语言进行编写的开源的DNS服务软件，具有服务/转发功能。同时 CoreDNS 也是云原生计算基金会的毕业项目。
CoreDNS 是一个快速的灵活的 DNS 服务器。
官网：https://coredns.io/
Github： https://github.com/coredns/coredns
k8s自定义DNS：https://kubernetes.io/zh/docs/tasks/administer-cluster/dns-custom-nameservers/

功能

• 从文件中提供区域数据；支持 DNSSEC（仅限 NSEC）和 DNS（file和auto）。
• 从主服务器检索区域数据，即充当辅助服务器（仅限 AXFR）（辅助服务器）。
• 即时签署区域数据 ( dnssec )。
• 响应的负载平衡（loadbalance）。
• 允许区域传输，即充当主服务器（文件+传输）。
• 自动从磁盘加载区域文件（auto）。
• 缓存 DNS 响应（缓存）。
• 使用 etcd 作为后端（替换SkyDNS）（etcd）。
• 使用 k8s (kubernetes) 作为后端 ( kubernetes )。
• 作为代理将查询转发到其他（递归）名称服务器（forward）。
• 提供指标（通过使用 Prometheus）（prometheus）。
• 提供查询 ( log ) 和错误 ( errors ) 日志记录。
• 与云提供商 ( route53 ) 集成。
• 支持CH班：version.bind和朋友们（混沌）。
• 支持 RFC 5001 DNS 名称服务器标识符 (NSID) 选项 ( nsid )。
• 分析支持 ( pprof )。
• 重写查询（qtype、qclass 和 qname）（重写和模板）。
• 阻止任何查询（任何）。
• 提供 DNS64 IPv6 转换 ( dns64 )。

部署

可以在k8s官方源码中查看获取部署的 yaml。

获取 yaml

root@k8smaster-11:/data/k8s/soft# wget https://dl.k8s.io/v1.22.5/kubernetes.tar.gz
root@k8smaster-11:/data/k8s/soft# tar xf kubernetes.tar.gz
root@k8smaster-11:/data/k8s/soft# cd kubernetes/cluster/addons/dns/coredns/

# coredns 部署模板文件
root@k8smaster-11:/data/k8s/soft/kubernetes/cluster/addons/dns/coredns# ls
Makefile coredns.yaml.base coredns.yaml.in coredns.yaml.sed transforms2salt.sed transforms2sed.sed
root@k8smaster-11:/data/k8s/soft/kubernetes/cluster/addons/dns/coredns# cp coredns.yaml.base /data/k8s/yaml/coredns/coredns.yaml

修改yaml

1. 增加权限配置

在 kind: ClusterRole 最后增加如下
- apiGroups:
  - discovery.k8s.io
  resources:
  - endpointslices
  verbs:
  - list
  - watch

2. 修改域名后缀
   可以起一个 pod 进入查看/etc/resolv.conf 进行确定

修改前：kubernetes __DNS__DOMAIN__ in-addr.arpa ip6.arpa {ZZ
修改后：kubernetes cluster.local in-addr.arpa ip6.arpa {

3. 修改 forward dns
   这里的作用是 coreDNS 解析不了的域名转发给谁，可以设置为内网的DNS服务器。

修改前：forward . /etc/resolv.conf {
修改后：forward . 172.16.0.110 {

4. 修改镜像地址

修改前：image: k8s.gcr.io/coredns/coredns:v1.8.0
修改后：image: coredns/coredns:1.8.0

5. 内存限制
   推荐 2-5 Gb

修改前：memory: __DNS__MEMORY__LIMIT__
修改后：memory: 256Mi

6. 修改 dns CluseterIP

可以起一个 pod 进入查看`/etc/resolv.conf` 进行确定
修改前：clusterIP: __DNS__SERVER__
修改后：clusterIP: 10.100.0.2

部署

root@k8smaster-11:/data/k8s/yaml/coredns# kubectl apply -f coredns.yaml 
serviceaccount/coredns unchanged
clusterrole.rbac.authorization.k8s.io/system:coredns unchanged
clusterrolebinding.rbac.authorization.k8s.io/system:coredns unchanged
configmap/coredns unchanged
deployment.apps/coredns unchanged
service/kube-dns unchanged
root@k8smaster-11:/data/k8s/yaml/coredns# kubectl get pods -A
NAMESPACE     NAME                                       READY   STATUS    RESTARTS      AGE
default       net-test13                                 1/1     Running   0             60m
default       net-test3                                  1/1     Running   0             74m
kube-system   calico-kube-controllers-59df8b6856-hskk9   1/1     Running   0             75m
kube-system   calico-node-d4hrl                          1/1     Running   0             75m
kube-system   calico-node-fbb8t                          1/1     Running   2 (63m ago)   75m
kube-system   calico-node-hckcj                          1/1     Running   0             75m
kube-system   calico-node-z4tl6                          1/1     Running   0             75m
kube-system   coredns-7cd5f7d88c-z8vq6                   1/1     Running   0             69s

测试

root@k8smaster-11:/etc/kubeasz# kubectl run net-test23 --image=harbor.pgoops.com/base/alpine:v1 sleep 60000
pod/net-test23 created
root@k8smaster-11:/etc/kubeasz# kubectl exec -it pod/net-test23 sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/ # ping baidu.com
PING baidu.com (220.181.38.251): 56 data bytes
64 bytes from 220.181.38.251: seq=0 ttl=127 time=10.077 ms
64 bytes from 220.181.38.251: seq=1 ttl=127 time=9.565 ms
^C
--- baidu.com ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 9.565/9.821/10.077 ms
/ # ping harbor.pgoops.com
PING harbor.pgoops.com (172.16.0.180): 56 data bytes
64 bytes from 172.16.0.180: seq=0 ttl=63 time=0.565 ms
64 bytes from 172.16.0.180: seq=1 ttl=63 time=8.178 ms

修改后的 yaml 文件

root@k8smaster-11:/data/k8s/yaml/coredns# cat coredns.yaml 
# __MACHINE_GENERATED_WARNING__

apiVersion: v1
kind: ServiceAccount
metadata:
  name: coredns
  namespace: kube-system
  labels:
      kubernetes.io/cluster-service: "true"
      addonmanager.kubernetes.io/mode: Reconcile
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
    addonmanager.kubernetes.io/mode: Reconcile
  name: system:coredns
rules:
- apiGroups:
  - ""
  resources:
  - endpoints
  - services
  - pods
  - namespaces
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - get
- apiGroups:
  - discovery.k8s.io
  resources:
  - endpointslices
  verbs:
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  annotations:
    rbac.authorization.kubernetes.io/autoupdate: "true"
  labels:
    kubernetes.io/bootstrapping: rbac-defaults
    addonmanager.kubernetes.io/mode: EnsureExists
  name: system:coredns
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:coredns
subjects:
- kind: ServiceAccount
  name: coredns
  namespace: kube-system
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns
  namespace: kube-system
  labels:
      addonmanager.kubernetes.io/mode: EnsureExists
data:
  Corefile: |
    .:53 {
        errors
        health {
            lameduck 5s
        }
        ready
        kubernetes cluster.local in-addr.arpa ip6.arpa {
            pods insecure
            fallthrough in-addr.arpa ip6.arpa
            ttl 30
        }
        prometheus :9153
        # dns 无法解析域名 交给谁处理默认 /etc/resolv.conf
        forward . 172.16.0.110 {
            max_concurrent 1000
        }
        cache 30
        loop
        reload
        loadbalance
    }
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: coredns
  namespace: kube-system
  labels:
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
    kubernetes.io/name: "CoreDNS"
spec:
  # replicas: not specified here:
  # 1. In order to make Addon Manager do not reconcile this replicas parameter.
  # 2. Default is 1.
  # 3. Will be tuned in real time if DNS horizontal auto-scaling is turned on.
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
  selector:
    matchLabels:
      k8s-app: kube-dns
  template:
    metadata:
      labels:
        k8s-app: kube-dns
    spec:
      securityContext:
        seccompProfile:
          type: RuntimeDefault
      priorityClassName: system-cluster-critical
      serviceAccountName: coredns
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                  - key: k8s-app
                    operator: In
                    values: ["kube-dns"]
              topologyKey: kubernetes.io/hostname
      tolerations:
        - key: "CriticalAddonsOnly"
          operator: "Exists"
      nodeSelector:
        kubernetes.io/os: linux
      containers:
      - name: coredns
        #修改镜像地址 docker pull coredns/coredns:1.8.0 原地址coredns/coredns/coredns:v1.8.0
        image: coredns/coredns:1.8.0
        imagePullPolicy: IfNotPresent
        resources:
          limits:
            # 内存限制 推荐 2-5g
            memory: 256Mi
          requests:
            cpu: 100m
            memory: 70Mi
        args: [ "-conf", "/etc/coredns/Corefile" ]
        volumeMounts:
        - name: config-volume
          mountPath: /etc/coredns
          readOnly: true
        ports:
        - containerPort: 53
          name: dns
          protocol: UDP
        - containerPort: 53
          name: dns-tcp
          protocol: TCP
        - containerPort: 9153
          name: metrics
          protocol: TCP
        livenessProbe:
          httpGet:
            path: /health
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 60
          timeoutSeconds: 5
          successThreshold: 1
          failureThreshold: 5
        readinessProbe:
          httpGet:
            path: /ready
            port: 8181
            scheme: HTTP
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            add:
            - NET_BIND_SERVICE
            drop:
            - all
          readOnlyRootFilesystem: true
      dnsPolicy: Default
      volumes:
        - name: config-volume
          configMap:
            name: coredns
            items:
            - key: Corefile
              path: Corefile
---
apiVersion: v1
kind: Service
metadata:
  name: kube-dns
  namespace: kube-system
  annotations:
    prometheus.io/port: "9153"
    prometheus.io/scrape: "true"
  labels:
    k8s-app: kube-dns
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
    kubernetes.io/name: "CoreDNS"
spec:
  selector:
    k8s-app: kube-dns
    # dns service ip
  clusterIP: 10.100.0.2
  ports:
  - name: dns
    port: 53
    protocol: UDP
  - name: dns-tcp
    port: 53
    protocol: TCP
  - name: metrics
    port: 9153
    protocol: TCP