K8S服务发现
- 服务发现就是服务（应用）之间相互定位的过程。
- 服务发现不是非云计算时代独有的，传统的单体架构时代也会用到。以下应用场景下，更需要服务发现。
- 服务（应用）的动态性强
- 服务（应用）更新发布频繁
- 服务（应用）支持自动伸缩
- 在K8S集群里，POD的IP是不断变化的，如何以不变应万变？
- 抽象出了service资源，通过标签选择器，管理一组pod
- 抽象出了集群网络，通过相对固定的“集群IP”，使服务接入点固定
- 那么如何自动关联service资源的“名称”和“集群网络IP”，从而达到服务被集群自动发现的目的呢？
- 考虑传统DNS的模型：hdss7-21.host.com → 10.4.7.21
- 能否在K8S里建立这样的模型：Nginx-ds → 192.168.0.5
K8S里服务发现的方式——DNS
实现K8S里DNS功能的插件（软件）
- kube-dns——kubereters-v1.2至kubernetes-v1.10
- Coredns——kubernetes-v1.11——至今（取代了kube-dns）作为k8s默认的DNS插件
注意：
- K8S里的DNS不是万能的！它应该只负责自动维护“服务名”→ 集群网络IP之间的关系
K8S服务发现插件——CoreDNS
部署K8S的内网资源配置清单HTTP服务
在运维主机
hdss7-200.host.com上，配置一个Nginx虚拟主机，用以提供k8s统一的资源配置清单访问入口
- 配置Nginx
[root@hdss7-200 ~]# cd /etc/nginx/conf.d/
[root@hdss7-200 conf.d]# vim /etc/nginx/conf.d/k8s-yaml.od.com.conf
server {
listen 80;
server_name k8s-yaml.od.com;
location / {
autoindex on;
default_type text/plain;
root /data/k8s-yaml;
}
}
创建目录并检查
[root@hdss7-200 conf.d]# mkdir /data/k8s-yaml
[root@hdss7-200 conf.d]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[root@hdss7-200 conf.d]# nginx -s reload
[root@hdss7-200 conf.d]# cd /data/k8s-yaml/
在7-11上
[root@hdss7-11 ~]# vim /var/named/od.com.zone
修改如下
2020080103; serial
k8s-yaml A 10.4.7.200
重启named服务，检查服务
[root@hdss7-11 ~]# systemctl restart named
[root@hdss7-11 ~]# dig -t A k8s-yaml.od.com @10.4.7.11 +short
10.4.7.200
回到7-200主机，创建coredns目录
[root@hdss7-200 k8s-yaml]# mkdir coredns
下载coredns镜像
[root@hdss7-200 conf.d]# docker pull docker.io/coredns/coredns:1.6.1
[root@hdss7-200 conf.d]# docker images | grep coredns
coredns/coredns 1.6.1 c0f6e815079e 12 months ago 42.2MB
[root@hdss7-200 conf.d]# docker tag c0f6e815079e harbor.od.com/public/coredns:v1.6.1
[root@hdss7-200 conf.d]# docker push !$
docker push harbor.od.com/public/coredns:v1.6.1
The push refers to repository [harbor.od.com/public/coredns]
da1ec456edc8: Pushed
225df95e717c: Pushed
v1.6.1: digest: sha256:c7bf0ce4123212c87db74050d4cbab77d8f7e0b49c041e894a35ef15827cf938 size: 739
需要四个资源配置清单
- rbac.yaml
- cm.yaml
- dp.yaml
- svc.yaml
rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: coredns
namespace: kube-system
labels:
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
kubernetes.io/bootstrapping: rbac-defaults
addonmanager.kubernetes.io/mode: Reconcile
name: system:coredns
rules:
- apiGroups:
- ""
resources:
- endpoints
- services
- pods
- namespaces
verbs:
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
kubernetes.io/bootstrapping: rbac-defaults
addonmanager.kubernetes.io/mode: EnsureExists
name: system:coredns
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:coredns
subjects:
- kind: ServiceAccount
name: coredns
namespace: kube-system
cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: coredns
namespace: kube-system
data:
Corefile: |
.:53 {
errors
log
health
ready
kubernetes cluster.local 192.168.0.0/16
forward . 10.4.7.11
cache 30
loop
reload
loadbalance
}
dp.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: coredns
namespace: kube-system
labels:
k8s-app: coredns
kubernetes.io/name: "CoreDNS"
spec:
replicas: 1
selector:
matchLabels:
k8s-app: coredns
template:
metadata:
labels:
k8s-app: coredns
spec:
priorityClassName: system-cluster-critical
serviceAccountName: coredns
containers:
- name: coredns
image: harbor.od.com/public/coredns:v1.6.1
args:
- -conf
- /etc/coredns/Corefile
volumeMounts:
- name: config-volume
mountPath: /etc/coredns
ports:
- containerPort: 53
name: dns
protocol: UDP
- containerPort: 53
name: dns-tcp
protocol: TCP
- containerPort: 9153
name: metrics
protocol: TCP
livenessProbe:
httpGet:
path: /health
port: 8080
scheme: HTTP
initialDelaySeconds: 60
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 5
dnsPolicy: Default
volumes:
- name: config-volume
configMap:
name: coredns
items:
- key: Corefile
path: Corefile
svc.yaml
apiVersion: v1
kind: Service
metadata:
name: coredns
namespace: kube-system
labels:
k8s-app: coredns
kubernetes.io/cluster-service: "true"
kubernetes.io/name: "CoreDNS"
spec:
selector:
k8s-app: coredns
clusterIP: 192.168.0.2
ports:
- name: dns
port: 53
protocol: UDP
- name: dns-tcp
port: 53
- name: metrics
port: 9153
protocol: TCP
在7-21上操作
用陈述式资源管理方法生成声明式资源管理配置清单
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/coredns/rbac.yaml
serviceaccount/coredns created
clusterrole.rbac.authorization.k8s.io/system:coredns created
clusterrolebinding.rbac.authorization.k8s.io/system:coredns created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/coredns/cm.yaml
configmap/coredns created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/coredns/dp.yaml
deployment.apps/coredns created
[root@hdss7-21 ~]# kubectl apply -f http://k8s-yaml.od.com/coredns/svc.yaml
service/coredns created
查看POD资源
[root@hdss7-21 ~]# kubectl get all -n kube-system
NAME READY STATUS RESTARTS AGE
pod/coredns-6b6c4f9648-ttfg8 1/1 Running 0 2m3s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/coredns ClusterIP 192.168.0.2 <none> 53/UDP,53/TCP,9153/TCP 105s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/coredns 1/1 1 1 2m3s
NAME DESIRED CURRENT READY AGE
replicaset.apps/coredns-6b6c4f9648 1 1 1 2m3s
查看sh文件
[root@hdss7-21 ~]# cat /opt/kubernetes/server/bin/kubelet.sh
#!/bin/sh
./kubelet
..........
--cluster-dns 192.168.0.2
..........
可以看到集群id已经固定了，是192.168.0.2这个IP，这是这个集群DNS统一接入的点
[root@hdss7-21 ~]# dig -t A www.baidu.com @192.168.0.2 +short
www.a.shifen.com.
180.101.49.12
180.101.49.11
[root@hdss7-21 ~]# dig -t A hdss7-21.host.com @192.168.0.2 +short
10.4.7.21
因为我们在cm里面已经指定了forward是10.4.7.11，我们自建的dns是coredns的上级dns，
[root@hdss7-21 ~]# kubectl get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes ClusterIP 192.168.0.1 <none> 443/TCP 18d <none>
[root@hdss7-21 ~]# kubectl create deployment nginx-dp --image=harbor.od.com/public/nginx:v1.7.9 -n kube-public
deployment.apps/nginx-dp created
[root@hdss7-21 ~]# kubectl expose deployment nginx-dp --port=80 -n kube-public
service/nginx-dp exposed
[root@hdss7-21 ~]# kubectl get pods -n kube-public
NAME READY STATUS RESTARTS AGE
nginx-dp-5dfc689474-vjpp8 1/1 Running 0 9s
[root@hdss7-21 ~]# kubectl get svc -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes ClusterIP 192.168.0.1 <none> 443/TCP 19d <none>
[root@hdss7-21 ~]# kubectl get svc -n kube-public
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx-dp ClusterIP 192.168.188.157 <none> 80/TCP 3m52s
[root@hdss7-21 ~]# dig -t A nginx-dp @192.168.0.2 +short
没有返回！
这里想通过coredns去查询service的名称，需要用fqdn，需要加上
[root@hdss7-21 ~]# dig -t A nginx-dp.kube-public.svc.cluster.local. @192.168.0.2 +short
192.168.188.157
[root@hdss7-21 ~]# kubectl get pod -n kube-public -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-dp-5dfc689474-vjpp8 1/1 Running 0 10m 172.7.22.3 hdss7-22.host.com <none> <none>
进入容器
[root@hdss7-21 ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-ds-gwswr 1/1 Running 0 4h55m 172.7.22.2 hdss7-22.host.com <none> <none>
nginx-ds-jh2x5 1/1 Running 0 4h55m 172.7.21.2 hdss7-21.host.com <none> <none>
[root@hdss7-21 ~]# kubectl exec -it nginx-ds-jh2x5 /bin/bash
root@nginx-ds-jh2x5:/# curl 192.168.188.157
<!DOCTYPE html>
...........
通过名称空间访问
root@nginx-ds-jh2x5:/# curl nginx-dp.kube-public.svc.cluster.local
<!DOCTYPE html>
...........
因为一个default，就是默认的名称空间，一个是public名称空间，在集群里面，curl service的时候可以简写成nginx-dp.kube-public
root@nginx-ds-jh2x5:/# curl nginx-dp.kube-public
<!DOCTYPE html>
...........
这是为什么呢？查看一下之前配好的一个配置文件。
root@nginx-ds-jh2x5:/# cat /etc/resolv.conf
nameserver 192.168.0.2
search default.svc.cluster.local svc.cluster.local cluster.local host.com
options ndots:5
可以看到，这里已经将default.svc.cluster.local，svc.cluster.local cluster.local都加到svc这个域里了。
把coredns安装到k8s集群里，最重要的一点就是把service的名字和cluster做一个自动关联。做自动关联就实现了所谓的服务发现。（只需要记service的名字）