部署一个简单的 demo 发现一个很奇妙的错误,yaml 文件内容在下方,镜像仓库是内网自建的 harbor-1.9.1 单节点仓库,测试就没有整高可用了。
用户名:admin
密码:admin123
[root@k8s-m1 ~]# cat /opt/work/jenkins/jenkins_home/workspace/k8s-web-demo/web.yaml
#deploy apiVersion: apps/v1 kind: Deployment metadata: name: k8s-web-demo spec: selector: matchLabels: app: k8s-web-demo replicas: 1 template: metadata: labels: app: k8s-web-demo spec: containers: - name: k8s-web-demo image: harbor.unchch.com/kubernetes/k8s-web-demo:20191031_105142_3551 ports: - containerPort: 8080 --- #service apiVersion: v1 kind: Service metadata: name: k8s-web-demo spec: ports: - port: 80 protocol: TCP targetPort: 8080 selector: app: k8s-web-demo type: ClusterIP --- #ingress apiVersion: extensions/v1beta1 kind: Ingress metadata: name: k8s-web-demo spec: rules: - host: k8s-web.unchch.com http: paths: - path: / backend: serviceName: k8s-web-demo servicePort: 80
检查 pods,发现报错:
继续查看错误信息
# kubectl describe po k8s-web-demo-675df649c7-s2scg
报错内容是:
pulling image "harbor.unchch.com/kubernetes/k8s-web-demo:20191031_105142_3551" Warning Failed 36s (x3 over 71s) kubelet, k8s-n1 Failed to pull image "harbor.unchch.com/kubernetes/k8s-web-demo:20191031_105142_3551": rpc error: code = Unknown desc = Error response from daemon: pull access denied for harbor.unchch.com/kubernetes/k8s-web-demo, repository does not exist or may require 'docker login': denied: requested access to the resource is denied Warning Failed 36s (x3 over 71s) kubelet, k8s-n1 Error: ErrImagePull Normal BackOff 10s (x4 over 70s) kubelet, k8s-n1 Back-off pulling image "harbor.unchch.com/kubernetes/k8s-web-demo:20191031_105142_3551" Warning Failed 10s (x4 over 70s) kubelet, k8s-n1 Error: ImagePullBackOff
错误提示 “repository does not exist or may require 'docker login': denied: requested access to the resource is denied”,理解为“镜像不存在或者需要 docker login 拿到 pull 权限”。
但是在对应的 node 上执行 docker pull harbor.unchch.com/kubernetes/k8s-web-demo:20191031_105142_3551 是可以获取到镜像的,这个就尴尬了
排查过程:
1、去对应的 worker 节点手动 pull 该镜像,是 ok 的,说明之前的 docker login 的用户名和密码是正确的,排除镜像不存在和用户名密码输错的可能;
2、检查 /root/.docker/config 文件中的用户名密码是不是有问题,解密之后也排除了:
3、测试了 docker push 等操作,排除了 harbor 仓库的功能出错等问题;
4、最后想到是不是 kubectl apply -f web.yaml 时,不能正确的获取 docker login 的认证信息,基本就这一种解释了,于是,手工创建了一个secret,使用 secret 来指定 pull 权限:
(1)创建 secret,此处在 harbor 上新建了一个用户 manager:Harbor123
kubectl create secret docker-registry registry-harbor --namespace=default --docker-server=harbor.unchch.com --docker-username=manager --docker-password=Harbor123 --docker-email=unchch.xt@gmail.com
(2)查看创建的 secret
[root@k8s-m1 script]# kubectl get secrets NAME TYPE DATA AGE default-token-rqf8z kubernetes.io/service-account-token 3 9d registry-harbor kubernetes.io/dockerconfigjson 1 3h24m [root@k8s-m1 script]# kubectl describe secrets registry-harbor Name: registry-harbor Namespace: default Labels: <none> Annotations: <none> Type: kubernetes.io/dockerconfigjson Data ==== .dockerconfigjson: 141 bytes
(3)在 yaml 文件对应的部分指定secret:
再次 部署,一切正常。