Kubernetes入门（五）

部署Harbor

Harbor是由VMWare中国团队开源的容器镜像仓库。事实上，harbor是在Docker Registry上进行了相应的企业级扩展，从而获得了更加广泛的应用，这些新的企业级特性包括：管理用户界面，基于角色的访问控制，水平扩展，同步，AD/LDAP集成以及日志审计等。

github地址：https://github.com/goharbor/harbor/releases

手动部署

之前已经搭建了kubernetes集群，其中192.168.30.150作为master2节点，我们把它作为harbor服务器。

下载最新的docker-compose二进制文件：

[root@master2 ~]# vim /etc/hosts                    #添加一行
54.231.48.160 github-production-release-asset-2e65be.s3.amazonaws.com

[root@master2 ~]# curl -L https://github.com/docker/compose/releases/download/1.24.0-rc1/docker-compose-`uname -s`-`uname -m` -o /usr/bin/docker-compose

[root@master2 ~]# chmod +x /usr/bin/docker-compose

下载harbor离线安装包：

[root@master2 ~]# wget https://storage.googleapis.com/harbor-releases/release-1.7.0/harbor-offline-installer-v1.7.0-rc2.tgz

[root@master2 ~]# tar zxf harbor-offline-installer-v1.7.0-rc2.tgz

准备CA证书

[root@master2 ~]# mkdir /data/ && cd /data

上传证书 harbor.lzxlinux.com.key.tar.gz                #自制证书

[root@master2 data]# tar zxf harbor.lzxlinux.com.key.tar.gz

[root@master2 data]# ls cert/
ca.crt  server.crt  server.key
修改配置文件：

[root@master2 cert]# cd harbor/

[root@master2 harbor]# vim harbor.cfg                   #做下面修改
hostname = harbor.lzxlinux.com
ui_url_protocol = https

执行安装：

[root@master2 harbor]# sh install.sh

[root@master2 harbor]# docker ps
CONTAINER ID        IMAGE                                    COMMAND                  CREATED             STATUS                   PORTS                                                              NAMES
5cad50fe8189        goharbor/nginx-photon:v1.7.0             "nginx -g 'daemon of…"   5 minutes ago       Up 5 minutes (healthy)   0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:4443->4443/tcp   nginx
7f536f3285ff        goharbor/harbor-jobservice:v1.7.0        "/harbor/start.sh"       5 minutes ago       Up 5 minutes                                                                                harbor-jobservice
f4c2f03e9315        goharbor/harbor-portal:v1.7.0            "nginx -g 'daemon of…"   5 minutes ago       Up 5 minutes (healthy)   80/tcp                                                             harbor-portal
45755eaf06e3        goharbor/harbor-core:v1.7.0              "/harbor/start.sh"       5 minutes ago       Up 5 minutes (healthy)                                                                      harbor-core
d1b57df6021c        goharbor/harbor-adminserver:v1.7.0       "/harbor/start.sh"       5 minutes ago       Up 5 minutes (healthy)                                                                      harbor-adminserver
f65f40cac8cc        goharbor/harbor-db:v1.7.0                "/entrypoint.sh post…"   5 minutes ago       Up 5 minutes (healthy)   5432/tcp                                                           harbor-db
dd50b19871b8        goharbor/harbor-registryctl:v1.7.0       "/harbor/start.sh"       5 minutes ago       Up 5 minutes (healthy)                                                                      registryctl
034de27d8a84        goharbor/registry-photon:v2.6.2-v1.7.0   "/entrypoint.sh /etc…"   5 minutes ago       Up 5 minutes (healthy)   5000/tcp                                                           registry
dd0c1abf428c        goharbor/redis-photon:v1.7.0             "docker-entrypoint.s…"   5 minutes ago       Up 5 minutes             6379/tcp                                                           redis
97530153c3b7        goharbor/harbor-log:v1.7.0               "/bin/sh -c /usr/loc…"   5 minutes ago       Up 5 minutes (healthy)   127.0.0.1:1514->10514/tcp                                          harbor-log
0baf1dbfb8d0        ff281650a721                             "/opt/bin/flanneld -…"   2 hours ago         Up 2 hours                                                                                  k8s_kube-flannel_kube-flannel-ds-amd64-lfgv5_kube-system_f40a3fe8-34d6-11e9-83ed-000c297ff3a2_0
96ce131643a0        mirrorgooglecontainers/pause-amd64:3.1   "/pause"                 2 hours ago         Up 2 hours                                                                                  k8s_POD_kube-flannel-ds-amd64-lfgv5_kube-system_f40a3fe8-34d6-11e9-83ed-000c297ff3a2_0

浏览器访问：

编辑Windows上的hosts文件，路径：C:\Windows\System32\drivers\etc\hosts，增加一行：192.168.30.150 harbor.lzxlinux.com 。

用户名：admin ，密码：Harbor12345（默认密码）

 新建项目：

一般项目不设为公开

 拉取公共镜像:

[root@master2 harbor]# docker pull busybox

[root@master2 harbor]# docker pull tomcat

打标签：

[root@master2 harbor]# docker tag busybox harbor.lzxlinux.com/lzx/busybox

[root@master2 harbor]# echo 192.168.30.150 harbor.lzxlinux.com >> /etc/hosts

[root@master2 harbor]# docker login https://harbor.lzxlinux.com
Username: admin
Password:
Error response from daemon: Get https://harbor.lzxlinux.com/v2/: x509: certificate signed by unknown authority

解决509问题：

[root@master2 harbor]# echo -n | openssl s_client -showcerts -connect harbor.lzxlinux.com:443 2>/dev/null |sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/P' >> /etc/ssl/certs/ca-bundle.trust.crt

[root@master2 harbor]# systemctl restart docker

[root@master2 harbor]# docker-compose up -d                 # -d 后台启动

如果上面方法还是不行，可以这样操作

# mkdir -p /etc/docker/certs.d/harbor.lzxlinux.com

# cp /data/cert/ca.crt /etc/docker/certs.d/harbor.lzxlinux.com/ca.crt

# chmod 400 !$

# systemctl restart docker

再次登录：

[root@master2 harbor]# docker login https://harbor.lzxlinux.com
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
推送busybox镜像：

[root@master2 harbor]# docker push harbor.lzxlinux.com/lzx/busybox
The push refers to repository [harbor.lzxlinux.com/lzx/busybox]
683f499823be: Pushed
latest: digest: sha256:bbb143159af9eabdf45511fd5aab4fd2475d4c0e7fd4a5e154b98e838488e510 size: 527
浏览器查看：

刷新一下，可以查看到新传上来的镜像。

推送tomcat镜像：

[root@master2 harbor]# docker tag tomcat harbor.lzxlinux.com/lzx/tomcat

[root@master2 harbor]# docker push harbor.lzxlinux.com/lzx/tomcat

再次刷新查看

在Kubernetes中使用harbor

废了很大劲部署harbor，那它有什么作用呢？接下来在kubernetes中使用它。

---

以下操作在master节点中执行。

创建secret：

[root@master ~]# kubectl create secret docker-registry my-secret --docker-server=harbor.lzxlinux.com --docker-username=admin --docker-password=Harbor12345

[root@master ~]# kubectl get secret
NAME                  TYPE                                  DATA      AGE
default-token-lfx98   kubernetes.io/service-account-token   3         1d
my-secret             kubernetes.io/dockerconfigjson        1         38s                   # secret可创建多个，在拉取镜像时指定

推送httpd镜像：

[root@master ~]# docker pull httpd

[root@master ~]# docker tag httpd harbor.lzxlinux.com/lzx/httpd

[root@master ~]# docker login https://harbor.lzxlinux.com

[root@master ~]# docker push harbor.lzxlinux.com/lzx/httpd

刷新查看

 定义一个pod：

[root@master ~]# vim httpd-pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: httpd-pod
spec:
  containers:
  - image: harbor.lzxlinux.com/lzx/httpd:latest
    name: httpd-pod
  imagePullSecrets:
  - name: my-secret                     #引用my-secret

[root@master ~]# kubectl create -f httpd-pod.yaml

[root@master ~]# kubectl get pod
NAME                     READY     STATUS             RESTARTS   AGE
httpd-pod                0/1       ImagePullBackOff   0          53s                #状态不对
nginx-6f858d4d45-wwnm9   1/1       Running            0          4h

[root@master ~]# kubectl describe pod httpd-pod
  Warning  Failed     25s (x2 over 53s)  kubelet, 192.168.30.130  Error: ErrImagePull
  Normal   BackOff    11s (x2 over 52s)  kubelet, 192.168.30.130  Back-off pulling image "harbor.lzxlinux.com/lzx/httpd:latest"
  Warning  Failed     11s (x2 over 52s)  kubelet, 192.168.30.130  Error: ImagePullBackOff
  Normal   Pulling    0s (x3 over 1m)    kubelet, 192.168.30.130  pulling image "harbor.lzxlinux.com/lzx/httpd:latest"

解决问题：

[root@master2 harbor]# scp /data/cert/ca.crt 192.168.30.129:/root/

[root@master2 harbor]# scp /data/cert/ca.crt 192.168.30.130:/root/

[root@node1 ~]# mkdir -p /etc/docker/certs.d/harbor.lzxlinux.com

[root@node1 ~]# cp ca.crt /etc/docker/certs.d/harbor.lzxlinux.com/ca.crt

[root@node1 ~]# chmod 400 !$

[root@node1 ~]# systemctl restart docker

[root@node1 ~]# vim /etc/hosts                  #增加一行
192.168.30.150 harbor.lzxlinux.com

[root@node2 ~]# mkdir -p /etc/docker/certs.d/harbor.lzxlinux.com

[root@node2 ~]# cp ca.crt /etc/docker/certs.d/harbor.lzxlinux.com/ca.crt

[root@node2 ~]# chmod 400 !$

[root@node2 ~]# systemctl restart docker

[root@node2 ~]# vim /etc/hosts                  #增加一行
192.168.30.150 harbor.lzxlinux.com

继续创建pod：

[root@master ~]# kubectl delete pod httpd-pod

[root@master ~]# kubectl create -f httpd-pod.yaml

[root@master ~]# kubectl describe pod httpd-pod
  Normal  Scheduled  <invalid>  default-scheduler        Successfully assigned default/httpd-pod to 192.168.30.130
  Normal  Pulling    <invalid>  kubelet, 192.168.30.130  pulling image "harbor.lzxlinux.com/lzx/httpd:latest"
  Normal  Pulled     <invalid>  kubelet, 192.168.30.130  Successfully pulled image "harbor.lzxlinux.com/lzx/httpd:latest"
  Normal  Created    <invalid>  kubelet, 192.168.30.130  Created container
  Normal  Started    <invalid>  kubelet, 192.168.30.130  Started container
 
[root@master ~]# kubectl get pod
NAME                     READY     STATUS             RESTARTS   AGE
httpd-pod                1/1       Running            0          3m
nginx-6f858d4d45-wwnm9   1/1       Running            0          4h

这次成功创建，正在运行中。刷新浏览器，可以看到下载次数已更新。

免费证书申请：https://freessl.cn/

harbor443: connect: connection refused问题解决：https://blog.csdn.net/miss1181248983/article/details/88661825