kubernetes环境下私有仓库搭建

前期在客户那里搭建了基本运行环境，鉴于很多企业的环境都是内部网无法连接外部，因此搭建私有仓库是逃避不开的问题，按照网上的步骤搭建，虽然遇到一些问题，但还好都算容易解决了，下面大致把步骤记录一下便于下次去客户那里更新。

首先在需要在启动registry Pod的机器上把registry images获取下来

docker pull registry

然后在生成几个构建persistence volumn(pv), persistence volumn claim(pvc),以及registry rc和service的文件

[root@k8s-master registry]# cat pv.yaml 
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv01
  release: stable
spec:
  capacity:
    storage: 20Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Recycle
  nfs:
    path: /k8s/test
    server: 10.182.168.99

[root@k8s-master registry]# cat pvc.yaml 
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: myclaim2
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 20G

[root@k8s-master registry]# cat registry-rc.yaml 
apiVersion: v1
kind: ReplicationController
metadata:
  name: registry
  labels:
    name: registry
spec:
  replicas: 1
  selector:
    name: registry
  template:
    metadata:
      labels:
        name: registry
    spec:
      containers:
      - name: registry
        image: registry
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 5000
        volumeMounts:
        - mountPath: "/var/lib/registry"
          name: mypd
      volumes:
      - name: mypd
        persistentVolumeClaim:
          claimName: myclaim2

[root@k8s-master registry]# cat registry-srv.yaml 
apiVersion: v1
kind: Service
metadata:
    name: registry
    labels:
      name: registry
spec:
    type: NodePort
    ports:
    - port: 5000
      nodePort: 30002
    selector:
      name: registry

10.182.168.99是k8s-node-1的地址,非flannel集群地址

一个一个建立起来，当然需要在k8s-node-1上打个标签

kubectl label node k8s-node-1 name=registry

在建立registry-rc的时候遇到一些问题。

registry pod总是处于container creating的状态。

• 首先需要在启动registry pod的机器上创建相关的目录，我的是在k8s-node-1上创建/k8s/test

通过describe pods 一看，基本都是nfs mount的问题，解决方式如下：

• 启动nfs service

systemctl start nfs

• 遇到

Output: mount.nfs: access denied by server while mounting 10.182.168.99:/k8s/test错误时,修改配置文件/etc/exports，加入 insecure 选项

/k8s/test  *(insecure,rw,async,no_root_squash)

启动完成，一切顺利，

[root@k8s-master registry]# kubectl get pods
NAME                       READY     STATUS    RESTARTS   AGE
helloworld-service-62wl1   1/1       Running   6          88d
helloworld-service-8cbt2   1/1       Running   6          88d
registry-7nj8q             1/1       Running   2          1h
[root@k8s-master registry]# kubectl get services
NAME            CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
helloworldsvc   10.254.247.84   <nodes>       7001:30001/TCP   88d
kubernetes      10.254.0.1      <none>        443/TCP          120d
registry        10.254.174.54   <nodes>       5000:30002/TCP   1h

=====================================================================

接下来验证。

在k8s-node-1上有一大堆的本地images

[root@k8s-node-1 v2]# docker images
REPOSITORY                                             TAG                      IMAGE ID            CREATED             SIZE
docker.io/registry                                     latest                   3ebefe7c539b        5 days ago          33.19 MB
1213-domain                                            v2                       326bf14bb29f        3 months ago        2.055 GB
oracle/coherence                                       12.2.1.0.0-cacheserver   57a90e86e1d2        3 months ago        625 MB
oracle/coherence                                       12.2.1.0.0-proxy         238c85d61468        3 months ago        625 MB
gcr.io/google_containers/nginx-ingress-controller      0.9.0-beta.7             2c3d45bb8cb9        3 months ago        130.6 MB
gcr.io/google_containers/k8s-dns-sidecar-amd64         1.14.2                   7c4034e4ffa4        4 months ago        44.5 MB
gcr.io/google_containers/k8s-dns-kube-dns-amd64        1.14.2                   ca8759c215c9        4 months ago        52.36 MB
gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64   1.14.2                   e5c335701995        4 months ago        44.84 MB
1213-helloworld                                        v1                       351691157b77        4 months ago        2.064 GB

找一个小的，然后重新tag一下,这里我认为应该指向node的外部地址和端口而不是集群地址

docker tag docker.io/nginx  k8s-node-1:30002/nginx

然后修改/etc/sysconfig/docker文件，主要是修改两行（在需要用到registry的节点上都需要修改)

ADD_REGISTRY='--add-registry k8s-node-1'

INSECURE_REGISTRY='--insecure-registry k8s-node-1:30002'

然后重启docker

service docker restart

将images push到本地的registry,然后我们查看/k8s/test目录下就有内容了.

docker push k8s-node-1:30002/nginx

在其他机器运行docker pull,看到已经从本地拉回images.

[root@k8s-master registry]# docker pull k8s-node-1:30002/nginx
Using default tag: latest
Trying to pull repository k8s-node-1:30002/nginx ... 
sha256:c15f1fb8fd55c60c72f940a76da76a5fccce2fefa0dd9b17967b9e40b0355316: Pulling from k8s-node-1:30002/nginx
36a46ebd5019: Pull complete 
57168433389f: Pull complete 
332ec8285c50: Pull complete 
Digest: sha256:c15f1fb8fd55c60c72f940a76da76a5fccce2fefa0dd9b17967b9e40b0355316
Status: Downloaded newer image for k8s-node-1:30002/nginx:latest
[root@k8s-master registry]# docker images
REPOSITORY               TAG                 IMAGE ID            CREATED             SIZE
k8s-node-1:30002/nginx   latest              46102226f2fd        4 months ago        109.4 MB