小白学k8s(9)-gitlab-runner实现go项目的自动化发布

• gitlab构建CI/CD
  • 准备
  • docker部署gitlab
  • 使用二进制部署gitlab-runner
  • gitlab-runner注册
  • 配置Variables
  • 简单先来个测试
  • 开始构建
  • 遇到的报错

gitlab构建CI/CD

准备

docker部署gitlab

通过docker-compose启动gitlab

version: '3'
services:
  gitlab:
    image: 'gitlab/gitlab-ce:latest'
    restart: always
    hostname: '1.1.1.1'
    environment:
      TZ: 'Asia/Shanghai'
      GITLAB_OMNIBUS_CONFIG: |
        external_url 'http://1.1.1.1:9001'
        gitlab_rails['gitlab_shell_ssh_port'] = 1022
        unicorn['port'] = 8888
        nginx['listen_port'] = 9001
    ports:
      - '9001:9001'
      - '443:443'
      - '1022:22'
    volumes:
      - ./config:/etc/gitlab
      - ./data:/var/opt/gitlab
      - ./losg:/var/log/gitlab

使用二进制部署gitlab-runner

可参考官方的安装方式Install GitLab Runner manually on GNU/Linux

gitlab-runner注册

安装完成之后使用注册命令注册

$ gitlab-runner register

然后会提示输入gitlab的地址以及token信息

地址信息和token我们在下面可以看到

根据提示输入信息,需要注意的是里面的tags就是我们编写.gitlab-ci.yml对应填写的tag

之后在gitlab的runner中就可以看到我们注册的gitlab-runner了

配置Variables

在gitlab中可以配置我们gitlab-runner需要的变量，比如我们的docker-hub的密码，gitlab的账号密码等信息

简单先来个测试

先来个简单的gitlab-ci.yml测试下

stages:
  - test
  - build

variables:
  GOPROXY: https://goproxy.cn

lint:
  stage: test
  script:
    - echo "hello world lint"
  only:
    - branches
  tags:
    - golang-runner

test:
  stage: test
  script:
    - echo "hello world test"
  only:
    - branches
  cache:
    key: "bazel"
    paths:
      - .cache
  tags:
    - golang-runner

开始构建

通过helm和bazel实现在gitlab-runner中k8s应用的自动编译，发布。

镜像推送到docker-hub中,gitlab-runner中的helm需要配置好，这里我是用了helm默认初始化的charts结构来发布应用

gitlab-ci.yml

stages:
  - test
  - build
  - deploy

variables:
  GOPROXY: https://goproxy.cn

lint:
  stage: test
  script:
    - export GO_PROJECT_PATH="/home/gitlab-runner/goWork/src"
    - mkdir -p $GO_PROJECT_PATH
    - ln -s $(pwd) $GO_PROJECT_PATH/test
    - cd $GO_PROJECT_PATH/test
    - bash build/lint.sh
  only:
    - branches
  tags:
    - golang-runner

test:
  stage: test
  script:
    - go mod vendor
    - bash build/bazel-test.sh
  only:
    - branches
  cache:
    key: "bazel"
    paths:
      - .cache
  tags:
    - golang-runner

build:
  stage: build
  before_script:
    - url_host=`git remote get-url origin | sed -e "s/http://gitlab-ci-token:.*@//g"`
    - git remote set-url origin "http://$GIT_ACCESS_USER:$GIT_ACCESS_PASSWORD@${url_host}"
    - git config user.name $GIT_ACCESS_USER
    - git config user.email $GIT_ACCESS_EMAIL
    - git fetch --tags --force
  script:
    - docker login -u $DOCKER_ACCESS_USER -p $DOCKER_ACCESS_PASSWORD
    - go mod vendor
    - bash build/bazel-build.sh
  only:
    - master
  cache:
    key: "bazel"
    paths:
      - .cache
  tags:
    - golang-runner



include: '/.gitlab/deploy.yaml'

镜像打包

test::util:build_docker_images() {
  local docker_registry=$1
  local docker_tag=$2
  local base_image="alpine:3.7"

  query=$(test::util::find_changes)

  if [ "$query" == "" ]; then
    test::util::log "no change and exit..."
    exit 0
  fi


  for b in ${query}; do
    b=${b////src/"/src"}

    if [[ $b == *test* ]]
        then
        continue
    fi

    local binary_file_path=$(test::util::find_binary "$b")
    local binary_name=$(test::util::get_binary_name "$b")
    local docker_build_path="dockerbuild/${binary_name}"
    local docker_file_path="${docker_build_path}/Dockerfile"
    local docker_image_tag="${docker_registry}/${binary_name}:${docker_tag}"


    test::util::log "Starting docker build for image: ${binary_name}"
    (
      rm -rf "${docker_build_path}"
      mkdir -p "${docker_build_path}"
      cp "${binary_file_path}" "${docker_build_path}/${binary_name}"
      cat <<EOF >"${docker_file_path}"
FROM ${base_image}
COPY ${binary_name} /usr/local/bin/${binary_name}
RUN mkdir /lib64 && ln -s /lib/libc.musl-x86_64.so.1 /lib64/ld-linux-x86-64.so.2
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories 
  && apk update --no-cache 
ENTRYPOINT ["/usr/local/bin/${binary_name}"]
EOF
      docker build -q -t "${docker_image_tag}" "${docker_build_path}"
      docker push ${docker_image_tag}
    )


  cat <<EOF >>".gitlab/deploy.yaml"
${binary_name}:
  stage: deploy
  script:
    - bash build/deploy.sh ${docker_registry} ${binary_name} ${docker_tag}
  only:
    - tags
  when: manual
  environment:
    name: test
  tags:
    - golang-runner
EOF

done

  test::util::log "Docker builds done"
}

整体的处理思路是

1、通过bazel构建go项目。

2、构建的时候找到有改动的项目，编译，打包镜像，生成deploy脚本。

3、打上tag，推到gitlab中。

4、最后通过手动触发项目的deploy，通过helm发布对应的项目到k8s中。

项目的地址gitlab-runner构建go项目

最后附上deploy的截图

对应的项目已经部署上去了

$ kubectl get pods -n test
NAME                               READY   STATUS    RESTARTS   AGE
demo1-main-test-654fbd4df6-vlhwr   1/1     Running   0          27m
demo2-main-test-58ccfd4f8-hjcdf    1/1     Running   0          25m
demo3-main-test-6897ff9496-mhm5n   1/1     Running   0          27m

遇到的报错

go: writing go.mod cache: mkdir /home/goWork: permission denied

给用户gitlab-runner添加最用目录的执行权限

sudo chown -R $(whoami):gitlab-runner /Users/zhushuyan/go/pkg && sudo chmod -R g+rwx /Users/zhushuyan/go/pkg

需要预备的知识点

这里主要使用到了helm和bazel

helm使用

bazel使用