Kubernetes——使用Ingress发布Java服务(以Jenkins为例）

使用Ingress发布Java服务（以Jenkins为例）

  假设有这样一套环境：Kubernetes 集群上的 java-deploy 控制器生成了两个运行于 Pod 资源中的 java 实例，java-svc 是将它们统一暴露于集群中的访问入口。现在需要通过 Ingress 资源将 java-svc 发布给集群外部的客户端访问。

  规划图如下：

一、准备名称空间

  假设本示例中创建的所有资源都位于新建的 java-testing 名称空间中，与其他的资源逻辑上进行隔离，便于管理。

  下面的配置信息保存于 java-testing-namespaces.yaml 资源清单文件中： 

kind: Namespace
apiVersion: v1
metadata:
  name: java-testing
  labels:
    env: java-testing

  而后运行创建命令完成资源的额创建，并确认资源的存在：

[root@mh-k8s-master-247-10 java-testing]# kubectl apply -f java-testing-namespaces.yaml 
namespace/java-testing created
[root@mh-k8s-master-247-10 java-testing]# kubectl get namespaces java-testing
NAME           STATUS   AGE
java-testing   Active   34s
[root@mh-k8s-master-247-10 java-testing]# 

二、部署 java 实例（以jenkins为例子）

2.1、部署 NFS 服务

• 2.1.1 部署 NFS 服务端配置

  mkdir -p /data/k8s
  chown -R nfsnobody.nfsnobody /data
  echo "/data/k8s 10.255.247.0/24(rw,no_root_squash,sync)" >/etc/exports
  systemctl enable rpcbind
  systemctl enable nfs
  systemctl start rpcbind
  systemctl start nfs

• 2.1.1 客户端挂载

  systemctl start rpcbind
  systemctl enable rpcbind
  mkdir /data/k8s -p
  mount -t nfs 10.255.247.21:/mnt/data /data/k8s

2.2、创建 Jenkins 集群所需的 yaml 文件

• 2.2.1 为 jenkins 数据持久化存储创建一个pv

  apiVersion: v1
  kind: PersistentVolume
  metadata:
    name: java-testing-jenkins
    namespace: java-testing
  spec:
    capacity:
      storage: 200Gi
    accessModes:
      - ReadWriteOnce
    persistentVolumeReclaimPolicy: Delete
    nfs:
      server: 10.255.247.10
      path: /data/k8s
  
  ---
  kind: PersistentVolumeClaim
  apiVersion: v1
  metadata:
    name: java-testing-jenkins
    namespace: java-testing
  spec:
    accessModes:
      - ReadWriteOnce
    resources:
      requests:
        storage: 200Gi

• 2.2.2 为 jenkins 集群权限 serviceAccount 文件

  apiVersion: v1
  kind: ServiceAccount
  metadata:
    name: java-testing-jenkins
    namespace: java-testing
  
  ---
  kind: ClusterRole
  apiVersion: rbac.authorization.k8s.io/v1beta1
  metadata:
    name: java-testing-jenkins
  rules:
    - apiGroups: ["extensions", "apps"]
      resources: ["deployments"]
      verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
    - apiGroups: [""]
      resources: ["services"]
      verbs: ["create", "delete", "get", "list", "watch", "patch", "update"]
    - apiGroups: [""]
      resources: ["pods"]
      verbs: ["create","delete","get","list","patch","update","watch"]
    - apiGroups: [""]
      resources: ["pods/exec"]
      verbs: ["create","delete","get","list","patch","update","watch"]
    - apiGroups: [""]
      resources: ["pods/log"]
      verbs: ["get","list","watch"]
    - apiGroups: [""]
      resources: ["secrets"]
      verbs: ["get"]
  
  ---
  apiVersion: rbac.authorization.k8s.io/v1beta1
  kind: ClusterRoleBinding
  metadata:
    name: java-testing-jenkins
    namespace: java-testing
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: java-testing-jenkins
  subjects:
    - kind: ServiceAccount
      name: java-testing-jenkins
      namespace: java-testing

• 2.2.3 创建 jenkins Deployment

  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: java-testing-jenkins
    namespace: java-testing
  spec:
    replicas: 1
    selector:
      matchLabels:
          app: java-testing-jenkins
    template:
      metadata:
        labels:
          app: java-testing-jenkins
      spec:
        terminationGracePeriodSeconds: 10
        serviceAccount: java-testing-jenkins
        containers:
        - name: jenkins
          image: jenkins/jenkins:lts
          imagePullPolicy: IfNotPresent
          ports:
          - containerPort: 8080
            name: web
            protocol: TCP
          - containerPort: 50000
            name: agent
            protocol: TCP
          resources:
            limits:
              cpu: 1000m
              memory: 1Gi
            requests:
              cpu: 500m
              memory: 512Mi
          livenessProbe:
            httpGet:
              path: /login
              port: 8080
            initialDelaySeconds: 60
            timeoutSeconds: 5
            failureThreshold: 12
          readinessProbe:
            httpGet:
              path: /login
              port: 8080
            initialDelaySeconds: 60
            timeoutSeconds: 5
            failureThreshold: 12
          volumeMounts:
          - name: jenkinshome
            subPath: jenkins
            mountPath: /var/jenkins_home
          env:
          - name: JAVA_OPTS
            value: >-
                  -Xms 256Mi -Xmx 256Mi  -XX:MaxRAMPercentage=75.0
                  -XX:InitialRAMPercentage=75.0 -XX:MinRAMPercentage=75.0
                  -Dhudson.slaves.NodeProvisioner.initialDelay=20
                  -Dhudson.slaves.NodeProvisioner.MARGIN=50
                  -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
                  -Dhudson.model.LoaHeapDumpOnOutOfMemoryErrordStatistics.clock=5000
                  -Dhudson.model.LoadStatistics.decay=0.2
                  -Dhudson.slaves.NodeProvisioner.recurrencePeriod=5000
                  -Duser.timezone=Asia/Shanghai
                  -Dio.jenkins.plugins.casc.ConfigurationAsCode.initialDelay=10000
                  -XX:+HeapDumpOnOutOfMemoryError
                  -XX:HeapDumpPath=/var/jenkins_home/dump-%t.hprof -verbose:gc
                  -Xloggc:/var/jenkins_home/gc-%t.log -XX:NumberOfGCLogFiles=15
                  -XX:+UseGCLogFileRotation -XX:GCLogFileSize=100m -XX:+PrintGC
                  -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintHeapAtGC
                  -XX:+PrintGCCause -XX:+PrintGCApplicationStoppedTime
                  -XX:+PrintTenuringDistribution -XX:+PrintReferenceGC
                  -XX:+PrintAdaptiveSizePolicy -XX:+UseG1GC
                  -XX:+UseStringDeduplication -XX:+ParallelRefProcEnabled
                  -XX:+DisableExplicitGC -XX:+UnlockDiagnosticVMOptions
                  -XX:+UnlockExperimentalVMOptions 
                  -XX:+UseCGroupMemoryLimitForHeap      
        securityContext:
          fsGroup: 1000
        volumes:
        - name: jenkinshome
          persistentVolumeClaim:
            claimName: java-testing-jenkins

• 2.2.4 为 jenkins 创建 Service 资源

  apiVersion: v1
  kind: Service
  metadata:
    name: java-testing-jenkins-svc
    namespace: java-testing
    labels:
      app: java-testing-jenkins-svc
  spec:
    selector:
      app: java-testing-jenkins-svc
    ports:
    - name: web
      port: 80
      targetPort: 8080
  	protocol: TCP
    - name: agent
      port: 50000
      targetPort: 50000
      protocol: TCP

• 2.2.5 为 jenkins 创建 Ingress 资源

  apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
    name: java-testing-nfs
    namespace: java-testing
    annotations:
      kubernetes.io/ingress.class: "nginx"
  spec:
    rules:
    - host: java.zuoyang.tech
      http:
        paths:
        - path:
          backend:
            serviceName: java-testing-jenkins-svc
            servicePort: 80

•  2.2.6 配置 TLS Ingress 实例

  在 Ingress 控制器上配置 HTTPS 主机时，不能直接使用私钥和证书文件，而是要使用 Secret 资源对象来传递相关的数据。所以，接下来要根据私钥和证书生成用于配置 TLS Ingress 的 Secret 资源，在创建 Ingress 规则时由其将用到的 Secret 资源中的信息注入 Ingress 控制器的 Pod 对象中，用于为配置的 HTTPS 虚拟主机提供相应的私钥和证书。

  下面的命令会创建一个 TLS 类型名为：java-ingress-secret 的 Secret 资源：

• • kubectl create secret tls java-ingress-secret --cert=tls.crt --key=tls.key -n java-testing

  可使用下面的命令确认 Secrets 资源 java-ingress-secret 的 Secret 资源创建成功完成：

• • kubectl get secrets java-ingress-secret -n java-testing

  而后去定义创建 TLS 类型 Ingress 资源的配置清单。下面的配置清单通过 spec.rules 定义了一组转发规则，并通过 .spec.tls 将此主机定义为了 HTTPS 类型的虚拟主机，用到的私钥和证书信息来自于 Secret 资源 java-ingress-secret:

• • apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
      name: java-testing-nfs
      namespace: java-testing
      annotations:
        kubernetes.io/ingress.class: "nginx"
    spec:
      tls:
      - hosts:
        - java.zuoyang.tech
    	secretName: java-ingress-secret
      rules:
      - host: java.zuoyang.tech
        http:
          paths:
          - path: /
            backend:
              serviceName: java-testing-jenkins-svc
              servicePort: 80