• Kubernetes部署nginx-ingress


    一、Ingress介绍

    ingress可以让互联网客户访问kubernetes集群,而ClientIP和coredns只能在集群内部访问,Ingress的架构图如下:

    要使用 Ingress的步骤

    1. 先部署 Ingress Controller 实体(相当于前端 Nginx)
    2. 然后再创建 Ingress (相当于 Nginx 配置的 k8s 资源体现)
    3. Ingress Controller 部署好后会动态检测 Ingress 的创建情况生成相应配置

    Ingress Controller 的实现有很多种:

    1. 基于 Nginx 的,基于 Nginx 的 Ingress Controller 有两种,一种是 k8s 社区提供的 ingress-nginx,另一种是 Nginx 社区提供的nginx-ingress(常见,亦本文的方式),参见他们的区别 
    2. 基于 HAProxy的,
    3. 基于 OpenResty 的 Kong Ingress Controller 等
    4. 更多 Controller 见:https://kubernetes.io/docs/concepts/services-networking/ingress-controllers

    基于Nginx Ingress的拓扑图如下

     

      

    二、安装nginx-ingress

     先查找仓库

    $ helm search repo nginx-ingress
    NAME                                    CHART VERSION   APP VERSION     DESCRIPTION
    aliyuncs/nginx-ingress                  1.30.3          0.28.0          An nginx Ingress controller that uses ConfigMap...
    aliyuncs/nginx-ingress-controller       5.3.4           0.29.0          Chart for the nginx Ingress controller
    bitnami/nginx-ingress-controller        9.0.2           1.0.4           Chart for the nginx Ingress controller
    aliyuncs/nginx-lego                     0.3.1                           Chart for nginx-ingress-controller and kube-lego

     安装版本最新的第三个,要仔细看安装以后给的提示

    $ helm install nginx-ingress bitnami/nginx-ingress-controller
    NAME: nginx-ingress
    LAST DEPLOYED: Fri Nov  5 15:33:30 2021
    NAMESPACE: default
    STATUS: deployed
    REVISION: 1
    TEST SUITE: None
    NOTES:
    CHART NAME: nginx-ingress-controller
    CHART VERSION: 9.0.3
    APP VERSION: 1.0.4
    
    ** Please be patient while the chart is being deployed **
    
    The nginx-ingress controller has been installed.
    
    Get the application URL by running these commands:
    
     NOTE: It may take a few minutes for the LoadBalancer IP to be available.
            You can watch its status by running 'kubectl get --namespace default svc -w nginx-ingress-nginx-ingress-controller'
    
        export SERVICE_IP=$(kubectl get svc --namespace default nginx-ingress-nginx-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
        echo "Visit http://${SERVICE_IP} to access your application via HTTP."
        echo "Visit https://${SERVICE_IP} to access your application via HTTPS."
    
    An example Ingress that makes use of the controller:
    
      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        annotations:
          kubernetes.io/ingress.class:
        name: example
        namespace: default
      spec:
        rules:
          - host: www.example.com
            http:
              paths:
                - backend:
                    service:
                      name: example-service
                      port:
                        number: 80
                  path: /
                  pathType: Prefix
        # This section is only required if TLS is to be enabled for the Ingress
        tls:
            - hosts:
                - www.example.com
              secretName: example-tls
    
    If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:
    
      apiVersion: v1
      kind: Secret
      metadata:
        name: example-tls
        namespace: default
      data:
        tls.crt: <base64 encoded cert>
        tls.key: <base64 encoded key>
      type: kubernetes.io/tls

    查看安装的release

    $ helm list
    NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                   APP VERSION
    nginx-ingress   default         1               2021-11-02 08:22:43.135546577 +0000 UTC deployed        nginx-ingress-1.30.3    0.28.0

    查看pods

    $ kubectl get pods -o wide
    NAME                                             READY   STATUS    RESTARTS      AGE     IP               NODE   NOMINATED NODE   READINESS GATES
    gostarter-dep-df898587f-9gfmh                    1/1     Running   0             3h20m   192.168.205.50   kbsm   <none>           <none>
    gostarter-dep-df898587f-p76kn                    1/1     Running   0             3h20m   192.168.205.39   kbsm   <none>           <none>
    gostarter-dep-df898587f-scn8g                    1/1     Running   0             3h20m   192.168.205.37   kbsm   <none>           <none>
    gostarter-dep-df898587f-wq2cz                    1/1     Running   0             3h20m   192.168.184.36   kbs2   <none>           <none>
    gostarter-dep-df898587f-ztsmc                    1/1     Running   0             3h20m   192.168.205.62   kbsm   <none>           <none>
    nginx-ingress-controller-6f4cf4656d-m7wvn        0/1     Running   2 (30s ago)   2m13s   192.168.151.2    kbs1   <none>           <none>
    nginx-ingress-default-backend-78669dcf66-md9bp   1/1     Running   0             2m13s   192.168.151.57   kbs1   <none>           <none>

     查看Service

    $ kubectl get svc
    NAME                            TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
    gostarter-svc                   ClusterIP      10.109.68.204    <none>        8000/TCP                     5h44m
    kubernetes                      ClusterIP      10.96.0.1        <none>        443/TCP                      21d
    nginx-ingress-controller        LoadBalancer   10.101.177.250   <pending>     80:31534/TCP,443:32619/TCP   15m
    nginx-ingress-default-backend   ClusterIP      10.108.181.102   <none>        80/TCP                       15m

    访问一下backend

    $ curl http://10.108.181.102
    default backend - 404
    三、nginx-ingress错误

    我安装过程中出现过这种问题

    $ kubectl get pods
    NAME                                             READY   STATUS             RESTARTS      AGE
    nginx-ingress-controller-6f4cf4656d-m7wvn        0/1     CrashLoopBackOff   9 (43s ago)   18m
    nginx-ingress-default-backend-78669dcf66-md9bp   1/1     Running            0             18m

    查看这个pod的日志,有大量这种错误:Failed to list *v1beta1.Ingress: the server could not find the requested resource

    $ kubectl logs nginx-ingress-controller-6f4cf4656d-m7wvn
    I1102 08:39:16.674917       8 flags.go:205] Watching for Ingress class: nginx
    -------------------------------------------------------------------------------
    NGINX Ingress controller
      Release:       0.28.0
      Build:         git-1f93cb8f3
    W1102 08:39:16.675175       8 flags.go:250] SSL certificate chain completion is disabled (--enable-ssl-chain-completion=false)
      Repository:    https://github.com/kubernetes/ingress-nginx
      nginx version: nginx/1.17.7
    
    -------------------------------------------------------------------------------
    
    W1102 08:39:16.675227       8 client_config.go:543] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
    I1102 08:39:16.675433       8 main.go:193] Creating API client for https://10.96.0.1:443
    I1102 08:39:16.842896       8 main.go:237] Running in Kubernetes cluster version v1.22 (v1.22.2) - git (clean) commit 8b5a19147530eaac9476b0ab82980b4088bbc1b2 - platform linux/amd64
    I1102 08:39:16.858079       8 main.go:91] Validated default/nginx-ingress-default-backend as the default backend.
    I1102 08:39:17.000127       8 main.go:102] SSL fake certificate created /etc/ingress-controller/ssl/default-fake-certificate.pem
    W1102 08:39:17.047593       8 store.go:636] Unexpected error reading configuration configmap: configmaps "nginx-ingress-controller" not found
    I1102 08:39:17.054741       8 nginx.go:263] Starting NGINX Ingress controller
    E1102 08:39:18.379226       8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource
    E1102 08:39:19.449966       8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource
    E1102 08:39:20.753084       8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource
    E1102 08:39:22.109971       8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource
    E1102 08:39:23.136053       8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource
    E1102 08:39:24.458595       8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource
    E1102 08:39:25.492555       8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource
    I1102 08:39:53.382051       8 main.go:152] Received SIGTERM, shutting down
    I1102 08:39:53.382078       8 nginx.go:391] Shutting down controller queues
    I1102 08:39:53.382091       8 status.go:117] updating status of Ingress rules (remove)
    E1102 08:39:53.382177       8 store.go:185] timed out waiting for caches to sync
    I1102 08:39:53.382249       8 nginx.go:307] Starting NGINX process
    I1102 08:39:53.382435       8 leaderelection.go:242] attempting to acquire leader lease  default/ingress-controller-leader-nginx...
    E1102 08:39:53.382666       8 queue.go:78] queue has been shutdown, failed to enqueue: &ObjectMeta{Name:initial-sync,GenerateName:,Namespace:,SelfLink:,UID:,ResourceVersion:,Generation:0,CreationTimestamp:0001-01-01 00:00:00 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{},Annotations:map[string]string{},OwnerReferences:[]OwnerReference{},Finalizers:[],ClusterName:,ManagedFields:[]ManagedFieldsEntry{},}
    I1102 08:39:53.398779       8 leaderelection.go:252] successfully acquired lease default/ingress-controller-leader-nginx
    I1102 08:39:53.398841       8 status.go:86] new leader elected: nginx-ingress-controller-6f4cf4656d-m7wvn
    E1102 08:39:53.398884       8 queue.go:78] queue has been shutdown, failed to enqueue: &ObjectMeta{Name:sync status,GenerateName:,Namespace:,SelfLink:,UID:,ResourceVersion:,Generation:0,CreationTimestamp:0001-01-01 00:00:00 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{},Annotations:map[string]string{},OwnerReferences:[]OwnerReference{},Finalizers:[],ClusterName:,ManagedFields:[]ManagedFieldsEntry{},}
    I1102 08:39:53.422921       8 status.go:136] removing address from ingress status ([192.168.0.106])
    I1102 08:39:53.423027       8 nginx.go:407] Stopping NGINX process
    2021/11/02 08:39:53 [notice] 36#36: signal process started
    I1102 08:39:56.444623       8 nginx.go:420] NGINX process has stopped
    I1102 08:39:56.444662       8 main.go:160] Handled quit, awaiting Pod deletion
    I1102 08:40:06.445065       8 main.go:163] Exiting with 0

    原因 : 我的kubernetes:v1.22 不再支持v1beta1,所以与低版本的aliyuncs/nginx-ingress:0.28.0不匹配,要改成使用高版本的bitnami/nginx-ingress-controller:1.0.4

    下面再列一下helm的nginx-ingress的chart的源的查询

    $ helm search repo nginx-ingress
    NAME                                    CHART VERSION   APP VERSION     DESCRIPTION
    aliyuncs/nginx-ingress                  1.30.3          0.28.0          An nginx Ingress controller that uses ConfigMap...
    aliyuncs/nginx-ingress-controller       5.3.4           0.29.0          Chart for the nginx Ingress controller
    bitnami/nginx-ingress-controller        9.0.2           1.0.4           Chart for the nginx Ingress controller
    aliyuncs/nginx-lego                     0.3.1                           Chart for nginx-ingress-controller and kube-lego
    五、安装Nginx

    helm中 查看chart

    $ helm search repo nginx
    NAME                                    CHART VERSION   APP VERSION             DESCRIPTION
    aliyuncs/nginx                          5.1.5           1.16.1                  Chart for the nginx server
    aliyuncs/nginx-ingress                  1.30.3          0.28.0                  An nginx Ingress controller that uses ConfigMap...
    aliyuncs/nginx-ingress-controller       5.3.4           0.29.0                  Chart for the nginx Ingress controller
    aliyuncs/nginx-lego                     0.3.1                                   Chart for nginx-ingress-controller and kube-lego
    aliyuncs/nginx-php                      1.0.0           nginx-1.10.3_php-7.0    Chart for the nginx php server
    bitnami/nginx                           9.5.12          1.21.3                  Chart for the nginx server
    bitnami/nginx-ingress-controller        9.0.2           1.0.4                   Chart for the nginx Ingress controller
    bitnami/kong                            4.1.7           2.6.0                   Kong is a scalable, open source API layer (aka ...

    执行安装

    helm install nginx bitnami/nginx
    NAME: nginx
    LAST DEPLOYED: Fri Nov  5 16:01:35 2021
    NAMESPACE: default
    STATUS: deployed
    REVISION: 1
    TEST SUITE: None
    NOTES:
    CHART NAME: nginx
    CHART VERSION: 9.5.13
    APP VERSION: 1.21.4
    
    ** Please be patient while the chart is being deployed **
    
    NGINX can be accessed through the following DNS name from within your cluster:
    
        nginx.default.svc.cluster.local (port 80)
    
    To access NGINX from outside the cluster, follow the steps below:
    
    1. Get the NGINX URL by running these commands:
    
      NOTE: It may take a few minutes for the LoadBalancer IP to be available.
            Watch the status with: 'kubectl get svc --namespace default -w nginx'
    
        export SERVICE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].port}" services nginx)
        export SERVICE_IP=$(kubectl get svc --namespace default nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
        echo "http://${SERVICE_IP}:${SERVICE_PORT}"
    root@kbsm:~/k8s/nginx-ingress#

    查看pod

    $ kubectl get pods
    NAME                                                              READY   STATUS    RESTARTS        AGE
    nginx-588469f6d6-rd5gx                                            1/1     Running   0               82s
    nginx-ingress-nginx-ingress-controller-7bdbcc7787-pgk82           1/1     Running   0               29m
    nginx-ingress-nginx-ingress-controller-default-backend-57ftmtdx   1/1     Running   0               29m
    六、Ingress配置

     先配置nginx自身的ingress:vi nginx-ingress.yaml

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: ingress-nginx
      annotations:
        # use the shared ingress-nginx
        kubernetes.io/ingress.class: "nginx"
    spec:
      rules:
      - host: kbsm
        http:
          paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: nginx
                port: 
                  number: 80

    ingress的配置新版本的格式与老版本格式不一致,可以查看官方文档

    对于安装的kubernetes集群支持的apiVersion版本,可以通过这个命令查看:kubectl api-versions

    然后发布ingress

    $ kubectl apply -f nginx-ingress.yaml
    ingress.networking.k8s.io/ingress-nginx created
    七、Ingress测试
    八、删除本文配置

     如果nginx-ingress配置失败,删除本文所有的配置

    kubectl delete -f nginx-ingress.yaml
    helm uninstall nginx
    helm uninstall nginx-ingress

    参考资料:

    1. 使用 Kubernetes Ingress 对外暴露服务

    作者    :秋时

    本文版权归作者和博客园共有,欢迎转载,但未经作者同意必须保留此段声明,且在文章页面明显位置给出原文连接。

  • 相关阅读:
    cf D. Vessels
    cf C. Hamburgers
    zoj 3758 Singles' Day
    zoj 3777 Problem Arrangement
    zoj 3778 Talented Chef
    hdu 5087 Revenge of LIS II
    zoj 3785 What day is that day?
    zoj 3787 Access System
    判断给定图是否存在合法拓扑排序
    树-堆结构练习——合并果子之哈夫曼树
  • 原文地址:https://www.cnblogs.com/Netsharp/p/15498909.html
Copyright © 2020-2023  润新知