• Kubernetes部署nginx-ingress


    一、Ingress介绍

    ingress可以让互联网客户访问kubernetes集群,而ClientIP和coredns只能在集群内部访问,Ingress的架构图如下:

    要使用 Ingress的步骤

    1. 先部署 Ingress Controller 实体(相当于前端 Nginx)
    2. 然后再创建 Ingress (相当于 Nginx 配置的 k8s 资源体现)
    3. Ingress Controller 部署好后会动态检测 Ingress 的创建情况生成相应配置

    Ingress Controller 的实现有很多种:

    1. 基于 Nginx 的,基于 Nginx 的 Ingress Controller 有两种,一种是 k8s 社区提供的 ingress-nginx,另一种是 Nginx 社区提供的nginx-ingress(常见,亦本文的方式),参见他们的区别 
    2. 基于 HAProxy的,
    3. 基于 OpenResty 的 Kong Ingress Controller 等
    4. 更多 Controller 见:https://kubernetes.io/docs/concepts/services-networking/ingress-controllers

    基于Nginx Ingress的拓扑图如下

     

      

    二、安装nginx-ingress

     先查找仓库

    $ helm search repo nginx-ingress
    NAME                                    CHART VERSION   APP VERSION     DESCRIPTION
    aliyuncs/nginx-ingress                  1.30.3          0.28.0          An nginx Ingress controller that uses ConfigMap...
    aliyuncs/nginx-ingress-controller       5.3.4           0.29.0          Chart for the nginx Ingress controller
    bitnami/nginx-ingress-controller        9.0.2           1.0.4           Chart for the nginx Ingress controller
    aliyuncs/nginx-lego                     0.3.1                           Chart for nginx-ingress-controller and kube-lego

     安装版本最新的第三个,要仔细看安装以后给的提示

    $ helm install nginx-ingress bitnami/nginx-ingress-controller
    NAME: nginx-ingress
    LAST DEPLOYED: Fri Nov  5 15:33:30 2021
    NAMESPACE: default
    STATUS: deployed
    REVISION: 1
    TEST SUITE: None
    NOTES:
    CHART NAME: nginx-ingress-controller
    CHART VERSION: 9.0.3
    APP VERSION: 1.0.4
    
    ** Please be patient while the chart is being deployed **
    
    The nginx-ingress controller has been installed.
    
    Get the application URL by running these commands:
    
     NOTE: It may take a few minutes for the LoadBalancer IP to be available.
            You can watch its status by running 'kubectl get --namespace default svc -w nginx-ingress-nginx-ingress-controller'
    
        export SERVICE_IP=$(kubectl get svc --namespace default nginx-ingress-nginx-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
        echo "Visit http://${SERVICE_IP} to access your application via HTTP."
        echo "Visit https://${SERVICE_IP} to access your application via HTTPS."
    
    An example Ingress that makes use of the controller:
    
      apiVersion: networking.k8s.io/v1
      kind: Ingress
      metadata:
        annotations:
          kubernetes.io/ingress.class:
        name: example
        namespace: default
      spec:
        rules:
          - host: www.example.com
            http:
              paths:
                - backend:
                    service:
                      name: example-service
                      port:
                        number: 80
                  path: /
                  pathType: Prefix
        # This section is only required if TLS is to be enabled for the Ingress
        tls:
            - hosts:
                - www.example.com
              secretName: example-tls
    
    If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:
    
      apiVersion: v1
      kind: Secret
      metadata:
        name: example-tls
        namespace: default
      data:
        tls.crt: <base64 encoded cert>
        tls.key: <base64 encoded key>
      type: kubernetes.io/tls

    查看安装的release

    $ helm list
    NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                   APP VERSION
    nginx-ingress   default         1               2021-11-02 08:22:43.135546577 +0000 UTC deployed        nginx-ingress-1.30.3    0.28.0

    查看pods

    $ kubectl get pods -o wide
    NAME                                             READY   STATUS    RESTARTS      AGE     IP               NODE   NOMINATED NODE   READINESS GATES
    gostarter-dep-df898587f-9gfmh                    1/1     Running   0             3h20m   192.168.205.50   kbsm   <none>           <none>
    gostarter-dep-df898587f-p76kn                    1/1     Running   0             3h20m   192.168.205.39   kbsm   <none>           <none>
    gostarter-dep-df898587f-scn8g                    1/1     Running   0             3h20m   192.168.205.37   kbsm   <none>           <none>
    gostarter-dep-df898587f-wq2cz                    1/1     Running   0             3h20m   192.168.184.36   kbs2   <none>           <none>
    gostarter-dep-df898587f-ztsmc                    1/1     Running   0             3h20m   192.168.205.62   kbsm   <none>           <none>
    nginx-ingress-controller-6f4cf4656d-m7wvn        0/1     Running   2 (30s ago)   2m13s   192.168.151.2    kbs1   <none>           <none>
    nginx-ingress-default-backend-78669dcf66-md9bp   1/1     Running   0             2m13s   192.168.151.57   kbs1   <none>           <none>

     查看Service

    $ kubectl get svc
    NAME                            TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
    gostarter-svc                   ClusterIP      10.109.68.204    <none>        8000/TCP                     5h44m
    kubernetes                      ClusterIP      10.96.0.1        <none>        443/TCP                      21d
    nginx-ingress-controller        LoadBalancer   10.101.177.250   <pending>     80:31534/TCP,443:32619/TCP   15m
    nginx-ingress-default-backend   ClusterIP      10.108.181.102   <none>        80/TCP                       15m

    访问一下backend

    $ curl http://10.108.181.102
    default backend - 404
    三、nginx-ingress错误

    我安装过程中出现过这种问题

    $ kubectl get pods
    NAME                                             READY   STATUS             RESTARTS      AGE
    nginx-ingress-controller-6f4cf4656d-m7wvn        0/1     CrashLoopBackOff   9 (43s ago)   18m
    nginx-ingress-default-backend-78669dcf66-md9bp   1/1     Running            0             18m

    查看这个pod的日志,有大量这种错误:Failed to list *v1beta1.Ingress: the server could not find the requested resource

    $ kubectl logs nginx-ingress-controller-6f4cf4656d-m7wvn
    I1102 08:39:16.674917       8 flags.go:205] Watching for Ingress class: nginx
    -------------------------------------------------------------------------------
    NGINX Ingress controller
      Release:       0.28.0
      Build:         git-1f93cb8f3
    W1102 08:39:16.675175       8 flags.go:250] SSL certificate chain completion is disabled (--enable-ssl-chain-completion=false)
      Repository:    https://github.com/kubernetes/ingress-nginx
      nginx version: nginx/1.17.7
    
    -------------------------------------------------------------------------------
    
    W1102 08:39:16.675227       8 client_config.go:543] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
    I1102 08:39:16.675433       8 main.go:193] Creating API client for https://10.96.0.1:443
    I1102 08:39:16.842896       8 main.go:237] Running in Kubernetes cluster version v1.22 (v1.22.2) - git (clean) commit 8b5a19147530eaac9476b0ab82980b4088bbc1b2 - platform linux/amd64
    I1102 08:39:16.858079       8 main.go:91] Validated default/nginx-ingress-default-backend as the default backend.
    I1102 08:39:17.000127       8 main.go:102] SSL fake certificate created /etc/ingress-controller/ssl/default-fake-certificate.pem
    W1102 08:39:17.047593       8 store.go:636] Unexpected error reading configuration configmap: configmaps "nginx-ingress-controller" not found
    I1102 08:39:17.054741       8 nginx.go:263] Starting NGINX Ingress controller
    E1102 08:39:18.379226       8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource
    E1102 08:39:19.449966       8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource
    E1102 08:39:20.753084       8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource
    E1102 08:39:22.109971       8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource
    E1102 08:39:23.136053       8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource
    E1102 08:39:24.458595       8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource
    E1102 08:39:25.492555       8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource
    I1102 08:39:53.382051       8 main.go:152] Received SIGTERM, shutting down
    I1102 08:39:53.382078       8 nginx.go:391] Shutting down controller queues
    I1102 08:39:53.382091       8 status.go:117] updating status of Ingress rules (remove)
    E1102 08:39:53.382177       8 store.go:185] timed out waiting for caches to sync
    I1102 08:39:53.382249       8 nginx.go:307] Starting NGINX process
    I1102 08:39:53.382435       8 leaderelection.go:242] attempting to acquire leader lease  default/ingress-controller-leader-nginx...
    E1102 08:39:53.382666       8 queue.go:78] queue has been shutdown, failed to enqueue: &ObjectMeta{Name:initial-sync,GenerateName:,Namespace:,SelfLink:,UID:,ResourceVersion:,Generation:0,CreationTimestamp:0001-01-01 00:00:00 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{},Annotations:map[string]string{},OwnerReferences:[]OwnerReference{},Finalizers:[],ClusterName:,ManagedFields:[]ManagedFieldsEntry{},}
    I1102 08:39:53.398779       8 leaderelection.go:252] successfully acquired lease default/ingress-controller-leader-nginx
    I1102 08:39:53.398841       8 status.go:86] new leader elected: nginx-ingress-controller-6f4cf4656d-m7wvn
    E1102 08:39:53.398884       8 queue.go:78] queue has been shutdown, failed to enqueue: &ObjectMeta{Name:sync status,GenerateName:,Namespace:,SelfLink:,UID:,ResourceVersion:,Generation:0,CreationTimestamp:0001-01-01 00:00:00 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{},Annotations:map[string]string{},OwnerReferences:[]OwnerReference{},Finalizers:[],ClusterName:,ManagedFields:[]ManagedFieldsEntry{},}
    I1102 08:39:53.422921       8 status.go:136] removing address from ingress status ([192.168.0.106])
    I1102 08:39:53.423027       8 nginx.go:407] Stopping NGINX process
    2021/11/02 08:39:53 [notice] 36#36: signal process started
    I1102 08:39:56.444623       8 nginx.go:420] NGINX process has stopped
    I1102 08:39:56.444662       8 main.go:160] Handled quit, awaiting Pod deletion
    I1102 08:40:06.445065       8 main.go:163] Exiting with 0

    原因 : 我的kubernetes:v1.22 不再支持v1beta1,所以与低版本的aliyuncs/nginx-ingress:0.28.0不匹配,要改成使用高版本的bitnami/nginx-ingress-controller:1.0.4

    下面再列一下helm的nginx-ingress的chart的源的查询

    $ helm search repo nginx-ingress
    NAME                                    CHART VERSION   APP VERSION     DESCRIPTION
    aliyuncs/nginx-ingress                  1.30.3          0.28.0          An nginx Ingress controller that uses ConfigMap...
    aliyuncs/nginx-ingress-controller       5.3.4           0.29.0          Chart for the nginx Ingress controller
    bitnami/nginx-ingress-controller        9.0.2           1.0.4           Chart for the nginx Ingress controller
    aliyuncs/nginx-lego                     0.3.1                           Chart for nginx-ingress-controller and kube-lego
    五、安装Nginx

    helm中 查看chart

    $ helm search repo nginx
    NAME                                    CHART VERSION   APP VERSION             DESCRIPTION
    aliyuncs/nginx                          5.1.5           1.16.1                  Chart for the nginx server
    aliyuncs/nginx-ingress                  1.30.3          0.28.0                  An nginx Ingress controller that uses ConfigMap...
    aliyuncs/nginx-ingress-controller       5.3.4           0.29.0                  Chart for the nginx Ingress controller
    aliyuncs/nginx-lego                     0.3.1                                   Chart for nginx-ingress-controller and kube-lego
    aliyuncs/nginx-php                      1.0.0           nginx-1.10.3_php-7.0    Chart for the nginx php server
    bitnami/nginx                           9.5.12          1.21.3                  Chart for the nginx server
    bitnami/nginx-ingress-controller        9.0.2           1.0.4                   Chart for the nginx Ingress controller
    bitnami/kong                            4.1.7           2.6.0                   Kong is a scalable, open source API layer (aka ...

    执行安装

    helm install nginx bitnami/nginx
    NAME: nginx
    LAST DEPLOYED: Fri Nov  5 16:01:35 2021
    NAMESPACE: default
    STATUS: deployed
    REVISION: 1
    TEST SUITE: None
    NOTES:
    CHART NAME: nginx
    CHART VERSION: 9.5.13
    APP VERSION: 1.21.4
    
    ** Please be patient while the chart is being deployed **
    
    NGINX can be accessed through the following DNS name from within your cluster:
    
        nginx.default.svc.cluster.local (port 80)
    
    To access NGINX from outside the cluster, follow the steps below:
    
    1. Get the NGINX URL by running these commands:
    
      NOTE: It may take a few minutes for the LoadBalancer IP to be available.
            Watch the status with: 'kubectl get svc --namespace default -w nginx'
    
        export SERVICE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].port}" services nginx)
        export SERVICE_IP=$(kubectl get svc --namespace default nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
        echo "http://${SERVICE_IP}:${SERVICE_PORT}"
    root@kbsm:~/k8s/nginx-ingress#

    查看pod

    $ kubectl get pods
    NAME                                                              READY   STATUS    RESTARTS        AGE
    nginx-588469f6d6-rd5gx                                            1/1     Running   0               82s
    nginx-ingress-nginx-ingress-controller-7bdbcc7787-pgk82           1/1     Running   0               29m
    nginx-ingress-nginx-ingress-controller-default-backend-57ftmtdx   1/1     Running   0               29m
    六、Ingress配置

     先配置nginx自身的ingress:vi nginx-ingress.yaml

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: ingress-nginx
      annotations:
        # use the shared ingress-nginx
        kubernetes.io/ingress.class: "nginx"
    spec:
      rules:
      - host: kbsm
        http:
          paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: nginx
                port: 
                  number: 80

    ingress的配置新版本的格式与老版本格式不一致,可以查看官方文档

    对于安装的kubernetes集群支持的apiVersion版本,可以通过这个命令查看:kubectl api-versions

    然后发布ingress

    $ kubectl apply -f nginx-ingress.yaml
    ingress.networking.k8s.io/ingress-nginx created
    七、Ingress测试
    八、删除本文配置

     如果nginx-ingress配置失败,删除本文所有的配置

    kubectl delete -f nginx-ingress.yaml
    helm uninstall nginx
    helm uninstall nginx-ingress

    参考资料:

    1. 使用 Kubernetes Ingress 对外暴露服务

    作者    :秋时

    本文版权归作者和博客园共有,欢迎转载,但未经作者同意必须保留此段声明,且在文章页面明显位置给出原文连接。

  • 相关阅读:
    CMMI全称是Capability Maturity Model Integration,即能力成熟度模型集成
    【beta】Scrum站立会议第5次....11.7
    【beta】Scrum站立会议第3次....11.6
    补交进度条
    【beta】Scrum站立会议第1次....11.3
    【week7】psp
    【week6】psp
    【week6】约跑App视频链接
    【week6】团队贡献分
    【week6】用户数
  • 原文地址:https://www.cnblogs.com/Netsharp/p/15498909.html
Copyright © 2020-2023  润新知