Kubernetes v1.19 [stable] 管理对集群中服务的外部访问的API对象，通常是HTTP。 Ingress可以提供负载平衡、SSL终止和基于名称的虚拟主机 Ingress 是什么? Ingress将来自集群外部的HTTP和HTTPS路由暴露给集群内的服务。流量路由由Ingress资源上定义的规则控制。 可以将Ingress配置为提供服务外部可访问的 URL、负载均衡流量、SSL/TLS，以及提供基于名称的虚拟主机。Ingress 控制器 通常负责通过负载均衡器来实现 ingress，尽管它也可以配置边缘路由器或其他前端来帮助处理流量。 Ingress不会公开任意端口或协议。若将 HTTP 和 HTTPS 以外的服务公开到 Internet 时，通常使用 Service.Type=NodePort 或者 Service.Type=LoadBalancer 类型的服务。 ingress是k8s集群里工作在osi网络参考模型下，第7层的应用，对外暴露的接口； service只能进行L4流量调度器，表现形式是ip+port； ingress则可以调度不同业务域、不同URL访问路径的业务流量； 文档：https://kubernetes.io/docs/concepts/services-networking/ingress/ Ingress-Nginx官网地址：https://kubernetes.github.io/ingress-nginx/ GitHub地址：https://github.com/kubernetes/ingress-nginx/ ingress所需的yaml文件： namespace.yaml rabc.yaml configmap.yaml tcp-services-configmap.yaml udp-services-cnfigmap.yaml with-rabc.yaml # kubectl apply -f namespace.yaml # kubectl apply -f rabc.yaml .... # kubectl get pod -n ingress-nginx 这是一个简单的示例，其中Ingress将其所有流量发送到一个Service： Ingress可以配置为向服务提供外部可访问的 URL、负载平衡流量、终止SSL/TLS并提供基于名称的虚拟主机。一个入口控制器负责履行入口，通常有一个负载均衡器，虽然它也可以配置您的边缘路由器或额外的前端，以帮助处理流量。 Ingress不会公开任意端口或协议。向Internet公开HTTP和HTTPS以外的服务通常使用Service.Type=NodePort或Service.Type=LoadBalancer类型的服务 示例1： https://cloud.tencent.com/developer/article/1718482 https://www.cnblogs.com/zhanglianghhh/p/13721288.html 镜像下载与重命名：因quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0无法拉取下来镜像，所以： docker pull registry.cn-beijing.aliyuncs.com/google_registry/nginx-ingress-controller:0.30.0 docker tag 89ccad40ce8e quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0 docker rmi registry.cn-beijing.aliyuncs.com/google_registry/nginx-ingress-controller:0.30.0 使用nginx:0.30 ingress-nginx的yaml文件修改后并启动： 下载包： wget https://github.com/kubernetes/ingress-nginx/archive/nginx-0.30.0.tar.gz tar -xf nginx-0.30.0.tar.gz 解压后：yaml文件在下载包中的位置：ingress-nginx-nginx-0.30.0/deploy/static/mandatory.yaml cp -a ingress-nginx-nginx-0.30.0/deploy/static/mandatory.yaml ./ yaml文件配置修改： # vim mandatory.yaml ……………… apiVersion: apps/v1 kind: DaemonSet # 从Deployment改为DaemonSet metadata: name: nginx-ingress-controller namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: #replicas: 1 # 注释掉 ……………… nodeSelector: kubernetes.io/hostname: vm1 # 修改处，需注意打的master的标签label值 # 如下几行为新加行 作用【允许在master节点运行】 tolerations: - key: node-role.kubernetes.io/master effect: NoSchedule ……………… ports: - name: http containerPort: 80 hostPort: 80 # 添加处【可在宿主机通过该端口访问Pod】 protocol: TCP - name: https containerPort: 443 hostPort: 443 # 添加处【可在宿主机通过该端口访问Pod】 protocol: TCP ……………… # kubectl apply -f mandatory.yaml namespace/ingress-nginx created configmap/nginx-configuration created configmap/tcp-services created configmap/udp-services created serviceaccount/nginx-ingress-serviceaccount created clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created role.rbac.authorization.k8s.io/nginx-ingress-role created rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created daemonset.apps/nginx-ingress-controller created limitrange/ingress-nginx created # kubectl get ds -n ingress-nginx -owide NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE CONTAINERS IMAGES SELECTOR nginx-ingress-controller 1 1 1 1 1 kubernetes.io/hostname=vm1 62s nginx-ingress-controller registry.cn-beijing.aliyuncs.com/google_registry/nginx-ingress-controller:0.30.0 app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx # kubectl get pod -n ingress-nginx -owide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-ingress-controller-9hs4s 1/1 Running 0 66s 10.244.0.7 vm1 <none> <none> 创建deployment和service： deployment_service1的yaml: # cat deploy_service1.yaml apiVersion: apps/v1 kind: Deployment metadata: name: myapp-deploy1 namespace: default spec: relicas: 3 selector: matchLabels: app: myapp release: v1 template: metadata: labels: app: myapp release: v1 spec: containers: - name: myapp image: registry.cn-beijing.aliyuncs.com/google_registry/myapp:v1 imagePullPolicy: IfNotPresent ports: - name: http containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: myapp-clusterip1 namespace: default spec: type: ClusterIP #默认类型 selector: app: myapp release: v1 ports: - name: http port: 80 targetPort: 80 启动Deployment和Service # kubectl apply -f deploy_service1.yaml # kubectl get deployment -owide NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR myapp-deploy1 3/3 3 3 5m19s myapp registry.cn-beijing.aliyuncs.com/google_registry/myapp:v1 app=myapp,release=v1 # kubectl get rs -owide NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR myapp-deploy1-7dc65c445c 3 3 3 5m55s myapp registry.cn-beijing.aliyuncs.com/google_registry/myapp:v1 app=myapp,pod-template-hash=7dc65c445c,release=v1 # kubectl get pod -owide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES myapp-deploy1-7dc65c445c-7sckz 1/1 Running 0 67s 10.244.1.34 vm2 <none> <none> myapp-deploy1-7dc65c445c-mqnst 1/1 Running 0 6m15s 10.244.1.33 vm2 <none> <none> myapp-deploy1-7dc65c445c-wn5vb 1/1 Running 0 67s 10.244.1.35 vm2 <none> <none> # kubectl get svc -owide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR kubernetes ClusterIP 10.1.0.1 <none> 443/TCP 44d <none> myapp-clusterip1 ClusterIP 10.1.165.199 <none> 80/TCP 3m19s app=myapp,release=v1 curl访问pod： # curl 10.244.1.34 # curl 10.244.1.34/hostname.html # curl 10.244.1.33/hostname.html # curl 10.244.1.35/hostname.html curl访问svc: # curl 10.1.165.199 # curl 10.1.165.199/hostname.html #多次curl deployment_service2的yaml: # cat deploy_service2.yaml apiVersion: apps/v1 kind: Deployment metadata: name: myapp-deploy2 namespace: default spec: replicas: 3 selector: matchLabels: app: myapp release: v2 template: metadata: labels: app: myapp release: v2 spec: containers: - name: myapp image: registry.cn-beijing.aliyuncs.com/google_registry/myapp:v2 imagePullPolicy: IfNotPresent ports: - name: http containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: myapp-clusterip2 namespace: default spec: type: ClusterIP #默认类型 selector: app: myapp release: v2 ports: - name: http port: 80 targetPort: 80 启动Deployment和Service # kubectl apply -f deploy_service2.yaml # kubectl get deployment -owide NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR myapp-deploy1 3/3 3 3 14m myapp registry.cn-beijing.aliyuncs.com/google_registry/myapp:v1 app=myapp,release=v1 myapp-deploy2 3/3 3 3 110s myapp registry.cn-beijing.aliyuncs.com/google_registry/myapp:v2 app=myapp,release=v2 # kubectl get rs -owide NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR myapp-deploy1-7dc65c445c 3 3 3 14m myapp registry.cn-beijing.aliyuncs.com/google_registry/myapp:v1 app=myapp,pod-template-hash=7dc65c445c,release=v1 myapp-deploy2-744ff669dc 3 3 3 118s myapp registry.cn-beijing.aliyuncs.com/google_registry/myapp:v2 app=myapp,pod-template-hash=744ff669dc,release=v2 # kubectl get pod -owide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES myapp-deploy1-7dc65c445c-7sckz 1/1 Running 0 9m51s 10.244.1.34 vm2 <none> <none> myapp-deploy1-7dc65c445c-mqnst 1/1 Running 0 14m 10.244.1.33 vm2 <none> <none> myapp-deploy1-7dc65c445c-wn5vb 1/1 Running 0 9m51s 10.244.1.35 vm2 <none> <none> myapp-deploy2-744ff669dc-5rzrm 1/1 Running 0 2m3s 10.244.1.36 vm2 <none> <none> myapp-deploy2-744ff669dc-mbkxv 1/1 Running 0 2m3s 10.244.1.37 vm2 <none> <none> myapp-deploy2-744ff669dc-zwkkz 1/1 Running 0 2m3s 10.244.1.38 vm2 <none> <none> # kubectl get svc -owide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR kubernetes ClusterIP 10.1.0.1 <none> 443/TCP 44d <none> myapp-clusterip1 ClusterIP 10.1.165.199 <none> 80/TCP 16m app=myapp,release=v1 myapp-clusterip2 ClusterIP 10.1.125.102 <none> 80/TCP 2m9s app=myapp,release=v2 curl访问svc: # curl 10.1.125.102 # curl 10.1.125.102/hostname.html #多次curl Ingress HTTP代理访问 yaml文件(由于自建的service在默认default名称空间，因此这里也是default名称空间) # cat ingress-http.yaml apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: nginx-http namespace: default spec: rules: - host: www.zytest01.com http: paths: - path: / backend: serviceName: myapp-clusterip1 servicePort: 80 - host : www.zytest02.com http: paths: - path: / backend: serviceName: myapp-clusterip2 servicePort: 80 启动ingress并查看状态： # kubectl apply -f ingress-http.yaml ingress.networking.k8s.io/nginx-http created # kubectl get ingress NAME CLASS HOSTS ADDRESS PORTS AGE nginx-http <none> www.zytest01.com,www.zytest02.com 80 5s 查看Nginx的配置： # kubectl get pod -n ingress-nginx NAME READY STATUS RESTARTS AGE nginx-ingress-controller-9hs4s 1/1 Running 0 47m # kubectl exec -it nginx-ingress-controller-9hs4s -n ingress-nginx bash $ cat /etc/nginx/nginx.conf ...... ## start server www.zytest01.com server { server_name www.zytest01.com ; listen 80 ; listen 443 ssl http2 ; set $proxy_upstream_name "-"; ssl_certificate_by_lua_block { certificate.call() } location / { set $namespace "default"; set $ingress_name "nginx-http"; set $service_name "myapp-clusterip1"; set $service_port "80"; set $location_path "/"; ...... ## start server www.zytest02.com server { server_name www.zytest02.com ; listen 80 ; listen 443 ssl http2 ; set $proxy_upstream_name "-"; ssl_certificate_by_lua_block { certificate.call() } location / { set $namespace "default"; set $ingress_name "nginx-http"; set $service_name "myapp-clusterip2"; set $service_port "80"; set $location_path "/"; ...... 浏览器访问: hosts文件修改，添加如下信息(192.168.1.84为master/ingress机器的ip) 文件位置：C:\WINDOWS\System32\drivers\etc\hosts 192.168.1.84 www.zytest01.com www.zytest02.com 浏览器里输入：http://www.zytest01.com http://www.zytest02.com Ingress HTTPS代理访问 SSL证书创建： # mkdir cert # openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/C=CN/ST=BJ/L=BeiJing/O=BTC/OU=MOST/CN=zhang/emailAddress=ca@test.com" cert]# ls -l total 8 -rw-r--r-- 1 root root 1359 Oct 6 01:06 tls.crt -rw-r--r-- 1 root root 1708 Oct 6 01:06 tls.key # kubectl create secret tls tls-secret --key tls.key --cert tls.crt secret/tls-secret created 创建ingress https yaml文件(由于自建的service在默认default名称空间，因此这里也是default名称空间) # cat ingress-https.yaml apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: nginx-https namespace: default spec: tls: - hosts: - www.zytest01.com - www.zytest02.com secretName: tls-secret rules: - host: www.zytest01.com http: paths: - path: / backend: serviceName: myapp-clusterip1 servicePort: 80 - host: www.zytest02.com http: paths: - path: / backend: serviceName: myapp-clusterip2 servicePort: 80 启动ingress https并查看状态 # kubectl apply -f ingress-https.yaml # kubectl get ingress -owide NAME CLASS HOSTS ADDRESS PORTS AGE nginx-http <none> www.zytest01.com,www.zytest02.com 80 26m nginx-https <none> www.zytest01.com,www.zytest02.com 80, 443 79s 浏览器访问 hosts文件修改后再在浏览器访问： 浏览器里输入：https://www.zytest01.com https://www.zytest02.com Ingress-Nginx实现BasicAuth认证 官网地址： https://kubernetes.github.io/ingress-nginx/examples/auth/basic/ 准备工作： # htpasswd -c auth foo New password: Re-type new password: Adding password for user foo # ls -l -rw-r--r-- 1 root root 42 Oct 6 11:03 auth # cat auth foo:$apr1$Z3CNOfQA$kgNyXwHUz9gwGLmqnFJ.p1 # kubectl create secret generic basic-auth --from-file=auth secret/basic-auth created # kubectl get secret NAME TYPE DATA AGE basic-auth Opaque 1 8s default-token-4fzfg kubernetes.io/service-account-token 3 44d tls-secret kubernetes.io/tls 2 9h # kubectl get secret basic-auth -oyaml apiVersion: v1 data: auth: Zm9vOiRhcHIxJFozQ05PZlFBJGtnTnlYd0hVejlnd0dMbXFuRkoucDEK kind: Secret metadata: creationTimestamp: "2021-10-06T03:03:28Z" name: basic-auth namespace: default resourceVersion: "785781" selfLink: /api/v1/namespaces/default/secrets/basic-auth uid: f2144306-cafe-41ac-a2d0-15ab8f438602 type: Opaque ingress yaml文件： cat ingress-basicauth.yaml apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: ingress-with-auth annotations: # type of authentication nginx.ingress.kubernetes.io/auth-type: basic # name of the secret that contains the user/password definitions nginx.ingress.kubernetes.io/auth-secret: basic-auth # message to display with an appropriate context why the authentication is required nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - foo' spec: rules: - host: www.auth.zytest01.com http: paths: - path: / backend: serviceName: myapp-clusterip1 servicePort: 80 启动ingress并且查看状态： # kubectl apply -f ingress-basicauth.yaml ingress.networking.k8s.io/ingress-with-auth created # kubectl get ingress -owide NAME CLASS HOSTS ADDRESS PORTS AGE ingress-with-auth <none> www.auth.zytest01.com 80 68s 浏览器访问：写hosts后访问，需要输入用户名和密码 http://www.auth.zytest01.com Ingress-Nginx实现Rewrite重写 官网地址： https://kubernetes.github.io/ingress-nginx/examples/rewrite/ 名称描述值nginx.ingress.kubernetes.io/rewrite-target必须重定向的目标URLStringnginx.ingress.kubernetes.io/ssl-redirect指示位置部分是否只能由SSL访问(当Ingress包含证书时，默认为True)Boolnginx.ingress.kubernetes.io/force-ssl-redirect即使Ingress没有启用TLS，也强制重定向到HTTPSBoolnginx.ingress.kubernetes.io/app-root定义应用程序根目录，Controller在“/”上下文中必须重定向该根目录Stringnginx.ingress.kubernetes.io/use-regex指示Ingress上定义的路径是否使用正则表达式Bool ingress yaml文件： # cat ingress-rewrite.yaml apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: annotations: nginx.ingress.kubernetes.io/rewrite-target: https://www.baidu.com name: rewrite namespace: default spec: rules: - host: www.rewrite.zytest01.com http: paths: - backend: serviceName: myapp-clusterip1 servicePort: 80 启动ingress并且查看状态： # kubectl apply -f ingress-rewrite.yaml ingress.networking.k8s.io/rewrite created # kubectl get ingress -owide NAME CLASS HOSTS ADDRESS PORTS AGE rewrite <none> www.rewrite.zytest.com 80 66s 浏览器访问：写hosts后访问，需要输入用户名和密码 http://www.rewrite.zytest01.com 之后，可见重定向到了https://www.baidu.com 百度页面 ---------------------------------- 其它示例： service.yaml apiVersion:v1 kind: Service metadata: name: myapp namespace: default spec: selector: app: myapp release: canary ports: - name: http targetPort: 80 port: 80 --- apiVersion:apps/v1 kind: Deployment metadata: name: myapp-deploy namespace: default spec: replicas: 3 selector: mathLables: app: myapp release: canary template: metadata: labels: app: myapp release: canary spec: containers: - name: myapp image: ikubernets/app: v2 ports: - http: containerPort: 80 services-nodeport.yaml apiVersion: v1 kind: Service metadata: name: ingress-nginx namespace: ingress-nginx spec: type: Nodeport ports: - name: http port: 80 targetPort: 80 protocol: TCP nodePort: 30080 - name: https port: 443 targetPort: 443 protocol: TCP nodePort: 300443 selector: app: ingress-nginx 基于ingress的后端app应用： myapp-ingress.yaml apiVsersion:extensions/v1beta1 kind:Ingress metadata: name: myapp-ingress namespace: default annotations: kubernetes.io/ingress.class: "nginx" spec: rules: - host: rancher.domain.com http: path: - path: backend: serviceName:myapp servicePort: 80 # kubectl apply -f myapp-ingress.yaml # kubectl get ingress 在浏览器访问：rancher.domain.com:30080 tomcat应用： tomcat-deploy.yaml apiVersion: v1 kind: Service metadata: name: tomcat namespace: default spec: selector: app: tomcat release: canary ports: - name: http port: 8080 targetPort: 8080 - name: ajp port: 8009 targetPort: 8009 --- apiVersion: v1 kind: Deployment metadata: name: tomcat-deploy namespace: default spec: replicas: 3 selector: matchLabels: app: tomcat release: canary template: metadata: labels: app: tomcat release: canary spec: containers: - name: tomcat image: tomcat:8.5.32-jre8-alpine ports: - name: http containerPort: 8080 - name: ajp containerPort: 8009 ingress-tomcat.yaml apiVsersion:extensions/v1beta1 kind:Ingress metadata: name: ingress-tomcat namespace: default annotations: kubernetes.io/ingress.class: "nginx" spec: rules: - host: tomcat.domain.com http: path: - path: backend: serviceName:tomcat servicePort: 8080 # kubectl apply -f tomcat-deploy.yaml # kubectl apply -f ingress-tomcat.yaml # kuebctl get pod # kubectl get svc # kubectl get ingress NAME HOSTS ADDRESS PORTS AGE ingtess-tomcat tomcat.domain.com 80 10m 浏览器访问：tomcat.domain.com:30080 基于https： # kubectl secret ingress-tomcat-tls.yaml apiVsersion:extensions/v1beta1 kind:Ingress metadata: name: ingress-tomcat-tls namespace: default annotations: kubernetes.io/ingress.class: "nginx" spec: tls: - hosts: - tomcat.domain.com secretName: tomcat-ingress.secret rules: - host: tomcat.domain.com http: path: - path: backend: serviceName:tomcat servicePort: 8080 # kubectl apply -f ingress-tomcat-tls.yaml # kuebctl get ingress NAME HOSTS ADDRESS PORTS AGE ingtess-tomcat tomcat.domain.com 80,443 10m ingtess-tomcat-tls tomcat.domain.com 80,443 1m 浏览器访问: https:tomcat.domain.com:30443 # kubectl get pods -n ingress nginx-ingress-controller--d658896cd-t55dm # kubectl exec -it nginx-ingress-controller--d658896cd-t55dm -n ingress cat nginx.conf