Kerbernetes的Ingress资源管理
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.Ingress概述
管理对集群中服务的外部访问的API对象，通常是HTTP。Ingress是允许入站连接访问群集服务的规则集合。 Ingress可以配置为提供外部可接收的url、负载平衡通信、SSL终端和基于名称的虚拟主机等功能。 Ingress其实也是一个控制器(Controller)，只不过它并不被"kube-controller-manager"打包管理,而且一般情况下Ingress作为Pod来运行。 Ingress仅是用于定义流量转发和调度的通用格式的配置信息，它们需要转换为特定的具有http协议转发和调度功能的应用程序(例如nginx,haproxy,traeik等)的配置文件，并由相应的应用程序生效相应的配置后完成流量转发。 此类能理解Ingress定义的配置信息，并可将其转换为自身配置的应用程序，即为Ingress Controller。 此类的控制区需要由Kubernetes管理员额外以Addons的形式部署为Pod资源对象,它们通过API Server获取Ingress的相关定义； 这与其他类型的控制器不同，它们通常作为"kube-controller-manager"二进制文件的一部分运行，并且通常作为集群创建的一部分自动启动; 选择最适合集群的入口控制器(ingress controller)实现，或者实现一个新的入口控制器(ingress controller)，Kubernetes目前支持并维护GCE和nginx控制器(https://github.com/kubernetes/ingress-nginx)。 Ingress自身不支持使用标签选择器挑选真正提供服务的Pod对象，因此，它需要由Service对象的辅助完成此类功能(如借助Service的标签选择器功能过滤出后端的Pod)。 Ingress自身不运行使用标签选择器挑选真正提供服务的Pod对象,它需要由Service对象的辅助完成此类功能。 Ingress Controller根据Igress定义的配置调度流量时,其报文将由Ingress Controller直接调度后直达Pod对象,而不再经由Service调度。 Ingress Controller也是Pod对象，它能够与各后端Pod直接进行通信。 Ingres官方文档: https://kubernetes.io/docs/concepts/services-networking/ingress/
二.在K8S集群部署Ingress-nginx
1>.参考kubernetes的官方文档(https://github.com/kubernetes/ingress-nginx/blob/master/docs/deploy/index.md)
2>.部署ingress-nginx
[root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get namespace NAME STATUS AGE default Active 3d17h kube-node-lease Active 3d17h kube-public Active 3d17h kube-system Active 3d17h myservice Active 4h19m testing Active 21h testing2 Active 11h [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.28.0/deploy/static/mandatory.yaml namespace/ingress-nginx created configmap/nginx-configuration created configmap/tcp-services created configmap/udp-services created serviceaccount/nginx-ingress-serviceaccount created clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created role.rbac.authorization.k8s.io/nginx-ingress-role created rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created deployment.apps/nginx-ingress-controller created limitrange/ingress-nginx created [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get namespace NAME STATUS AGE default Active 3d17h ingress-nginx Active 11s kube-node-lease Active 3d17h kube-public Active 3d17h kube-system Active 3d17h myservice Active 4h20m testing Active 21h testing2 Active 11h [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n ingress-nginx -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-ingress-controller-5556bd798f-hhmhn 1/1 Running 0 47s 10.244.3.5 node203.yinzhengjie.org.cn <none> <none> [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
3>.根据上一步创建的Ingress-nginx的label创建Service资源
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n ingress-nginx --show-labels NAME READY STATUS RESTARTS AGE LABELS nginx-ingress-controller-5556bd798f-hhmhn 1/1 Running 0 9m56s app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx,pod-template-hash=5556bd798f [root@master200.yinzhengjie.org.cn ~]# root@master200.yinzhengjie.org.cn ~]# kubectl get service -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE my-ingress NodePort 10.104.132.19 <none> 80:31080/TCP,443:31910/TCP 33s [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/ingress/ingress-nginx-service.yaml [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/ingress/ingress-nginx-service.yaml apiVersion: v1 kind: Service metadata: name: my-ingress namespace: ingress-nginx spec: selector: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx ports: - name: http port: 80 nodePort: 30080 - name: https port: 443 nodePort: 30443 type: NodePort [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/ingress/ingress-nginx-service.yaml service/my-ingress configured [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get service -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE my-ingress NodePort 10.104.132.19 <none> 80:30080/TCP,443:30443/TCP 2m58s [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# hostname -i #如下图所示，可以通过NodePort方式访问到Ingress-nginx则说明部署成功了，我们可以清晰的看到nginx的版本,接下来咱们就该配置Ingress-nginx了。 172.200.1.200 [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
三.应用Ingress-nginx实战案例
1>.创建后端Pod及Service
[root@master200.yinzhengjie.org.cn ~]# kubectl get ns NAME STATUS AGE default Active 3d23h ingress-nginx Active 5h4m kube-node-lease Active 3d23h kube-public Active 3d23h kube-system Active 3d23h myservice Active 9h testing Active 26h testing2 Active 16h [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl create namespace yinzhengjie-ns namespace/yinzhengjie-ns created [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get ns NAME STATUS AGE default Active 3d23h ingress-nginx Active 5h6m kube-node-lease Active 3d23h kube-public Active 3d23h kube-system Active 3d23h myservice Active 9h testing Active 26h testing2 Active 16h yinzhengjie-ns Active 2s [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/pod/yinzhengjie-ns-ingress-example.yaml [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/pod/yinzhengjie-ns-ingress-example.yaml apiVersion: apps/v1 kind: Deployment metadata: name: mynginx namespace: yinzhengjie-ns spec: replicas: 2 selector: matchLabels: app: mynginx rel: beta template: metadata: namespace: yinzhengjie-ns labels: app: mynginx rel: beta spec: containers: - name: mynginx image: nginx:1.14-alpine --- apiVersion: v1 kind: Service metadata: name: myapp namespace: yinzhengjie-ns spec: selector: app: myapp rel: beta ports: - name: http port: 80 targetPort: 80 [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/pod/yinzhengjie-ns-ingress-example.yaml deployment.apps/mynginx created service/myapp created [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get service -n yinzhengjie-ns -o wide --show-labels NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR LABELS myapp ClusterIP 10.102.191.103 <none> 80/TCP 114s app=myapp,rel=beta <none> [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get pod -n yinzhengjie-ns -o wide --show-labels NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS mynginx-c49cd4658-ks45t 1/1 Running 0 2m10s 10.244.1.12 node201.yinzhengjie.org.cn <none> <none> app=mynginx,pod-template-hash=c49cd4658,rel=beta mynginx-c49cd4658-l2676 1/1 Running 0 2m10s 10.244.3.6 node203.yinzhengjie.org.cn <none> <none> app=mynginx,pod-template-hash=c49cd4658,rel=beta [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get all -n yinzhengjie-ns -o wide --show-labels NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS pod/mynginx-c49cd4658-ks45t 1/1 Running 0 3m4s 10.244.1.12 node201.yinzhengjie.org.cn <none> <none> app=mynginx,pod-template-hash=c49cd4658,rel=beta pod/mynginx-c49cd4658-l2676 1/1 Running 0 3m4s 10.244.3.6 node203.yinzhengjie.org.cn <none> <none> app=mynginx,pod-template-hash=c49cd4658,rel=beta NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR LABELS service/myapp ClusterIP 10.102.191.103 <none> 80/TCP 3m4s app=myapp,rel=beta <none> NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR LABELS deployment.apps/mynginx 2/2 2 2 3m4s mynginx nginx:1.14-alpine app=mynginx,rel=beta <none> NAME DESIRED CURRENT READY AGE CONTAINERS IMAGES SELECTOR LABELS replicaset.apps/mynginx-c49cd4658 2 2 2 3m4s mynginx nginx:1.14-alpine app=mynginx,pod-template-hash=c49cd4658,rel=beta app=mynginx,pod-template-hash=c49cd4658,rel=beta [root@master200.yinzhengjie.org.cn ~]#
2>.创建Ingress资源,资源注解可参考官网(https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/annotations.md)
[root@master200.yinzhengjie.org.cn ~]# kubectl explain ingress KIND: Ingress VERSION: extensions/v1beta1 DESCRIPTION: Ingress is a collection of rules that allow inbound connections to reach the endpoints defined by a backend. An Ingress can be configured to give services externally-reachable urls, load balance traffic, terminate SSL, offer name based virtual hosting etc. DEPRECATED - This group version of Ingress is deprecated by networking.k8s.io/v1beta1 Ingress. See the release notes for more information. FIELDS: apiVersion <string> APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources kind <string> Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds metadata <Object> Standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata spec <Object> Spec is the desired state of the Ingress. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status status <Object> Status is the current state of the Ingress. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/ingress/myapp-ingress.yaml [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/ingress/myapp-ingress.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: myingress namespace: yinzhengjie-ns annotations: nginx.ingress.kubernetes.io/rewrite-target: / kubernetes.io/ingress.class: "nginx" spec: rules: - host: www.yinzhengjie.org.cn http: paths: - path: / backend: serviceName: myapp servicePort: 80 [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/ingress/myapp-ingress.yaml ingress.extensions/myingress created [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get ingress -n yinzhengjie-ns NAME HOSTS ADDRESS PORTS AGE myingress www.yinzhengjie.org.cn 80 4m [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get ingress -n yinzhengjie-ns -o yaml apiVersion: v1 items: - apiVersion: extensions/v1beta1 kind: Ingress metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/rewrite-target":"/"},"name":"myingress","namespace":"yinzhengjie-ns"},"spec":{"rules":[{"host":"www.yinzhengjie.org. cn","http":{"paths":[{"backend":{"serviceName":"myapp","servicePort":80},"path":"/"}]}}]}} kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/rewrite-target: / creationTimestamp: "2020-02-08T12:14:22Z" generation: 1 name: myingress namespace: yinzhengjie-ns resourceVersion: "260013" selfLink: /apis/extensions/v1beta1/namespaces/yinzhengjie-ns/ingresses/myingress uid: 1300a1ef-e059-411d-88e3-da1640477f3b spec: rules: - host: www.yinzhengjie.org.cn http: paths: - backend: serviceName: myapp servicePort: 80 path: / status: loadBalancer: {} kind: List metadata: resourceVersion: "" selfLink: "" [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl describe ingress -n yinzhengjie-ns Name: myingress Namespace: yinzhengjie-ns Address: Default backend: default-http-backend:80 (<none>) Rules: Host Path Backends ---- ---- -------- www.yinzhengjie.org.cn / myapp:80 (<none>) Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/rewrite-target":"/"},"name":"myingress","namespace":"yinzhengjie-ns"}," spec":{"rules":[{"host":"www.yinzhengjie.org.cn","http":{"paths":[{"backend":{"serviceName":"myapp","servicePort":80},"path":"/"}]}}]}} kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/rewrite-target: / Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 9m16s nginx-ingress-controller Ingress yinzhengjie-ns/myingress [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
3>.验证创建的Ingress资源
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n ingress-nginx NAME READY STATUS RESTARTS AGE nginx-ingress-controller-5556bd798f-hhmhn 1/1 Running 0 5h53m [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl exec -it nginx-ingress-controller-5556bd798f-hhmhn -n ingress-nginx -- /bin/sh /etc/nginx $ /etc/nginx $ ls -l total 88 -rw-r--r-- 1 www-data www-data 1077 Jan 14 17:09 fastcgi.conf -rw-r--r-- 1 www-data www-data 1077 Jan 14 17:09 fastcgi.conf.default -rw-r--r-- 1 www-data www-data 1007 Jan 14 17:09 fastcgi_params -rw-r--r-- 1 www-data www-data 1007 Jan 14 17:09 fastcgi_params.default drwxr-xr-x 2 www-data www-data 68 Jan 14 17:04 geoip -rw-r--r-- 1 www-data www-data 2837 Jan 14 17:09 koi-utf -rw-r--r-- 1 www-data www-data 2223 Jan 14 17:09 koi-win drwxr-xr-x 6 www-data www-data 267 Jan 28 11:13 lua -rw-r--r-- 1 www-data www-data 5231 Jan 14 17:09 mime.types -rw-r--r-- 1 www-data www-data 5231 Jan 14 17:09 mime.types.default drwxr-xr-x 2 www-data www-data 53 Jan 14 17:09 modsecurity drwxr-xr-x 2 www-data www-data 150 Jan 14 17:09 modules -rw-r--r-- 1 www-data www-data 17391 Feb 8 11:21 nginx.conf -rw-r--r-- 1 www-data www-data 2656 Jan 14 17:09 nginx.conf.default -rw-r--r-- 1 www-data www-data 2 Jan 28 11:13 opentracing.json drwxr-xr-x 6 www-data www-data 301 Jan 14 17:09 owasp-modsecurity-crs -rw-r--r-- 1 www-data www-data 636 Jan 14 17:09 scgi_params -rw-r--r-- 1 www-data www-data 636 Jan 14 17:09 scgi_params.default drwxr-xr-x 2 www-data www-data 24 Jan 28 11:13 template -rw-r--r-- 1 www-data www-data 664 Jan 14 17:09 uwsgi_params -rw-r--r-- 1 www-data www-data 664 Jan 14 17:09 uwsgi_params.default -rw-r--r-- 1 www-data www-data 3610 Jan 14 17:09 win-utf /etc/nginx $ /etc/nginx $ /etc/nginx $ more nginx.conf #如下图所示，如果看到虚拟主机信息就ok啦~我们就可以在浏览器通过"www.yinzhengjie.org.cn:30080"或者"www.yinzhengjie.org.cn:300443"进行访问啦
四.使用Ingress配置SSL(TLS)
1>.自建证书
[root@master200.yinzhengjie.org.cn ~]# cd /yinzhengjie/data/k8s/manifests/basic/ingress/ [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# ll total 8 -rw-r--r-- 1 root root 314 Feb 8 13:50 ingress-nginx-service.yaml -rw-r--r-- 1 root root 366 Feb 8 20:15 myapp-ingress.yaml [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# openssl genrsa -out myapp.key 2048 Generating RSA private key, 2048 bit long modulus .............................................................................+++ .........+++ e is 65537 (0x10001) [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# ll total 12 -rw-r--r-- 1 root root 314 Feb 8 13:50 ingress-nginx-service.yaml -rw-r--r-- 1 root root 366 Feb 8 20:15 myapp-ingress.yaml -rw-r--r-- 1 root root 1675 Feb 8 20:42 myapp.key [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# ll total 12 -rw-r--r-- 1 root root 314 Feb 8 13:50 ingress-nginx-service.yaml -rw-r--r-- 1 root root 366 Feb 8 20:15 myapp-ingress.yaml -rw-r--r-- 1 root root 1675 Feb 8 20:42 myapp.key [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# openssl req -new -x509 -key myapp.key -out myapp.crt -subj /C=CN/ST=Beijing/L=Beijing/O=Ops/CN=master.yinzhengjie.org.cn -days 3650 [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# ll total 16 -rw-r--r-- 1 root root 314 Feb 8 13:50 ingress-nginx-service.yaml -rw-r--r-- 1 root root 1310 Feb 8 20:45 myapp.crt -rw-r--r-- 1 root root 366 Feb 8 20:15 myapp-ingress.yaml -rw-r--r-- 1 root root 1675 Feb 8 20:42 myapp.key [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
2>.配置证书资源
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# kubectl create secret -h Create a secret using specified subcommand. Available Commands: docker-registry Create a secret for use with a Docker registry generic Create a secret from a local file, directory or literal value tls Create a TLS secret Usage: kubectl create secret [flags] [options] Use "kubectl <command> --help" for more information about a given command. Use "kubectl options" for a list of global command-line options (applies to all commands). [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# kubectl create secret tls -h Create a TLS secret from the given public/private key pair. The public/private key pair must exist before hand. The public key certificate must be .PEM encoded and match the given private key. Examples: # Create a new TLS secret named tls-secret with the given key pair: kubectl create secret tls tls-secret --cert=path/to/tls.cert --key=path/to/tls.key Options: --allow-missing-template-keys=true: If true, ignore any errors in templates when a field or map key is missing in the template. Only applies to golang and jsonpath output formats. --append-hash=false: Append a hash of the secret to its name. --cert='': Path to PEM encoded public key certificate. --dry-run=false: If true, only print the object that would be sent, without sending it. --generator='secret-for-tls/v1': The name of the API generator to use. --key='': Path to private key associated with given certificate. -o, --output='': Output format. One of: json|yaml|name|go-template|go-template-file|template|templatefile|jsonpath|jsonpath-file. --save-config=false: If true, the configuration of current object will be saved in its annotation. Otherwise, the annotation will be unchanged. This flag is useful when you want to perform kubectl apply on this object in the future. --template='': Template string or path to template file to use when -o=go-template, -o=go-template-file. The template format is golang templates [http://golang.org/pkg/text/template/#pkg-overview]. --validate=true: If true, use a schema to validate the input before sending it Usage: kubectl create secret tls NAME --cert=path/to/cert/file --key=path/to/key/file [--dry-run] [options] Use "kubectl options" for a list of global command-line options (applies to all commands). [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# ll total 16 -rw-r--r-- 1 root root 314 Feb 8 13:50 ingress-nginx-service.yaml -rw-r--r-- 1 root root 1310 Feb 8 20:45 myapp.crt -rw-r--r-- 1 root root 366 Feb 8 20:15 myapp-ingress.yaml -rw-r--r-- 1 root root 1675 Feb 8 20:42 myapp.key [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# kubectl create secret tls yinzhengjie.org.cn-cert -n yinzhengjie-ns --cert=myapp.crt --key=myapp.key --dry-run secret/yinzhengjie.org.cn-cert created (dry run) [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# kubectl create secret tls yinzhengjie.org.cn-cert -n yinzhengjie-ns --cert=myapp.crt --key=myapp.key secret/yinzhengjie.org.cn-cert created [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]# [root@master200.yinzhengjie.org.cn /yinzhengjie/data/k8s/manifests/basic/ingress]#
[root@master200.yinzhengjie.org.cn ~]# kubectl describe secret yinzhengjie.org.cn-cert -n yinzhengjie-ns Name: yinzhengjie.org.cn-cert Namespace: yinzhengjie-ns Labels: <none> Annotations: <none> Type: kubernetes.io/tls Data ==== tls.crt: 1310 bytes tls.key: 1675 bytes [root@master200.yinzhengjie.org.cn ~]#
3>.创建Ingress资源
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/ingress/myapp-tls-ingress-example.yaml [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/ingress/myapp-tls-ingress-example.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: myapp-tls namespace: yinzhengjie-ns annotations: kubernetes.io/ingress.class: "nginx" spec: tls: - hosts: - master.yinzhengjie.org.cn secretName: yinzhengjie.org.cn-cert rules: - host: master.yinzhengjie.org.cn http: paths: - path: / backend: serviceName: myapp servicePort: 80 [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/ingress/myapp-tls-ingress-example.yaml ingress.extensions/myapp-tls created [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get ingress -n yinzhengjie-ns NAME HOSTS ADDRESS PORTS AGE myapp-tls master.yinzhengjie.org.cn 80, 443 19s myingress www.yinzhengjie.org.cn 80 47m [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get ingress -n yinzhengjie-ns NAME HOSTS ADDRESS PORTS AGE myapp-tls master.yinzhengjie.org.cn 80, 443 117s myingress www.yinzhengjie.org.cn 80 49m [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl describe ingress -n yinzhengjie-ns myapp-tls Name: myapp-tls Namespace: yinzhengjie-ns Address: Default backend: default-http-backend:80 (<none>) TLS: yinzhengjie.org.cn-cert terminates master.yinzhengjie.org.cn Rules: Host Path Backends ---- ---- -------- master.yinzhengjie.org.cn / myapp:80 (<none>) Annotations: kubernetes.io/ingress.class: nginx kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"myapp-tls","namespace":"yinzhengjie-ns"},"spec":{"rules":[{"host":"master.yinzhengjie.org.c n","http":{"paths":[{"backend":{"serviceName":"myapp","servicePort":80},"path":"/"}]}}],"tls":[{"hosts":["master.yinzhengjie.org.cn"],"secretName":"yinzhengjie.org.cn-cert"}]}} Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 2m3s nginx-ingress-controller Ingress yinzhengjie-ns/myapp-tls [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n ingress-nginx NAME READY STATUS RESTARTS AGE nginx-ingress-controller-5556bd798f-hhmhn 1/1 Running 0 7h27m [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl exec -it nginx-ingress-controller-5556bd798f-hhmhn -n ingress-nginx -- /bin/sh /etc/nginx $ /etc/nginx $ ls fastcgi.conf fastcgi_params geoip koi-win mime.types modsecurity nginx.conf opentracing.json scgi_params template uwsgi_params.default fastcgi.conf.default fastcgi_params.default koi-utf lua mime.types.default modules nginx.conf.default owasp-modsecurity-crs scgi_params.default uwsgi_params win-utf /etc/nginx $ /etc/nginx $ more nginx.conf #如下图所示，依旧可以看到证书相关的配置则说明咱们的配置成功啦~
五.通过Ingress反向代理Tomcat实战案例
1>.部署tomcat的Pod及其Service资源
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/pod/yinzhengjie-tomcat.yaml [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/pod/yinzhengjie-tomcat.yaml apiVersion: v1 kind: Namespace metadata: name: yinzhengjie-eshop --- apiVersion: apps/v1 kind: Deployment metadata: name: mytomcat namespace: yinzhengjie-eshop spec: replicas: 2 selector: matchLabels: app: tomcat rel: beta template: metadata: namespace: yinzhengjie-eshop labels: app: tomcat rel: beta spec: containers: - name: mytomcat image: tomcat-alpine --- apiVersion: v1 kind: Service metadata: name: mytomcat namespace: yinzhengjie-eshop spec: selector: app: tomcat rel: beta ports: - name: http port: 8080 targetPort: 8080 - name: ajp port: 8089 targetPort: 8089 [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/pod/yinzhengjie-tomcat.yaml namespace/yinzhengjie-eshop created deployment.apps/mytomcat created service/mytomcat created [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get ns NAME STATUS AGE default Active 4d2h ingress-nginx Active 8h kube-node-lease Active 4d2h kube-public Active 4d2h kube-system Active 4d2h myservice Active 12h testing Active 29h testing2 Active 20h yinzhengjie-eshop Active 3m25s yinzhengjie-ns Active 3h17m [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get pod -n yinzhengjie-eshop NAME READY STATUS RESTARTS AGE mytomcat-6876768796-45rqs 0/1 ImagePullBackOff 0 3m38s mytomcat-6876768796-pjkwd 0/1 ImagePullBackOff 0 3m38s [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get pod -n yinzhengjie-eshop -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES mytomcat-6876768796-45rqs 0/1 ErrImagePull 0 3m53s 10.244.1.21 node201.yinzhengjie.org.cn <none> <none> mytomcat-6876768796-pjkwd 0/1 ErrImagePull 0 3m53s 10.244.2.17 node202.yinzhengjie.org.cn <none> <none> [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl get service -n yinzhengjie-eshop -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR mytomcat ClusterIP 10.106.29.218 <none> 8080/TCP,8089/TCP 3m59s app=tomcat,rel=beta [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get service -n yinzhengjie-eshop NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE mytomcat ClusterIP 10.106.29.218 <none> 8080/TCP,8089/TCP 4m3s [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
2>.创建Igress资源
[root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/ingress/tomcat-ingress.yaml [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/ingress/tomcat-ingress.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: tomcat namespace: yinzhengjie-eshop annotations: nginx.ingress.kubernetes.io/rewrite-target: / kubernetes.io/ingress.class: "nginx" spec: rules: - host: mytomcat.yinzhengjie.org.cn http: paths: - path: / backend: serviceName: mytomcat servicePort: 8080 [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/ingress/tomcat-ingress.yaml ingress.extensions/tomcat created [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl get ingress -n yinzhengjie-eshop NAME HOSTS ADDRESS PORTS AGE tomcat mytomcat.yinzhengjie.org.cn 80 13s [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
[root@master200.yinzhengjie.org.cn ~]# kubectl describe ingress -n yinzhengjie-eshop Name: tomcat Namespace: yinzhengjie-eshop Address: Default backend: default-http-backend:80 (<none>) Rules: Host Path Backends ---- ---- -------- mytomcat.yinzhengjie.org.cn / mytomcat:8080 () Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx","nginx.ingress.kubernetes.io/rewrite-target":"/"},"name":"tomcat","namespace":"yinzhengjie-eshop"}," spec":{"rules":[{"host":"mytomcat.yinzhengjie.org.cn","http":{"paths":[{"backend":{"serviceName":"mytomcat","servicePort":8080},"path":"/"}]}}]}} kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/rewrite-target: / Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 38s nginx-ingress-controller Ingress yinzhengjie-eshop/tomcat [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]#
3>.验证配置是否生效
[root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n ingress-nginx NAME READY STATUS RESTARTS AGE nginx-ingress-controller-5556bd798f-hhmhn 1/1 Running 0 8h [root@master200.yinzhengjie.org.cn ~]# [root@master200.yinzhengjie.org.cn ~]# kubectl exec -it nginx-ingress-controller-5556bd798f-hhmhn -n ingress-nginx -- /bin/sh /etc/nginx $ /etc/nginx $ ls -l total 92 -rw-r--r-- 1 www-data www-data 1077 Jan 14 17:09 fastcgi.conf -rw-r--r-- 1 www-data www-data 1077 Jan 14 17:09 fastcgi.conf.default -rw-r--r-- 1 www-data www-data 1007 Jan 14 17:09 fastcgi_params -rw-r--r-- 1 www-data www-data 1007 Jan 14 17:09 fastcgi_params.default drwxr-xr-x 2 www-data www-data 68 Jan 14 17:04 geoip -rw-r--r-- 1 www-data www-data 2837 Jan 14 17:09 koi-utf -rw-r--r-- 1 www-data www-data 2223 Jan 14 17:09 koi-win drwxr-xr-x 6 www-data www-data 267 Jan 28 11:13 lua -rw-r--r-- 1 www-data www-data 5231 Jan 14 17:09 mime.types -rw-r--r-- 1 www-data www-data 5231 Jan 14 17:09 mime.types.default drwxr-xr-x 2 www-data www-data 53 Jan 14 17:09 modsecurity drwxr-xr-x 2 www-data www-data 150 Jan 14 17:09 modules -rw-r--r-- 1 www-data www-data 24546 Feb 8 14:09 nginx.conf -rw-r--r-- 1 www-data www-data 2656 Jan 14 17:09 nginx.conf.default -rw-r--r-- 1 www-data www-data 2 Jan 28 11:13 opentracing.json drwxr-xr-x 6 www-data www-data 301 Jan 14 17:09 owasp-modsecurity-crs -rw-r--r-- 1 www-data www-data 636 Jan 14 17:09 scgi_params -rw-r--r-- 1 www-data www-data 636 Jan 14 17:09 scgi_params.default drwxr-xr-x 2 www-data www-data 24 Jan 28 11:13 template -rw-r--r-- 1 www-data www-data 664 Jan 14 17:09 uwsgi_params -rw-r--r-- 1 www-data www-data 664 Jan 14 17:09 uwsgi_params.default -rw-r--r-- 1 www-data www-data 3610 Jan 14 17:09 win-utf /etc/nginx $ /etc/nginx $ /etc/nginx $ more nginx.conf
