IngressController的session stick问题

周末爬坑，IngressController恢复因为镜像下载和版本问题折腾一下午，晚上终于折腾出个眉目。

首先，在Kubernetes的service中是可以设置Session Affinity的。例子如下：

[root@k8s-master ~]# cat rc.yaml
apiVersion: v1
kind: ReplicationController
metadata:
  name: helloworld-service
spec:
  replicas: 2 
  template:
    metadata:
      labels:
        weblogic-app: "helloworld"
        version: "0.1"
    spec:
      containers:
      - name: weblogichelloworld
        image: 1213-helloworld:v2
        ports:
        - containerPort: 7001
---
apiVersion: v1
kind: Service
metadata:
  name: helloworldsvc
  labels:
    weblogic-app: helloworld
spec:
  type: NodePort
  ports:
  - port: 7001
    protocol: TCP
    targetPort: 7001
    name: http
    nodePort: 30005
  selector:
    weblogic-app: helloworld
  sessionAffinity: ClientIP

最核心就是最后那句，会基于客户端访问服务的ip进行hash运算后把同一ip的请求路由到同一个pod.这样通过nodePort方式过来的请求就不会到处分发了。

但这并不意味着通过IngressController过来的请求不会到处发，实际上在gcr.io/google_containers/nginx-ingress-controller:0.61版本就加上了session stick的功能，需要你在

创建Ingress对象的时候添加Annotation,比如:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: dashboard-weblogic-ingress
  annotations:
    ingress.kubernetes.io/affinity: "cookie"
    ingress.kubernetes.io/session-cookie-name: "route"
    ingress.kubernetes.io/session-cookie-hash: "sha1"
spec:
  rules:
  - host: helloworld.paic.test
    http:
      paths:
      - backend:
          serviceName: helloworldsvc 
          servicePort: 7001
        path: /

如何验证这个annotation已经生效了呢？这个问题我折腾了很久，因为按照文档的说法通过

curl -v http://<ingress-svc-address> -H 'Host: example.com'

kubectl  exec -n kube-system nginx-ingress-lb-303jx  -- curl -v localhost  -H'Host: helloworld.test' 

全部返回的是404,503,并没有以下的cookie

< Set-Cookie: route=dc89ae303c62a8bfce8bf32f06d27c31f0980ef7; Path=/; HttpOnly

无奈进入IngressController,查看具体的最终Nginx配置

在 /etc/nginx/nginx.conf 中。

 而Pod的信息是

 可见Nginx是绕开了Service,直接把pod的ip写入到配置文件，所以也就是Service中的Session Affinity设置应该不起作用。

但Cookie到底有没有设置成功呢，通过chrome访问网站，然后查看Cookie

 成功看到，验证的时候也没问题，所以应该配置生效。

最后贴一个ingress-controller的yaml

[root@k8s-master ingress]# cat nginx-ingress-controller.yaml 
apiVersion: v1
kind: ReplicationController
metadata:
  name: nginx-ingress-lb
  labels:
    name: nginx-ingress-lb
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: nginx-ingress-lb
      annotations:
        prometheus.io/port: '10254'
        prometheus.io/scrape: 'true'
    spec:
      terminationGracePeriodSeconds: 60
      hostNetwork: true
      containers:
      - image: gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.7
        name: nginx-ingress-lb
        readinessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
        livenessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          timeoutSeconds: 1
        ports:
        - containerPort: 80
          hostPort: 80
        - containerPort: 443
          hostPort: 443
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: KUBERNETES_MASTER
            value: http://192.168.0.104:8080
        args:
        - /nginx-ingress-controller
        - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
        - --apiserver-host=http://192.168.0.104:8080