kubernetes1.5.2--部署监控服务

本文基于kubernetes 1.5.2版本编写

Heapster是kubernetes集群监控工具。在1.2的时候，kubernetes的监控需要在node节点上运行cAdvisor作为agent收集本机和容器的资源数据，包括cpu、内存、网络、文件系统等。在新版的kubernetes中，cAdvisor被集成到kubelet中。通过netstat可以查看到kubelet新开了一个4194的端口，这就是cAdvisor监听的端口，现在我们然后可以通过http://<node-ip>:4194的方式访问到cAdvisor。Heapster就是通过每个node上的kubelet，也就是实际的cAdvisor上收集数据并汇总，保存到后端存储中。

Heapster支持多种后端存储，包括influxDB，Elasticsearch，Kafka等，在这篇文档里，我们使用influxDB作为后端存储来展示heapster的相关配置。需要说明的是，heapster依赖kubernetes dns配置。

heapster

使用http访问API Server

cat heapster-controller.yaml
apiVersion: v1
kind: ReplicationController
metadata:
  labels:
    k8s-app: heapster
    name: heapster
    version: v6
  name: heapster
  namespace: kube-system
  selfLink: /api/v1/namespaces/kube-system/replicationcontrollers/heapster
spec:
  replicas: 1
  selector:
    k8s-app: heapster
    version: v6
  template:
    metadata:
      creationTimestamp: null
      labels:
        k8s-app: heapster
        version: v6
    spec:
      containers:
      - command:
        - /heapster
        - --source=kubernetes:http://192.168.20.128:8080?inClusterConfig=false
        - --sink=influxdb:http://172.17.114.120:8086
        image: kubernetes/heapster:canary
        imagePullPolicy: IfNotPresent
        name: heapster
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      securityContext: {}
      terminationGracePeriodSeconds: 30

--source代表heapster的数据源，即从哪里获取数据，这里当然是从apiserver拿数据。 改成如下配置：--source=kubernetes:http://apiserver地址:8080?inClusterConfig=false

inClusterConfig=false代表不使用service accounts中的kube config信息。 (default: true)
kubeletPort - 指定kubelet端口
kubeletHttps - 是否使用httos访问kubelet(default: false)
apiVersion - API版本号
insecure - 是否使用https访问API Server (default: false)
auth - 指定客户端权限文件
useServiceAccount - whether to use the service account token if one is mounted at /var/run/secrets/kubernetes.io/serviceaccount/token (default: false)

--sink代表heapster获取到的数据存储到哪里，我们这里使用了influxdb，influxdb的地址在influxdb相关的yaml文件中有定义，可以自行查看。

使用https访问API Server

apiVersion: v1
kind: ReplicationController
metadata:
  labels:
    k8s-app: heapster
    name: heapster
    version: v6
  name: heapster
  namespace: kube-system
spec:
  replicas: 1
  selector:
    k8s-app: heapster
    version: v6
  template:
    metadata:
      labels:
        k8s-app: heapster
        version: v6
    spec:
      containers:
      - name: heapster
        image: kubernetes/heapster:canary
        imagePullPolicy:  IfNotPresent
        command:
        - /heapster
        - --source=kubernetes:https://192.168.20.128:6443?inClusterConfig=false&auth=/etc/kubernetes/kubeconfig/kubeconfig
        - --sink=influxdb:http://172.17.114.120:8086
        volumeMounts:#挂载认证需要的信息配置文件和证书
        - name: config
          mountPath: /etc/kubernetes/kubeconfig/kubeconfig
          readOnly: True
        - name: certs
          mountPath: /etc/ssl/kube
          readOnly: True
      volumes:#挂载宿主机的信息配置文件和证书
      - name: certs
        hostPath:
          path: /etc/ssl/kube
      - name: config
        hostPath:
          path: /etc/kubernetes/kubelet-config

service

cat heapster-service.yaml 
apiVersion: v1
kind: Service
metadata:
  labels:
    kubernetes.io/cluster-service: 'true'
    kubernetes.io/name: Heapster
  name: heapster
  namespace: kube-system
spec:
  ports:
  - port: 80
    targetPort: 8082
  selector:
    k8s-app: heapster

influxdb和grafana

cat influxdb-grafana-controller.yaml 
apiVersion: v1
kind: ReplicationController
metadata:
  labels:
    name: influxGrafana
  name: influxdb-grafana
  namespace: kube-system
  selfLink: /api/v1/namespaces/kube-system/replicationcontrollers/influxdb-grafana
spec:
  replicas: 1
  selector:
    name: influxGrafana
  template:
    metadata:
      creationTimestamp: null
      labels:
        name: influxGrafana
    spec:
      containers:
      - image: kubernetes/heapster_influxdb:v0.5
        imagePullPolicy: IfNotPresent
        name: influxdb
        resources: {}
        terminationMessagePath: /dev/termination-log
        volumeMounts:
        - mountPath: /data
          name: influxdb-storage
      - env:
        - name: INFLUXDB_SERVICE_URL
          value: http://192.168.20.128:8080
        - name: GF_AUTH_BASIC_ENABLED
          value: "false"
        - name: GF_AUTH_ANONYMOUS_ENABLED
          value: "true"
        - name: GF_AUTH_ANONYMOUS_ORG_ROLE
          value: Admin
        - name: GF_SERVER_ROOT_URL
          value: /api/v1/proxy/namespaces/kube-system/services/monitoring-grafana/
        image: docker.io/googlecontainer/heapster_grafana:v2.6.0-2
        imagePullPolicy: Never
        name: grafana
        resources: {}
        terminationMessagePath: /dev/termination-log
        volumeMounts:
        - mountPath: /var
          name: grafana-storage
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      securityContext: {}
      terminationGracePeriodSeconds: 30
      volumes:
      - emptyDir: {}
        name: influxdb-storage
      - emptyDir: {}
        name: grafana-storage

cat grafana-service.yaml 
apiVersion: v1
kind: Service
metadata:
  labels:
    kubernetes.io/cluster-service: 'true'
    kubernetes.io/name: monitoring-grafana
  name: monitoring-grafana
  namespace: kube-system
spec:
  ports:
  - port: 80
    targetPort: 3000
  selector:
    name: influxGrafana

cat influxdb-service.yaml 
apiVersion: v1
kind: Service
metadata:
  labels: null
  name: monitoring-influxdb
  namespace: kube-system
spec:
  clusterIP: 172.17.114.120#指定clusterIP地址，用于headster使用
  ports:
  - name: http
    port: 8083
    targetPort: 8083
  - name: api
    port: 8086
    targetPort: 8086
  selector:
    name: influxGrafana

部署

kubectl create -f influxdb-grafana-controller.yaml

kubectl create -f influxdb-service.yaml

kubectl create -f grafana-service.yaml

kubectl create -f heapster-service.yaml

kubectl create -f heapster-controller.yaml

测试

部署完之后，等待几分钟访问API Server

http：http://192.168.20.128:8080/ui

https：https://192.168.20.128:6443/ui

每个资源对象上是否有资源监控图。