Kubernetes prometheus+grafana k8s 监控

参考：
https://www.cnblogs.com/terrycy/p/10058944.html

https://www.cnblogs.com/weiBlog/p/10629966.html

用到的命令和遇到的坑：

https://www.cnblogs.com/terrycy/p/10058944.html
https://www.cnblogs.com/weiBlog/p/10629966.html

monitor-namespace.yaml
cluster-role.yaml
config-map.yaml
prometheus-deployment.yaml

第一个坑，k8s 1.16以后的版本API变了，详情看
https://www.infoq.cn/article/rrJ5IGIAXy4V-X2Hb0QA
no matches for kind "DaemonSet" in version "extensions/v1beta1"
no matches for kind "Deployment" in version "extensions/v1beta1"

第二个坑新版本的 apps.v1 API需要在yaml文件中，selector变为必选项
https://www.cnblogs.com/robinunix/p/11155860.html
error: error validating "prometheus-deployment.yaml": error validating data: ValidationError(Deployment.spec): missing required field "selector" in io.k8s.api.apps.v1.DeploymentSpec; if you choose to ignore these errors, turn validation off with --validate=false

prometheus-deployment.yaml文件如下：
apiVersion: apps/v1
kind: Deployment
metadata:
  name: prometheus-deployment
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app: prometheus-server
  replicas: 1
  template:
    metadata:
      labels:
        app: prometheus-server
    spec:
      containers:
        - name: prometheus
          image: prom/prometheus:v2.3.2
          args:
            - "--config.file=/etc/prometheus/prometheus.yml"
            - "--storage.tsdb.path=/prometheus/"
          ports:
            - containerPort: 9090
          volumeMounts:
            - name: prometheus-config-volume
              mountPath: /etc/prometheus/
            - name: prometheus-storage-volume
              mountPath: /prometheus/
      volumes:
        - name: prometheus-config-volume
          configMap:
            defaultMode: 420
            name: prometheus-server-conf
        - name: prometheus-storage-volume
          emptyDir: {}

kubectl create -f node-exporter.yaml
kubectl create -f rbac-setup.yaml
kubectl create -f configmap.yaml
kubectl create -f promethues.yaml

kubectl get pod -n kube-system
通过kubectl describe命令查看具体信息（或查看日志/var/log/message）
kubectl describe pod kubernetes-dashboard-2094756401-kzhnx --namespace=kube-system

修改docker为国内镜像
vi /etc/docker/daemon.json
{
  "registry-mirrors": ["https://6kx4zyno.mirror.aliyuncs.com","https://registry.docker-cn.com","http://hub-mirror.c.163.com"],
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}

service docker restart

同步系统时间
yum install ntp
ntpdate cn.pool.ntp.org
ntpdate 0.centos.pool.ntp.org
ln -s ../usr/share/zoneinfo/Asia/Shanghai /etc/localtime
date
clock -w
clock --hctosys