简单介绍:
如果使用kubernetes的自动扩容功能的话,那首先得有一个插件,然后该插件将收集到的信息(cpu、memory..)与自动扩容的设置的值进行比对,自动调整pod数量。关于该插件,在kubernetes的早些版本中采用的是heapster,1.13版本正式发布后,丢弃了heapster,官方推荐采用metrics-sever。
测试环境:
同之前部署的集群环境,如下:
System | Hostname | IP |
CentOS 7.6 | k8s-master | 138.138.82.14 |
CentOS 7.6 | k8s-node1 | 138.138.82.15 |
CentOS 7.6 | k8s-node2 | 138.138.82.16 |
操作步骤:
1. 下载相关yaml文件
https://github.com/kubernetes-incubator/metrics-server
~]# git clone https://github.com/kubernetes-incubator/metrics-server.git
[root@k8s-master ~]# cd metrics-server/deploy/1.8+/ [root@k8s-master 1.8+]# ll 总用量 28 -rw-r--r-- 1 root root 384 4月 28 09:46 aggregated-metrics-reader.yaml -rw-r--r-- 1 root root 308 4月 28 09:46 auth-delegator.yaml -rw-r--r-- 1 root root 329 4月 28 09:46 auth-reader.yaml -rw-r--r-- 1 root root 298 4月 28 09:46 metrics-apiservice.yaml -rw-r--r-- 1 root root 815 4月 28 09:46 metrics-server-deployment.yaml -rw-r--r-- 1 root root 291 4月 28 09:46 metrics-server-service.yaml -rw-r--r-- 1 root root 502 4月 28 09:46 resource-reader.yaml
2. 修改其中的metrics-server-deployment.yaml文件(用红色标亮处)
--- apiVersion: v1 kind: ServiceAccount metadata: name: metrics-server namespace: kube-system --- apiVersion: extensions/v1beta1 kind: Deployment metadata: name: metrics-server namespace: kube-system labels: k8s-app: metrics-server spec: selector: matchLabels: k8s-app: metrics-server template: metadata: name: metrics-server labels: k8s-app: metrics-server spec: serviceAccountName: metrics-server volumes: # mount in tmp so we can safely use from-scratch images and/or read-only containers - name: tmp-dir emptyDir: {} containers: - name: metrics-server image: mirrorgooglecontainers/metrics-server-amd64:v0.3.2 # image: k8s.gcr.io/metrics-server-amd64:v0.3.2 imagePullPolicy: IfNotPresent command: - /metrics-server - --kubelet-preferred-address-types=InternalIP - --kubelet-insecure-tls volumeMounts: - name: tmp-dir mountPath: /tmp
注意:以上修改了3个地方:①是被墙的镜像;②镜像拉取策略;③添加命令和相关参数;
如果不修改command区域的参数则会报如下错误:
E0428 19:35:05.894325 1 manager.go:111] unable to fully collect metrics: [unable to fully scrape metrics from source kubelet_summary:k8s-node2: unable to fetch metrics from Kubelet k8s-node2 (138.138.82.16): Get https://138.138.82.16:10250/stats/summary/: x509: cannot validate certificate for 138.138.82.16 because it doesn't contain any IP SANs, unable to fully scrape metrics from source kubelet_summary:k8s-node1: unable to fetch metrics from Kubelet k8s-node1 (138.138.82.15): Get https://138.138.82.15:10250/stats/summary/: x509: cannot validate certificate for 138.138.82.15 because it doesn't contain any IP SANs, unable to fully scrape metrics from source kubelet_summary:k8s-master: unable to fetch metrics from Kubelet k8s-master (138.138.82.14): Get https://138.138.82.14:10250/stats/summary/: x509: cannot validate certificate for 138.138.82.14 because it doesn't contain any IP SANs]
附上metrcs-server命令参数详解:
Usage: [flags] Flags: --alsologtostderr log to standard error as well as files --authentication-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create tokenaccessreviews.authentication.k8s.io. --authentication-skip-lookup If false, the authentication-kubeconfig will be used to lookup missing authentication configuration from the cluster. --authentication-token-webhook-cache-ttl duration The duration to cache responses from the webhook token authenticator. (default 10s) --authorization-kubeconfig string kubeconfig file pointing at the 'core' kubernetes server with enough rights to create subjectaccessreviews.authorization.k8s.io. --authorization-webhook-cache-authorized-ttl duration The duration to cache 'authorized' responses from the webhook authorizer. (default 10s) --authorization-webhook-cache-unauthorized-ttl duration The duration to cache 'unauthorized' responses from the webhook authorizer. (default 10s) --bind-address ip The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank, all interfaces will be used (0.0.0.0 for all IPv4 interfaces and :: for all IPv6 interfaces). (default 0.0.0.0) --cert-dir string The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored. (default "apiserver.local.config/certificates") --client-ca-file string If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate. --contention-profiling Enable lock contention profiling, if profiling is enabled --enable-swagger-ui Enables swagger ui on the apiserver at /swagger-ui -h, --help help for this command --http2-max-streams-per-connection int The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default. --kubeconfig string The path to the kubeconfig used to connect to the Kubernetes API server and the Kubelets (defaults to in-cluster config) --kubelet-certificate-authority string Path to the CA to use to validate the Kubelet's serving certificates. --kubelet-insecure-tls Do not verify CA of serving certificates presented by Kubelets. For testing purposes only. --kubelet-port int The port to use to connect to Kubelets. (default 10250) --kubelet-preferred-address-types strings The priority of node address types to use when determining which address to use to connect to a particular node (default [Hostname,InternalDNS,InternalIP,ExternalDNS,ExternalIP]) --log-flush-frequency duration Maximum number of seconds between log flushes (default 5s) --log_backtrace_at traceLocation when logging hits line file:N, emit a stack trace (default :0) --log_dir string If non-empty, write log files in this directory --logtostderr log to standard error instead of files (default true) --metric-resolution duration The resolution at which metrics-server will retain metrics. (default 1m0s) --profiling Enable profiling via web interface host:port/debug/pprof/ (default true) --requestheader-allowed-names strings List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed. --requestheader-client-ca-file string Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests. --requestheader-extra-headers-prefix strings List of request header prefixes to inspect. X-Remote-Extra- is suggested. (default [x-remote-extra-]) --requestheader-group-headers strings List of request headers to inspect for groups. X-Remote-Group is suggested. (default [x-remote-group]) --requestheader-username-headers strings List of request headers to inspect for usernames. X-Remote-User is common. (default [x-remote-user]) --secure-port int The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all. (default 443) --stderrthreshold severity logs at or above this threshold go to stderr (default 2) --tls-cert-file string File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir. --tls-cipher-suites strings Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be use. Possible values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_RC4_128_SHA,TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_RC4_128_SHA,TLS_RSA_WITH_3DES_EDE_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_CBC_SHA256,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_RC4_128_SHA --tls-min-version string Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12 --tls-private-key-file string File containing the default x509 private key matching --tls-cert-file. --tls-sni-cert-key namedCertKey A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com". (default []) -v, --v Level log level for V logs --vmodule moduleSpec comma-separated list of pattern=N settings for file-filtered logging
3. 应用所有配置文件到系统中
[root@k8s-master 1.8+]# kubectl apply -f . clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created serviceaccount/metrics-server created deployment.extensions/metrics-server created service/metrics-server created clusterrole.rbac.authorization.k8s.io/system:metrics-server created clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created
过个一两分钟(下载镜像和获取数据都会耗时)检查metrics-server的状态
[root@k8s-master 1.8+]# kubectl get po -n kube-system NAME READY STATUS RESTARTS AGE calico-node-b78m4 1/1 Running 0 176m calico-node-r5mlj 1/1 Running 0 3h6m calico-node-z5tdh 1/1 Running 0 176m coredns-fb8b8dccf-6mgks 1/1 Running 0 3h21m coredns-fb8b8dccf-cbtlx 1/1 Running 0 3h21m etcd-k8s-master 1/1 Running 0 3h20m kube-apiserver-k8s-master 1/1 Running 0 3h20m kube-controller-manager-k8s-master 1/1 Running 0 3h20m kube-proxy-c9xd2 1/1 Running 0 3h21m kube-proxy-fp2r2 1/1 Running 0 176m kube-proxy-lrsw7 1/1 Running 0 176m kube-scheduler-k8s-master 1/1 Running 0 3h20m metrics-server-7579f696d8-pgcc4 1/1 Running 0 99s [root@k8s-master 1.8+]# kubectl top node NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% k8s-master 179m 8% 1660Mi 43% k8s-node1 81m 4% 908Mi 23% k8s-node2 78m 3% 1036Mi 26%
看的出来,metrics-server已经正常running,并且能够获取节点的信息。
结束.