trading# k8s生态下 kubectl命令操作pod性能监控
环境
1C2G 1POD
架构拓扑
查看命名空间，ns就是namespaces
[root@pr-dr-13-47 .kube]# kubectl get ns
NAME STATUS AGE
default Active 327d
itwork Active 326d
kube-public Active 327d
kube-system Active 327d
projectName-dev Active 326d
projectName-sit Active 323d
aaaaa-company-dev Active 326d
aaaaa-company-sit Active 326d
uat Active 255d
查看Node或Pod的资源使用情况，需要在集群中运行Metrics Service
[root@pr-dr-13-47 .kube]# kubectl top nodes
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
100.20.30.40 152m 1% 24004Mi 83%
100.20.30.41 225m 1% 24515Mi 85%
100.20.30.42 287m 1% 24028Mi 83%
100.20.30.43 148m 2% 20784Mi 72%
100.20.30.44 327m 4% 21441Mi 74%
100.20.30.45 107m 2% 2247Mi 68%
100.20.30.46 69m 1% 2076Mi 63%
100.20.30.47 126m 1% 20126Mi 70%
100.20.30.48 107m 2% 2012Mi 61%
100.20.30.49 163m 2% 20408Mi 71%
100.20.30.50 158m 2% 23506Mi 81%
100.20.30.51 135m 1% 19663Mi 68%
100.20.30.52 150m 2% 17834Mi 62%
查看所有pod的cpu和内存
[root@pr-dr-13-47 .kube]# kubectl top pod -n projectName-sit
NAME CPU(cores) MEMORY(bytes)
aa-aaaaa-web-16c9b-6cc4b47f66-wtdwh 6m 2269Mi
agent-web-7d23a-65645ddb85-ppqbp 5m 2874Mi
aaaaaaaa-src-5bd63-648fd86446-5mzz8 8m 1992Mi
back-aaaaaaa-web-fd620-69447b56d9-xbw7c 103m 2549Mi
back-aaaaaaa-service-2dd48-76d4dd74fd-btszn 34m 2980Mi
back-aaaaaaa-web-dcc32-54bc89d9f8-p6p7t 22m 2661Mi
back-promotion-web-c9efd-776867f767-462hh 10m 2075Mi
basics-promotion-service-4a92d-7c54b687c6-klqg4 6m 1925Mi
certificate-9d281-7bfdbfcbd4-pb44m 6m 1536Mi
aaa-web-2a866-d9c767d6-wf76c 7m 1956Mi
aaa-web-0341b-6c5b7c45fc-z86w8 6m 1829Mi
frontier-aaaaa-web-2a87b-6558f5c4cd-rsjbh 9m 1699Mi
h5-b2b-ba02f-866445c976-w7tjl 0m 148Mi
h5-b2c-51b44-7b8c45d87f-sgksc 1m 146Mi
index-3eb41-5c7b4f5d77-blxbk 18m 4035Mi
aaa-web-da544-6dbcd75646-r64d5 6m 2458Mi
aaaa-web-e57c4-69458f59f4-c8cn7 28m 2346Mi
aaa-scheduler-bccbd-5786b44d98-k7tjs 8m 2332Mi
aaa-api-e29c7-794c6b974f-qvwqv 10m 2726Mi
aaa-dataex-4b411-8655df4df6-mqc6z 12m 2351Mi
aaa-task-4eea2-6b7c5b85d7-48t2j 7m 2967Mi
aaa-task-54d4c-f974cf699-w95rw 12m 2431Mi
aaa-web-15126-6f9c9d7fc7-txtdl 7m 2577Mi
aaaa-web-94990-6b8b7b685f-skbzn 9m 3760Mi
aaaa-web-a3bfc-7f8f5bf55d-r2rqs 8m 2764Mi
aaaa-routing-d0728-69f79c47bf-gmnxg 3m 898Mi
aaaaa-web-e2c0c-7795b88d97-sdgsq 15m 2186Mi
portal-f48a7-f587596cd-4lwtd 1m 22Mi
search-2baad-6d85c5b95c-z7mbv 18m 3224Mi
aaaaaa-back-web-6c194-c5c6bc88b-zzfgm 7m 1910Mi
aaaaaa-web-8ca6e-8675d64898-5nj7q 7m 2019Mi
vue-aaaaaa-c1ef5-7c857db699-f4dlz 1m 8Mi
xxl-aaa-admin-260d7-5477469576-65sww 25m 2087Mi
在Pod和本地之间复制文件
#把Pod上的/etc/fstab复制到本地的/tmp目录
kubectl cp nginx-6ddbbc47-sfdcv:/etc/fstab /tmp
watch kubectl top pod -n projectName-sit serviceName-fd620-69447b56d9-xbw7
top -Hp 1
部署metrics-server服务
https://github.com/kubernetes-sigs/metrics-server
metrics-server是Kubernetes内置的扩展，他从 Kubelets收集容器的资源数据，然后暴露Metrics API，这样我们通过kubectl就可以访问到pod的内存、cpu资源。
安装kubectl
window和linux都可以，按照官网步骤走就行
https://kubernetes.io/docs/tasks/tools/
为了让 kubectl 能发现并访问 Kubernetes 集群，home目录下新建.kube 目录并新增文件 config，通常，kubectl 的配置信息存放于文件 ~/.kube/config 中。
apiVersion: v1
clusters:
- cluster:
cert-authority-data: dsalj98ew7e98w==
server: https://1.2.3.4:60002
name: local
contexts:
- context:
cluster: local
user: admin
name: master
current-context: master
kind: Config
preferences: {}
users:
- name: admin
user:
token: dsjakhdkjsahdkj28e791e8wqdewqddsa
验证kubectl配置
如果返回一个 URL，则意味着 kubectl 成功的访问到了你的集群
[root@pr-dr-13-47 ~]# kubectl cluster-info
Kubernetes control plane is running at https://1.2.3.4:60002
CoreDNS is running at https://1.2.3.4:60002/api/v1/namespaces/kube-system/services/kube-dns:dns-tcp/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
三秒刷新一次宿主机的cpu和内存资源占用及百分比
[root@pr-dr-13-47 ~]# watch -n 3 kubectl top nodes 111.222.203.200
Every 3.0s: kubectl top nodes 111.222.203.200
Tue Feb 8 12:57:25 2022
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
111.222.203.200 183m 1% 17663Mi 61%
三秒刷新一次容器的cpu和内存资源占用
[root@pr-dr-13-47 ~]# watch -n 3 kubectl top pod -n projectName-sit aaa-web-a07a8-cc75f8b79-k2xrq
Every 3.0s: kubectl top pod -n projectName-sit aaa-web-a07a8-cc75f8b79-k2xrq Tue Feb 8 12:58:18 2022
NAME CPU(cores) MEMORY(bytes)
aaa-web-a07a8-cc75f8b79-k2xrq 23m 1760Mi
进入aaa-web容器
[root@pr-dr-13-47 ~]# kubectl exec -it aaa-web-a07a8-cc75f8b79-k2xrq -n projectName-sit -- bash
Defaulted container "aaa-web-a07a8" out of: aaa-web-a07a8, skywalking-agent (init), jmx-agent (init)
root@aaa-web-a07a8-cc75f8b79-k2xrq:/usr/local/tomcat#
查看当前容器的进程id
root@aaa-web-a07a8-cc75f8b79-k2xrq:/usr/local/tomcat# jps
1 Bootstrap
263 Jps
导出进程堆栈
root@aaa-web-a07a8-cc75f8b79-k2xrq:/usr/local/tomcat# jstack 1 > log2022.txt
1: Unable to open socket file: target process not responding or HotSpot VM not loaded
The -F option can be used when the target process is not responding
通过命令kubectl cp把堆栈日志从容器拷贝到宿主机上(当前所在目录)
[root@pr-dr-13-47 ~]# kubectl cp projectName-sit/aaa-web-a07a8-cc75f8b79-k2xrq:/usr/local/tomcat/log202202081247 log202202081247.txt
Defaulted container "aaa-web-a07a8" out of: aaa-web-a07a8, skywalking-agent (init), jmx-agent (init)
tar: Removing leading `/' from member names
从宿主机（node）上下载到本地做分析（命令sz或者其他文件上传下载工具）