could not find a JWS signature in the cluster-info ConfigMap for token ID "qpqoq3"

[root@localhost ~]# kubeadm join 10.10.16.82:6443 --token qpqoq3.y2lo787xtima2xaz     --discovery-token-ca-cert-hash sha256:374990d65ea0b1dd227fe68aa994fa16439d0ddf99735642eee6116d98e1b829 
W0623 02:46:44.245577    6525 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
        [WARNING Service-Docker]: docker service is not enabled, please run 'systemctl enable docker.service'
        [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
error execution phase preflight: couldn't validate the identity of the API Server: could not find a JWS signature in the cluster-info ConfigMap for token ID "qpqoq3"
To see the stack trace of this error execute with --v=5 or higher
[root@localhost ~]# hostnamectl set-hostname centos7
[root@localhost ~]# hostname 
centos7

　　

这个问题是在kube-public下的 configmap 的 cluster-info 中没有JWS签名, 本质上是 token 过期.

可以通过 kube config 命令查看 cluster-info 的内容:

root@ubuntu:~# kubectl get configmap cluster-info --namespace=kube-public -o yaml
apiVersion: v1
data:
  kubeconfig: |
    apiVersion: v1
    clusters:
    - cluster:
        certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1EWXhPREV4TVRFeU1Wb1hEVE14TURZeE5qRXhNVEV5TVZvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTkcvCnIrQzh0ZjIxRGlIcG1aR1N4L0RtL295RFNXMVhMRjQzeEZPTy9Lb0djTytDZll3WVNhMWlsU3F5dDZOeGo0bDEKc05KMUdEQ2lIbERiRENVemkzbjRKeDRkRGdpaUlIUysya3ZvNUNVQm5GeGx0OWlsaWl4bTBKRE5EQitoU25lTAoxYWE5N0dIRk85OU1KTWRIVFdDZ2VHc2JUSnQzU1VEcUJtU0cvZEZuQXU2L3pnaHJiVEIreE44eXVvMDJZaEhoCnY2TE9VTGc5bFhIUldqcHB6SXo1djJQWU5Nb0I4eDBlakxMa2wydGxuOVpXTTVWa2xtMERqRnRPUlUrd1BJOUgKYXNldm1uNmVKUXZKUmQ2RFlEZXplejNZeUNZZmJaeHlOTjhVenFYRU5xeVdGNGFxR2RFVUtHVnVVMjlmWnFHTAowS1pkQ1NHckN2aHUvVUVyV0RrQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFBSVJwVVpIOWdqdllaY3U3ZnlpNDErb01odmMKRmxKNTM0bnpIZ3FoMzRhbS9tTHBlS29xVWhTSjUvWkVYQkxZdndmVklIdDF1Y1Q1UXRpZGZIeWE2amJiU0hmeAp2V2orcTNuRWhoNUIveTQydllJblRoQkt6U3JyRXBzRHQ1SENvcTU5WE1FQnhqUk5aOGxtc0J5U1lDeDd5VEVQCm90MnMrL3hhOW5PQjBVZ2pDdGNlaGMrQ0diQzUyL3VXakdzWU9BSW1nY25Mem5jeHNYdndUT0RpdU5uVGljQTUKYUxDZWluSGVWQm45YmVCYmcwZGV0dFhMRVpHbUFIS0ZidmZiVXJqR3djL1V6K2l0Tzk1V2dZRHY0QXAweVcwbQptM2ZiSVBOazd6Tmx4RmZnbWYvMVlXeTI5dkpqTWlBWkZzdzVsWUZEaVZsU056NGlqb0dqUVF6Rm94RT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
        server: https://10.10.16.82:6443
      name: ""
    contexts: null
    current-context: ""
    kind: Config
    preferences: {}
    users: null
kind: ConfigMap
metadata:
  creationTimestamp: "2021-06-18T11:12:35Z"
  managedFields:
  - apiVersion: v1
    fieldsType: FieldsV1
    fieldsV1:
      f:data:
        .: {}
        f:kubeconfig: {}
    manager: kubeadm
    operation: Update
    time: "2021-06-18T11:12:35Z"
  name: cluster-info
  namespace: kube-public
  resourceVersion: "211053"
  selfLink: /api/v1/namespaces/kube-public/configmaps/cluster-info
  uid: fec5b9e4-7550-44a9-97c1-acbfa230a8f3
root@ubuntu:~# 

当然我们也可以通过 token list 直接查看当前有效的令牌:

$ kubeadm token list
# 此处没有任何输出, 表明没有存活的token

二. 解决问题

那么如何解决呢? 我们 kubectl join 的时候, 需要2个参数: token 和 discovery-token-ca-cert-hash. 那么解决方案就是重新生成 token 和 discovery-token-ca-cert-hash.

2.1 生成token

首先我们通过以下命令生成一个新的 token:

生成 token 和 hash 可以在生成token的时候加上 --print-join-command 直接打印出来. 毕竟生成 token 就是用来添加节点用的.

root@ubuntu:~# kubeadm token create --print-join-command --ttl=0
W0623 14:56:22.340262   44305 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
kubeadm join 10.10.16.82:6443 --token hun613.jtnvs519jtvrjcy7     --discovery-token-ca-cert-hash sha256:374990d65ea0b1dd227fe68aa994fa16439d0ddf99735642eee6116d98e1b829 
root@ubuntu:~# 

其中 --ttl=0 表示生成的 token 永不失效. 如果不带 --ttl 参数, 那么默认有效时间为24小时. 在24小时内, 可以无数量限制添加 worker.

[root@localhost ~]# kubeadm join 10.10.16.82:6443 --token hun613.jtnvs519jtvrjcy7     --discovery-token-ca-cert-hash sha256:374990d65ea0b1dd227fe68aa994fa16439d0ddf99735642eee6116d98e1b829
W0623 02:57:11.552771    7329 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
        [WARNING Hostname]: hostname "centos7" could not be reached
        [WARNING Hostname]: hostname "centos7": lookup centos7 on 8.8.8.8:53: no such host
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.18" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

[root@localhost ~]#