目录
认证安全
任何用途操作集群的资源对象是,都要经历三种安全相关的操作:
- 任何用户来访问时, 都需要完成kubernetes系统认证操作
- 认证通过后, 进行授权检查
- 准入控制, 检查是否有权限操作其它的一些资源操作
认证方式:
- 令牌认证：token
- SSL 秘钥认证：也是最常用的方式，能确认服务器身份
- RBAC :全称
Role Base AccessControl，用于授权操作
访问认证流程
客户端 访问 API Server是常用的参数.
user: username, uid
group:
extra:
API
Request path:
http：//172.27.1.241:8888/apis/apps/v1/namespaces/default/deployments/myapp-deploy/ # 8888端口是因为之前在master上执行了kubectl proxy --port=8888 ，查询所在组，可kubectl api-versions查看组
HTTP request verb: #请求方法，使用curl需要指定请求方法
get post put delete
API requets verb： #映射到kubernetes请求的动作
get list create update patch watch proxy redirect delete deletecollection
Resource： #请求访问的资源
Subresource： #请求的子资源
API group：
连接Api-Server的两类账号
kubernetes 有两类认证值的用户账号，分别是
ServiceAccountName: 是用于Pod客户端认证使用的账户
UserAccount: 用于真实用户，或者说操作集群资源的人使用的账户
ServiceAccount 创建
可以使用kubectl create serviceaccount USER_NAME 来创建
[root@master manifests]# kubectl create serviceaccount admin -o yaml --dry-run
apiVersion: v1
kind: ServiceAccount
metadata: #可以用来导出标准yaml格式。--dry-run执行但不真实创建sa
creationTimestamp: null
name: admin
[root@master manifests]# kubectl create serviceaccount admin
serviceaccount/admin created
[root@master manifests]# kubectl get sa
NAME SECRETS AGE
admin 1 2s
default 1 37d
[root@master manifests]# kubectl describe sa admin
Name: admin
Namespace: default
Labels: <none>
Annotations: <none>
Image pull secrets: <none>
Mountable secrets: admin-token-48wdj
Tokens: admin-token-48wdj # 自动生产的token
Events: <none>
当创建一个sa后，会自动创建一个secret
[root@master manifests]# kubectl get secret
NAME TYPE DATA AGE
admin-token-48wdj kubernetes.io/service-account-token 3 3m3s # 这里自动创建的是认证信息，但不代表有权限。
default-token-bc86p kubernetes.io/service-account-token 3 37d
[root@master manifests]# kubectl describe secret admin-token-48wdj
Name: admin-token-48wdj
Namespace: default
Labels: <none>
Annotations: kubernetes.io/service-account.name: admin
kubernetes.io/service-account.uid: dc343339-8eaf-4022-a03a-4cc8df8c9ebf
Type: kubernetes.io/service-account-token
Data
====
token: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6ImFkbWluLXRva2VuLTQ4d2RqIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiZGMzNDMzMzktOGVhZi00MDIyLWEwM2EtNGNjOGRmOGM5ZWJmIiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50OmRlZmF1bHQ6YWRtaW4ifQ.xgehKowaM6sHykd6vbfi4hu4M0DxFAYrfuY9fkN_8vkcpi_c4HC8yl0MZD4GQivRVyEPb_7X6A81BkaHLVEHvbt7bl20Gk5F2tnFxWbSGMGNh2pRa0GxtnWEO03EyGHt1Yl_YGdqT13UnwfUFC5302P7dvuC15OKjd3az-vaMu3YgUkAKyceZZRltBasmoWVHdAY1u2kVFpjT60TYPtAyV6eAeDaucB94Ye60EmyhAHBNP8DkdFbWcHT25rdRQa72B_IVZ_ZaeCfJtZkpsg4XplniC1OuXV_25nqXCNRbIqp_yaH7dT0NKhT9_EYEC4b8MYZcL2e_RZyNgBWSUf-9Q
ca.crt: 1025 bytes
namespace: 7 bytes
使用admin 的SA
定义一个清单文件，使用admin的SA
[root@master manifests]# cat pod-sa-demo.yaml
apiVersion: v1
kind: Pod
metadata:
name: pod-sa-demo
namespace: default
labels:
app: myapp
tier: frontend
annotations:
jubaozhu.com/created-by: "cluster admin"
spec:
containers:
- name: myapp
image: ikubernetes/myapp:v1
serviceAccountName: admin #在此处给pod定义sa为admin
[root@master manifests]# kubectl apply -f pod-sa-demo.yaml
pod/pod-sa-demo created
[root@master manifests]# kubectl get pods
NAME READY STATUS RESTARTS AGE
pod-sa-demo 1/1 Running 0 3s
查看该Pods的详细信息中servername
[root@master manifests]# kubectl describe pods pod-sa-demo | grep SecretName
SecretName: admin-token-46wdj # 这里已经和admin这个sa对应自动生成的secret名称是一样的。
也可以通过--dry-run 和 -o yaml 来获取yaml配置文件，该命令亦可用于导出标准yaml格式文件
[root@master manifests]# kubectl create serviceaccount admin --dry-run -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
creationTimestamp: null
name: admin
这样就能把导出的yaml格式的保存到文件中，在使用kubectl apply 来执行。
测试 URL访问kubernetes资源
首先，本地要启动一个proxy代理，之后通过这个代理来访问
[root@master ~]# kubectl proxy --port=8080
Starting to serve on 127.0.0.1:8080
再起一个新的终端进行测试
[root@master pki]# curl http://localhost:8080/apis/apps/v1/namespaces/kube-system/deployments/coredns
{
"kind": "Deployment",
"apiVersion": "apps/v1",
"metadata": {
"name": "coredns",
"namespace": "kube-system",
"selfLink": "/apis/apps/v1/namespaces/kube-system/deployments/coredns",
"uid": "c730eb98-8fe7-47a8-9856-1fc3ff2044aa",
"resourceVersion": "1039",
"generation": 1,
"creationTimestamp": "2019-07-09T08:36:50Z",
"labels": {
"k8s-app": "kube-dns"
},
"annotations": {
"deployment.kubernetes.io/revision": "1"
}
},
"spec": {
"replicas": 2,
"selector": {
"matchLabels": {
"k8s-app": "kube-dns"
... ...
... ...
/apis/apps/v1/namespaces/kube-system/deployments/coredns 这里的访问路径都是固定套路。
这里的路径区别
以 api 其实的资源，均为核心资源
剩下的其他资源，都需要以 apis 为起始路径。
有哪些资源的客户端需要和APIserver有交互
[root@master pki]# kubectl config set-cluster mycluster --kubeconfig=/tmp/test.cnf --server="https://172.27.1.241:6443" --certificate-authority=/etc/kubernetes/pki/ca.crt --embed-certs=true
Cluster "mycluster" set.
[root@master pki]# kubectl config view --kubeconfig=/tmp/test.cnf
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://172.27.1.241:6443
name: mycluster
contexts: []
current-context: ""
kind: Config
preferences: {}
users: []
授权插件:
- Node
- ABAC
- RBAC
- Webhook http的回调机制
角色机制: 给特定的角色赋权, 在把对应的角色赋予给用户, 那么该用户就拥有该角色的权限.
APIserver客户端定义的配置文件
包括kubectl、kubelet、kube-controller-manager等在内的API Server的各类客户端都可以使用kubeconfig配置文件提供接入多个集群的相关配置信息，即kubeconfig是用来存放客户端访问认证信息的
包括API Server的URL及认证信息，且可设置不同的上下文，并在各环境之间快速切换
/etc/kubernetes/admin.conf即为kubeconfig格式的配置文件
查看config命令帮助：**
[root@master manifests]# kubectl config --help
Modify kubeconfig files using subcommands like "kubectl config set current-context my-context"
The loading order follows these rules:
1. If the --kubeconfig flag is set, then only that file is loaded. The flag may only be set once and no merging takes
place.
2. If $KUBECONFIG environment variable is set, then it is used as a list of paths (normal path delimiting rules for
your system). These paths are merged. When a value is modified, it is modified in the file that defines the stanza. When
a value is created, it is created in the first file that exists. If no files in the chain exist, then it creates the
last file in the list.
3. Otherwise, ${HOME}/.kube/config is used and no merging takes place.
Available Commands:
current-context Displays the current-context
delete-cluster Delete the specified cluster from the kubeconfig
delete-context Delete the specified context from the kubeconfig
get-clusters Display clusters defined in the kubeconfig
get-contexts Describe one or many contexts
rename-context Renames a context from the kubeconfig file.
set Sets an individual value in a kubeconfig file
set-cluster Sets a cluster entry in kubeconfig
set-context Sets a context entry in kubeconfig
set-credentials Sets a user entry in kubeconfig
unset Unsets an individual value in a kubeconfig file
use-context Sets the current-context in a kubeconfig file
view Display merged kubeconfig settings or a specified kubeconfig file
Usage:
kubectl config SUBCOMMAND [options]
Use "kubectl <command> --help" for more information about a given command.
Use "kubectl options" for a list of global command-line options (applies to all commands).
| 名称 | 注解 |
|---|---|
| set-cluster | 设定集群 |
| set-credentials | 设定用户账号 |
| set-context | 设定上下文 |
| use-context | 设定当前上下文 |
可以直接通过命令的方式进行创建，也可以使用配置清单来定义后，使用kubectl apply -f 来执行设定；
查看当前配置文件的示例说明
[root@master manifests]# kubectl config view
apiVersion: v1 # 这里可以看出，config也是一个标准的kubernetes资源对象
clusters: # 集群列表
- cluster: # 定义一个集群，在下面可以定义多个
certificate-authority-data: DATA+OMITTED # 服务器发过来的认证方式
server: https://172.27.1.241:6443 # 访问验证地址
name: kubernetes # 集群名称
contexts: # 上下文列表，即账号访问权限列表
- context:
cluster: kubernetes # 对应用户能访问的集群名称
user: kubernetes-admin # 该上下文对应的用户
name: kubernetes-admin@kubernetes # 此上下文的名称
current-context: kubernetes-admin@kubernetes # 当前定义的哪个账号可以访问哪个集群
kind: Config
preferences: {}
users: # 用户列表
- name: kubernetes-admin # 用户已名称
user:
client-certificate-data: REDACTED # 客户端的认证方式
client-key-data: REDACTED # key信息
kubeconfig配置文件简单来说就是在Api-server上有多个集群（clusters）以及多个账号（users），contents就是定义哪些账号可以访问哪些集群的。
kubernetes 集群相关的私有CA证书
[root@master ~]# ll /etc/kubernetes/pki/
-rw-r--r-- 1 root root 1233 Jul 9 16:36 apiserver.crt
-rw-r--r-- 1 root root 1090 Jul 9 16:36 apiserver-etcd-client.crt
-rw------- 1 root root 1675 Jul 9 16:36 apiserver-etcd-client.key
-rw------- 1 root root 1675 Jul 9 16:36 apiserver.key
-rw-r--r-- 1 root root 1099 Jul 9 16:36 apiserver-kubelet-client.crt
-rw------- 1 root root 1679 Jul 9 16:36 apiserver-kubelet-client.key
-rw-r--r-- 1 root root 1025 Jul 9 16:36 ca.crt
-rw------- 1 root root 1679 Jul 9 16:36 ca.key
drwxr-xr-x 2 root root 162 Jul 9 16:36 etcd
-rw-r--r-- 1 root root 1038 Jul 9 16:36 front-proxy-ca.crt
-rw------- 1 root root 1675 Jul 9 16:36 front-proxy-ca.key
-rw-r--r-- 1 root root 1058 Jul 9 16:36 front-proxy-client.crt
-rw------- 1 root root 1679 Jul 9 16:36 front-proxy-client.key
-rw------- 1 root root 1679 Jul 9 16:36 sa.key
-rw------- 1 root root 451 Jul 9 16:36 sa.pub
创建新的apiserver的账号及证书
新建的密钥对就直接放到kubernetes集群对应的pki目录下
[root@master ~]# cd /etc/kubernetes/pki/
[root@master pki]# ll
total 60
-rw-r--r-- 1 root root 1233 Jul 9 16:36 apiserver.crt
-rw-r--r-- 1 root root 1090 Jul 9 16:36 apiserver-etcd-client.crt
-rw------- 1 root root 1675 Jul 9 16:36 apiserver-etcd-client.key
-rw------- 1 root root 1675 Jul 9 16:36 apiserver.key
-rw-r--r-- 1 root root 1099 Jul 9 16:36 apiserver-kubelet-client.crt
-rw------- 1 root root 1679 Jul 9 16:36 apiserver-kubelet-client.key
-rw-r--r-- 1 root root 1025 Jul 9 16:36 ca.crt
-rw------- 1 root root 1679 Jul 9 16:36 ca.key
drwxr-xr-x 2 root root 162 Jul 9 16:36 etcd
-rw-r--r-- 1 root root 1038 Jul 9 16:36 front-proxy-ca.crt
-rw------- 1 root root 1675 Jul 9 16:36 front-proxy-ca.key
-rw-r--r-- 1 root root 1058 Jul 9 16:36 front-proxy-client.crt
-rw------- 1 root root 1679 Jul 9 16:36 front-proxy-client.key
-rw------- 1 root root 1679 Jul 9 16:36 sa.key
-rw------- 1 root root 451 Jul 9 16:36 sa.pub
新创建私钥
# 创建过程均在/etc/kubernetes/pki/目录下
[root@master pki]# (umask 077; openssl genrsa -out jerry.key 2048)
Generating RSA private key, 2048 bit long modulus
.........................+++
.....................................................+++
e is 65537 (0x10001)
[root@master pki]# ll jerry.key
-rw------- 1 root root 1679 Aug 19 16:07 jerry.key
然后基于jerry.key私钥来生成证书使用kubernetes集群的ca.crt来签署
生成证书签署请求
[root@master pki]# openssl req -new -key jerry.key -out jerry.csr -subj "/CN=jerry"
# 注意，最后的 "/CN=jerry" 中的jerry，就是用户名
用ca.crt 来签署
[root@master pki]# openssl x509 -req -in jerry.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out jerry.crt -days 365
Signature ok
subject=/CN=jerry
Getting CA Private Key
[root@master pki]# ll jerry.*
-rw-r--r-- 1 root root 973 Aug 19 16:14 jerry.csr
-rw------- 1 root root 1679 Aug 19 16:07 jerry.key
验证查看生成的证书
[root@master pki]# openssl x509 -in jerry.crt -text -noout
Certificate:
Data:
Version: 1 (0x0)
Serial Number:
ea:48:9e:47:1b:2e:19:ac
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=kubernetes # 这里可以看到是kubernetes自己的ca.crt证书签署的
Validity
Not Before: Aug 19 08:14:53 2019 GMT # 证书有效期限起始
Not After : Aug 18 08:14:53 2020 GMT # 证书有效期限结束
Subject: CN=jerry # 作为用户账号连入kubernetes用户
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus:
00:cd:e3:a1:7a:db:17:70:57:16:b3:f6:6a:6c:87:
50:5f:2f:38:86:67:93:fc:b2:ba:5f:c4:60:16:66:
2f:67:29:01:c7:f0:b5:d8:ca:c6:db:ca:42:8a:a8:
ce:17:ac:d1:43:64:eb:a8:1b:34:6c:97:d9:49:dc:
24:b9:ab:8c:cd:be:6a:ad:ff:70:64:40:51:40:0f:
16:d5:61:73:dc:3f:aa:57:4f:7a:c8:60:a7:37:ee:
53:d0:ec:25:78:fc:4a:af:81:67:2c:d8:08:0f:a0:
4d:38:0f:28:36:4b:f7:c6:bb:a2:0f:4b:33:e4:77:
44:83:4d:14:2e:ef:ce:d4:6a:47:37:09:0b:9d:41:
ad:07:62:8e:06:94:71:04:f7:1b:a2:e7:6a:d6:86:
8e:7c:45:69:5e:a2:74:22:6a:8f:47:c8:ef:ba:f7:
d4:08:05:6e:c0:ae:36:bc:68:4b:7f:a5:c8:1c:87:
c4:34:a7:28:94:96:49:b1:cc:53:c0:75:7b:c9:44:
f4:78:c9:f4:fa:e2:5a:55:64:0d:dd:0a:40:31:19:
4e:80:f2:cf:02:f4:48:ae:65:93:a9:e7:41:d7:0b:
e1:7a:27:38:dc:4f:9d:12:be:8e:62:2b:73:98:0c:
45:07:c5:ad:43:91:06:0e:30:4c:5c:63:9d:25:de:
bb:29
Exponent: 65537 (0x10001)
Signature Algorithm: sha256WithRSAEncryption
74:7b:c9:fd:9e:2d:4e:84:c1:f8:83:8d:9a:37:c1:50:27:e3:
f6:1c:d4:ea:ae:4f:83:f3:dc:23:81:01:fd:3d:02:e1:26:7c:
8a:94:a7:dd:0a:3c:86:52:45:2a:25:b3:63:3f:51:15:87:0f:
f9:25:8b:eb:b0:7e:0f:7c:4a:f1:23:b9:72:7e:c6:87:55:16:
7a:bf:29:2e:da:5f:3c:4c:f2:b2:77:09:fb:fe:5a:38:67:1b:
db:c5:db:4b:64:87:1c:c7:e8:71:84:92:fd:39:17:9d:7b:b3:
dc:4b:1a:56:a8:75:63:1e:be:80:ac:98:a6:d7:41:96:8d:44:
b4:13:79:c6:08:07:b8:4a:8c:5b:04:09:a6:8d:73:47:b5:a8:
0b:97:40:66:cf:04:bc:01:93:4a:6a:ed:b2:91:94:51:12:0e:
dd:c9:70:c9:fd:65:ed:49:c2:1a:89:0f:6e:ed:69:1c:d5:2c:
cb:42:f4:ce:69:bd:99:a2:ff:ea:a7:69:5f:f7:43:b5:c3:03:
86:67:95:6e:1e:80:c7:8c:35:ac:7f:e6:e2:4e:11:08:dc:cd:
16:02:e1:cd:59:4d:70:e8:fe:5a:85:4a:18:97:bb:97:f4:6a:
82:ee:e3:c1:73:b4:2a:24:73:bd:f1:fc:08:02:6c:5d:e1:53:
设定用户账号
[root@master pki]# kubectl config set-credentials jerry --client-certificate=jerry.crt --client-key=jerry.key --embed-certs=true
User "jerry" set.
[root@master pki]# kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://172.27.1.241:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
- name: jerry # 这里可以看到刚刚设定的用户账号
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
给jerry用户加入上下文
[root@master pki]# kubectl config set-context jerry@kubernetes --cluster=kubernetes --user=jerry
Context "jerry@kubernetes" created.
[root@master pki]# kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://172.27.1.241:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
- context: # 这里开始向下就是jerry的上下文相关的配置了
cluster: kubernetes
user: jerry
name: jerry@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
- name: jerry
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
切换至刚刚增加的jerry用户的对应的
[root@master pki]# kubectl config use-context jerry@kubernetes
Switched to context "jerry@kubernetes".
[root@master pki]# kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://172.27.1.241:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
- context:
cluster: kubernetes
user: jerry
name: jerry@kubernetes
current-context: jerry@kubernetes # 看到当前上下文已经切换到 jerry@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
- name: jerry
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
测试jerry用户权限
由于jerry@kubernetes 没有赋予任何角色，所以对集群是没有任何操作权限
[root@master pki]# kubectl get pods
Error from server (Forbidden): pods is forbidden: User "jerry" cannot list resource "pods" in API group "" in the namespace "default"
这里报错意思是 default 名称空间中，不能获取数据
重新切换至kubernetes-admin@kubernetes测试
[root@master pki]# kubectl config use-context kubernetes-admin@kubernetes
Switched to context "kubernetes-admin@kubernetes".
[root@master pki]# kubectl get pods
NAME READY STATUS RESTARTS AGE
pod-sa-demo 1/1 Running 0 3d3h
保存配置文件
默认 kubernetes-admin 的配置文件保存目录是~/.kube/config文件中。
也可以保存到别的目录下：
[root@master ~]# kubectl config set-cluster test --kubeconfig=/tmp/test.conf --server="https://172.27.1.241:6443" --certificate-authority=/etc/kubernetes/pki/ca.crt --embed-certs=true
Cluster "test" set.
[root@master ~]# kubectl config view --kubeconfig=/tmp/test.conf
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: DATA+OMITTED
server: https://172.27.1.241:6443
name: test
contexts: []