一.首先了解Kubernetes核心概率和功能：
Kubernetes集群包含有节点代理kubelet和Master组件(APIs, scheduler, etc)，一切都基于分布式的存储系统。下面这张图是Kubernetes的架构图。
二.单机环境部署
1.配置yum源：
wget http://mirrors.163.com/.help/CentOS7-Base-163.repo （另外设置yum源方式yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo）
查看服务器releasever和basearch
[root@docker yum.repos.d]# rpm -qi centos-release
Name : centos-release
Version : 7
Release : 1.1503.el7.centos.2.8
Architecture: x86_64
Install Date: Thu 05 Jul 2018 02:27:56 AM EDT
Group : System Environment/Base
Size : 34409
License : GPLv2
Signature : RSA/SHA256, Tue 31 Mar 2015 06:28:11 PM EDT, Key ID 24c6a8a7f4a80eb5
Source RPM : centos-release-7-1.1503.el7.centos.2.8.src.rpm
Build Date : Tue 31 Mar 2015 06:27:03 PM EDT
Build Host : worker1.bsys.centos.org
Relocations : (not relocatable)
Packager : CentOS BuildSystem <http://bugs.centos.org>
Vendor : CentOS
Summary : CentOS Linux release file
Description :
CentOS Linux release files
查看 basearch
x86_64
替换releasever版本
sed -i 's/$releasever/7/g' CentOS7-Base-163.repo
sed -i 's/$basearch/x86_64/g' CentOS7-Base-163.repo
2.关闭centos自带防火墙和selinux
systemctl disable firewalld
systemctl stop firewalld
3.yum install -y etcd kubernetes
4.启动相应服务：
systemctl start etcd
systemctl start docker
启动docker 服务报错
/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled refer#https://www.cnblogs.com/amoyzhu/p/5261393.html
使用如下方式排错
curl -fsSL https://get.docker.com/ | sh
# Executing docker install script, commit: 36b78b2
Warning: the "docker" command appears to already exist on this system.
If you already have Docker installed, this script can cause trouble, which is
why we're displaying this warning and provide the opportunity to cancel the
installation.
删除上次docker目录：
rm -rf /var/lib/docker/*
重新启动dokcer服务正常。
三.使用kubernets创建RC和service
1.创建mysql-rc.yaml（参考kubernets说明文档编写mysql-rc.yaml文件https://kubernetes.io/docs/concepts/workloads/controllers/replicationcontroller）
kubectl create -f mysql-rc.yaml
2.查看POD状态：
kubectl get pods
出现报错No resources found.
检查RC状态
kubectl describe ReplicationController/mysql
Name: mysql
Namespace: default
Image(s): mysql
Selector: app=mysql
Labels: app=mysql
Replicas: 0 current / 3 desired
Pods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 Failed
No volumes.
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
18m 0s 60 {replication-controller } Warning FailedCreate Error creating: No API token found for service account "default", retry after the token is automatically created and added to the service account
解决办法是编辑/etc/kubernetes/apiserver 去除 KUBE_ADMISSION_CONTROL中的SecurityContextDeny,ServiceAccount，并重启kube-apiserver.service服务： #vim /etc/kubernetes/apiserver KUBE_ADMISSION_CONTROL="--admission_control=NamespaceLifecycle,NamespaceExists,LimitRanger,ResourceQuota" #systemctl restart kube-apiserver.service 之后重新创建pod:
3.重新创建POC服务后，POC创建成果。查看pods一直处于ContainerCreating状态
[root@docker ~]# kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-5pffq 0/1 ContainerCreating 0 5h
mysql-93zg9 0/1 ContainerCreating 0 5h
mysql-h7frf 0/1 ContainerCreating 0 5h
docker未运行mysql进程
[root@docker ~]# docker ps |grep mysql
一直处于ContainerCreating状态，开始查找原因：
[root@docker ~]# kubectl describe pod mysql-5pffq
Name: mysql-5pffq
Namespace: default
Node: 127.0.0.1/127.0.0.1
Start Time: Mon, 09 Jul 2018 13:02:43 -0400
Labels: app=mysql
Status: Pending
IP:
Controllers: ReplicationController/mysql
Containers:
mysql:
Container ID:
Image: mysql
Image ID:
Port: 3306/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Volume Mounts: <none>
Environment Variables: <none>
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
No volumes.
QoS Class: BestEffort
Tolerations: <none>
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
5h 1m 68 {kubelet 127.0.0.1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "POD" with ErrImagePull: "image pull failed for registry.access.redhat.com/rhel7/pod-infrastructure:latest, this may be because there are no credentials on this request. details: (open /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt: no such file or directory)"
5h 12s 1420 {kubelet 127.0.0.1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "POD" with ImagePullBackOff: "Back-off pulling image "registry.access.redhat.com/rhel7/pod-infrastructure:latest""
尝试手动pull image：
[root@docker ~]# docker pull registry.access.redhat.com/rhel7/pod-infrastructure:latest
Trying to pull repository registry.access.redhat.com/rhel7/pod-infrastructure ...
open /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt: no such file or directory
进入：
/etc/docker/certs.d/registry.access.redhat.com/
查看到有个文件redhat-ca.crt -> /etc/rhsm/ca/redhat-uep.pem 一直在闪
[root@docker registry.access.redhat.com]# yum whatprovides */redhat-uep.pem
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
python-rhsm-certificates-1.19.10-1.el7_4.x86_64 : Certificates required to communicate with a Red Hat Unified Entitlement
: Platform
Repo : base
Matched from:
Filename : /etc/rhsm/ca/redhat-uep.pem
安装rhsm
yum install -y python-rhsm-certificates-1.19.10-1.el7_4.x86_64
重新查看POD状态：
[root@docker registry.access.redhat.com]# kubectl get pod
NAME READY STATUS RESTARTS AGE
mysql-5pffq 0/1 ContainerCreating 0 5h
mysql-93zg9 0/1 ContainerCreating 0 5h
mysql-h7frf 0/1 ContainerCreating 0 5h
还是证书不存在：
[root@docker registry.access.redhat.com]# docker pull registry.access.redhat.com/rhel7/pod-infrastructure:latest
Trying to pull repository registry.access.redhat.com/rhel7/pod-infrastructure ...
open /etc/docker/certs.d/registry.access.redhat.com/redhat-ca.crt: no such file or directory
安装完rhsm后证书依旧不存在
查看需要安装的包python-rhsm-certificates-1.19.10-1.el7_4.x86_64被subscription-manager-rhsm-certificates-1.20.11-1.el7.centos.x86_64替代没有生成证书文件
从centos镜像网站下载对应文件：
手动安装：
rpm -ivh python-rhsm-certificates-1.19.10-1.el7_4.x86_64
再次查看/etc/rhsm/ca/redhat-uep.pem文件已经生成
ll /etc/rhsm/ca/redhat-uep.pem
-rw-r--r-- 1 root root 7732 Oct 19 2017 /etc/rhsm/ca/redhat-uep.pem
[root@docker 7]# docker pull registry.access.redhat.com/rhel7/pod-infrastructure:latest
Trying to pull repository registry.access.redhat.com/rhel7/pod-infrastructure ...
latest: Pulling from registry.access.redhat.com/rhel7/pod-infrastructure
26e5ed6899db: Already exists
66dbe984a319: Already exists
9138e7863e08: Already exists
Digest: sha256:92d43c37297da3ab187fc2b9e9ebfb243c1110d446c783ae1b989088495db931
Status: Image is up to date for registry.access.redhat.com/rhel7/pod-infrastructure:latest
4.创建mysql service：
kubectl create -f mysql-svc.yaml
查看POD状态处于Pending 状态：
[root@docker ~]# kubectl describe pod mysql-1cg54
Name: mysql-1cg54
Namespace: default
Node: 127.0.0.1/127.0.0.1
Start Time: Mon, 09 Jul 2018 19:24:31 -0400
Labels: app=mysql
Status: Pending
IP:
Controllers: ReplicationController/mysql
Containers:
mysql:
Container ID:
Image: mysql
Image ID:
Port: 3306/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Volume Mounts: <none>
Environment Variables: <none>
Conditions:
Type Status
Initialized True
Ready False
PodScheduled True
No volumes.
QoS Class: BestEffort
Tolerations: <none>
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
2m 2m 1 {default-scheduler } Normal Scheduled Successfully assigned mysql-1cg54 to 127.0.0.1
2m 2m 1 {kubelet 127.0.0.1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "POD" with RunContainerError: "runContainer: Error response from daemon: {"message":"error creating overlay mount to /var/lib/docker/overlay2/e59f2093257452779d9a17bcd555409d7e6b0492e2530d426494003083b081ff-init/merged: invalid argument"}"
1m 1m 1 {kubelet 127.0.0.1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "POD" with RunContainerError: "runContainer: Error response from daemon: {"message":"error creating overlay mount to /var/lib/docker/overlay2/838d7c94495bcfbad3b668966a44acbd7e6468b3356835e47dc3d6325aa9b948-init/merged: invalid argument"}"
1m 1m 1 {kubelet 127.0.0.1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "POD" with RunContainerError: "runContainer: Error response from daemon: {"message":"error creating overlay mount to /var/lib/docker/overlay2/9f00835eaa13507b4bf2674dec732d8814b076a989db644510c81e8442564e5e-init/merged: invalid argument"}"
1m 1m 1 {kubelet 127.0.0.1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "POD" with RunContainerError: "runContainer: Error response from daemon: {"message":"error creating overlay mount to /var/lib/docker/overlay2/d7ff3025c0a133856be50007c53dde4e6ec87daa2d62a97cc6ce3494a3e641f4-init/merged: invalid argument"}"
1m 1m 1 {kubelet 127.0.0.1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "POD" with RunContainerError: "runContainer: Error response from daemon: {"message":"error creating overlay mount to /var/lib/docker/overlay2/eb14878a8937c5d87cd83b19c51dd85ab5c7bff5cf93d367e8a55d36ef8c18a6-init/merged: invalid argument"}"
52s 52s 1 {kubelet 127.0.0.1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "POD" with RunContainerError: "runContainer: Error response from daemon: {"message":"error creating overlay mount to /var/lib/docker/overlay2/cf31456fd3c3bddba7997c030ca4a9ead718711c2cd447005cd4214abe619788-init/merged: invalid argument"}"
39s 39s 1 {kubelet 127.0.0.1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "POD" with RunContainerError: "runContainer: Error response from daemon: {"message":"error creating overlay mount to /var/lib/docker/overlay2/9cfe8f28521d55f020fbb5e02c10a6604d82800c4e48d1aac4c66ec71fd4edf2-init/merged: invalid argument"}"
28s 28s 1 {kubelet 127.0.0.1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "POD" with RunContainerError: "runContainer: Error response from daemon: {"message":"error creating overlay mount to /var/lib/docker/overlay2/c044367cb344a42a9df4fe809344950e575013b9bcc920416a53c489bd5f74dd-init/merged: invalid argument"}"
17s 17s 1 {kubelet 127.0.0.1} Warning FailedSync Error syncing pod, skipping: failed to "StartContainer" for "POD" with RunContainerError: "runContainer: Error response from daemon: {"message":"error creating overlay mount to /var/lib/docker/overlay2/e852c1fef7eed8db0567052818a4f4f71bdffccc216a747d92ea0d78492347f1-init/merged: invalid argument"}"
2m 5s 10 {kubelet 127.0.0.1} Warning MissingClusterDNS kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to DNSDefault policy.
5s 5s 1 {kubelet 127.0.0.1} Warning FailedSync (events with common reason combined)
报错是因为用的overlay2文件系统，而系统默认只能识别overlay文件系统
所以我们就要更新文件系统了
#systemctl stop docker //停掉docker服务
#rm -rf /var/lib/docker //注意会清掉docker images的镜像
#vi /etc/sysconfig/docker-storage //将文件里的overlay2改成overlay即可
例：DOCKER_STORAGE_OPTIONS="--storage-driver overlay "
再次创建mysql service：
kubectl create -f mysql-svc.yaml