etcd 是一个分布式一致性k-v存储系统，可用于服务注册发现与共享配置，具有以下优点：
1.简单：相比于晦涩难懂的paxos算法，etcd基于相对简单且易实现的raft算法实现一致性，并通过gRPC提供接口调用
2.安全：支持TLS通信，并可以针对不同的用户进行对key的读写控制
3.高性能：10,000/秒的写性能
开放端口:
firewall-cmd --zone=public --add-port=2379/tcp --permanent firewall-cmd --zone=public --add-port=2380/tcp --permanent firewall-cmd --zone=public --add-port=4001/tcp --permanent firewall-cmd --reload
设置hosts
172.16.150.25 etcd1 172.16.150.26 etcd2 172.16.150.27 etcd3
创建用户
mkdir -p /opt/platform/etcd useradd etcd -d /opt/platform/etcd -c "Etcd user" -r -s /sbin/nologin
安装cfssl
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 mv cfssl_linux-amd64 /usr/local/bin/cfssl mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
创建CA证书配置，生成CA证书和私钥, 只需要在一台机器上操作即可
先用 cfssl 命令生成包含默认配置的 config.json和 csr.json文件
mkdir /opt/ssl cd /opt/ssl cfssl print-defaults config > config.json cfssl print-defaults csr > csr.json
然后分别修改这两个文件为如下内容
vim /opt/ssl/config.json
{
"signing": {
"default": {
"expiry": "87600h"
},
"profiles": {
"kubernetes": {
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
],
"expiry": "87600h"
}
}
}
}
config.json：可以定义多个 profiles，分别指定不同的过期时间、使用场景等参数；后续在签名证书时使用某个 profile；
signing：表示该证书可用于签名其它证书；生成的 ca.pem 证书中 CA=TRUE；
server auth：表示client可以用该 CA 对server提供的证书进行验证；
client auth：表示server可以用该CA对client提供的证书进行验证；
vim /opt/ssl/csr.json
{
"CN": "kubernetes",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "Wuhan",
"L": "Hubei",
"O": "k8s",
"OU": "System"
}
]
}
CN：Common Name，kube-apiserver 从证书中提取该字段作为请求的用户名 (UserName)；浏览器使用该字段验证网站是否合法；
O：Organization，kube-apiserver 从证书中提取该字段作为请求用户所属的组(Group)
生成CA 证书和私钥
cd /opt/ssl cfssl gencert -initca csr.json | cfssljson -bare ca
CA有关证书列表如下
[root@k8s-console ssl]# tree . ├── ca.csr ├── ca-key.pem ├── ca.pem ├── config.json └── csr.json
创建etcd证书配置，生成 etcd 证书和私钥
在/opt/ssl 下添加文件 etcd-csr.json，内容如下
{
"CN": "etcd",
"hosts": [
"127.0.0.1",
"172.16.150.25",
"172.16.150.26",
"172.16.150.27"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "Shanghai",
"L": "Shanghai",
"O": "etcd",
"OU": "Etcd Security"
}
]
}
生成etcd证书和密钥
cd /opt/ssl cfssl gencert -ca=/opt/ssl/ca.pem -ca-key=/opt/ssl/ca-key.pem -config=/opt/ssl/config.json -profile=kubernetes etcd-csr.json | cfssljson -bare etcd
给证书读权限
chmod 644 /opt/ssl/*
etcd 有关证书证书列表如下
etcd.csr etcd-csr.json etcd-key.pem etcd.pem
至此CA证书就弄完了, 将这台机器侠/opt/ssl下所有东西,拷贝到其他两台机器下面的/opt/ssl下即可
安装etcd
在三台上都安装etcd
yum install etcd -y
添加etcd配置
wget https://github.com/coreos/etcd/releases/download/v3.3.7/etcd-v3.3.7-linux-amd64.tar.gz tar -xvf etcd-v3.3.7-linux-amd64.tar.gz cd etcd-v3.3.7-linux-amd64 cp -a etcd* /opt/platform/etcd/
添加etcd为系统服务
vim /usr/lib/systemd/system/etcd.service
etcd1
[Unit] Description=etcd server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify User=etcd WorkingDirectory=/opt/platform/etcd EnvironmentFile=-/opt/platform/etcd/etcd.conf ExecStart=/opt/platform/etcd/etcd --name etcd1 --cert-file=/opt/ssl/etcd.pem --key-file=/opt/ssl/etcd-key.pem --trusted-ca-file=/opt/ssl/ca.pem --peer-cert-file=/opt/ssl/etcd.pem --peer-key-file=/opt/ssl/etcd-key.pem --peer-trusted-ca-file=/opt/ssl/ca.pem --initial-advertise-peer-urls https://172.16.150.25:2380 --listen-peer-urls https://172.16.150.25:2380 --listen-client-urls https://172.16.150.25:2379,https://127.0.0.1:2379 --advertise-client-urls https://172.16.150.25:2379 --initial-cluster-token consul_etcd --initial-cluster etcd1=https://172.16.150.25:2380,etcd2=https://172.16.150.26:2380,etcd3=https://172.16.150.27:2380 --initial-cluster-state new --data-dir=/opt/platform/etcd/data Restart=on-failure RestartSec=5
etcd2
[Unit] Description=etcd server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify User=etcd WorkingDirectory=/opt/platform/etcd EnvironmentFile=-/opt/platform/etcd/etcd.conf ExecStart=/opt/platform/etcd/etcd --name etcd2 --cert-file=/opt/ssl/etcd.pem --key-file=/opt/ssl/etcd-key.pem --trusted-ca-file=/opt/ssl/ca.pem --peer-cert-file=/opt/ssl/etcd.pem --peer-key-file=/opt/ssl/etcd-key.pem --peer-trusted-ca-file=/opt/ssl/ca.pem --initial-advertise-peer-urls https://172.16.150.26:2380 --listen-peer-urls https://172.16.150.26:2380 --listen-client-urls https://172.16.150.26:2379,https://127.0.0.1:2379 --advertise-client-urls https://172.16.150.26:2379 --initial-cluster-token consul_etcd --initial-cluster etcd1=https://172.16.150.25:2380,etcd2=https://172.16.150.26:2380,etcd3=https://172.16.150.27:2380 --initial-cluster-state new --data-dir=/opt/platform/etcd/data Restart=on-failure RestartSec=5
etcd3
[Unit] Description=etcd server After=network.target After=network-online.target Wants=network-online.target [Service] Type=notify User=etcd WorkingDirectory=/opt/platform/etcd EnvironmentFile=-/opt/platform/etcd/etcd.conf ExecStart=/opt/platform/etcd/etcd --name etcd3 --cert-file=/opt/ssl/etcd.pem --key-file=/opt/ssl/etcd-key.pem --trusted-ca-file=/opt/ssl/ca.pem --peer-cert-file=/opt/ssl/etcd.pem --peer-key-file=/opt/ssl/etcd-key.pem --peer-trusted-ca-file=/opt/ssl/ca.pem --initial-advertise-peer-urls https://172.16.150.27:2380 --listen-peer-urls https://172.16.150.27:2380 --listen-client-urls https://172.16.150.27:2379,https://127.0.0.1:2379 --advertise-client-urls https://172.16.150.27:2379 --initial-cluster-token consul_etcd --initial-cluster etcd1=https://172.16.150.25:2380,etcd2=https://172.16.150.26:2380,etcd3=https://172.16.150.27:2380 --initial-cluster-state new --data-dir=/opt/platform/etcd/data Restart=on-failure RestartSec=5
启动etcd:
mkdir -p /opt/platform/etcd/data && chown etcd:etcd -R /opt/platform/etcd systemctl enable etcd.service && systemclt daemon-reload && systemctl start etcd.service
添加alias:
vim /etc/profile.d/alias_bash.sh alias etcdctl='etcdctl --endpoints=https://172.16.150.25:2379,https://172.16.150.26:2379,https://172.16.150.27:2379 --cert-file=/opt/ssl/etcd.pem --ca-file=/opt/ssl/ca.pem --key-file=/opt/ssl/etcd-key.pem' source /etc/profile.d/alias_bash.sh
验证etcd集群状态
etcdctl cluster-health
查看etcd 集群成员
etcdctl member list
测试验证:
etcdctl set test/testkey0 0 etcdctl get test/testkey0