0. 说明
使用10.45.154.208环境中的docker部署4个节点的DS集群，各节点角色分布情况如下：
|
角色 节点 |
Master-server |
Worker-server |
Alert-server |
Api-server |
zookeeper |
|
ds1 |
|
|
|
|
|
|
ds2 |
|
|
|
|
|
|
ds3 |
|
|
|
|
|
|
ds4 |
|
|
|
|
|
选择ds1作为主部署机。
1. 准备部署主机
1.1 创建docker容器
docker run -itd --privileged -p 31188:8888 -p 31122:22 -p 31145:12345 --name yj_ds1 docker.io/centos:7 /usr/sbin/init
docker run -itd --privileged -p 32222:22 --name yj_ds2 docker.io/centos:7 /usr/sbin/init
docker run -itd --privileged -p 33022:22 --name yj_ds3 docker.io/centos:7 /usr/sbin/init
docker run -itd --privileged -p 34422:22 --name yj_ds4 docker.io/centos:7 /usr/sbin/init
1.2 设置主机名称解析
再在各台主机上，编辑/etc/hosts文件：
sudo vi /etc/hosts内容是一样的，均为：
172.17.0.8 ds1
172.17.0.9 ds2
172.17.0.11 ds3
172.17.0.10 ds4
1.3 创建部署用户
在所有部署调度的机器上创建部署用户，并且一定要配置sudo免密。假如我们计划在ds1,ds2,ds3,ds4这4台机器上部署调度，首先需要在每台机器上都创建部署用户。
# 设置用户名，请自行修改，后面以dolphinscheduler为例
useradd dolphinscheduler
# 设置用户密码，请自行修改，后面以dolphinscheduler123为例
echo "dolphinscheduler123" | passwd --stdin dolphinscheduler
# 配置sudo免密
echo 'dolphinscheduler ALL=(ALL) NOPASSWD: NOPASSWD: ALL' >> /etc/sudoers
注意：
- 因为是以 sudo -u {linux-user} 切换不同linux用户的方式来实现多租户运行作业，所以部署用户需要有 sudo 权限，而且是免密的。
- 如果/etc目录下没有sudoers文件，请使用yum install –y sodo安装
- 如果发现/etc/sudoers文件中有"Default requiretty"这行，也请注释掉
- 如果用到资源上传的话，还需要在`HDFS或者MinIO`上给该部署用户分配读写的权限
1.4 安装ssh
在所有部署机上安装ssh。
yum install –y openssh-server
yum install –y openssh-clients
安装之后检查ssh-server的启动状态
1.5 配置免密
ds1 dolphinscheduler ssh本机免密钥。
su dolphinscheduler
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
在ds1上，配置部署用户dolphinscheduler ssh打通到其他待部署的机器
su dolphinscheduler;
for ip in ds2 ds3 ds4;
do
ssh-copy-id $ip #该操作执行过程中需要手动输入dolphinscheduler用户的密码
done
1.6 同步时间
timedatectl set-local-rtc 1 # 将硬件时钟调整为与本地时钟一致, 0 为设置为 UTC 时间
timedatectl set-timezone Asia/Shanghai # 设置系统时区为上海
ntpdate ntp1.aliyun.com # 同步时间
2. 安装基础软件
2.1 JDK
手动安装，将JDK1.8下载解压后，添加环境变量。
添加java命令软链接到/bin/java。
ln -s /opt/soft/jdk1.8.0_221/java /bin/java
2.2 Zookeeper
zoo.cfg配置
# 服务器之间或客户端与服务器之间维持心跳的时间间隔
# tickTime以毫秒为单位。
tickTime=2000
# 集群中的follower服务器(F)与leader服务器(L)之间的初始连接心跳数
initLimit=10
# 集群中的follower服务器与leader服务器之间请求和应答之间能容忍的最多心跳数
syncLimit=5
# 快照保存目录
# 不要设置为/tmp，该目录重新启动后会被自动清除
dataDir=/home/hadoop/data/zookeeper/data
# 日志保存目录
dataLogDir=/home/hadoop/data/zookeeper/logs
# 客户端连接端口
clientPort=2181
# 客户端最大连接数。
# 根据自己实际情况设置，默认为60个
# maxClientCnxns=60
# 三个接点配置，格式为：
# server.服务编号=服务地址、LF通信端口、选举端口
server.1=ds1:2888:3888
server.2=ds2:2888:3888
server.3=ds3:2888:3888
将该ZK分发到ds2、ds3节点：
scp –r zookeeper/ ds2:/opt/soft/
scp –r zookeeper/ ds3:/opt/soft/
写入节点标记
在三个slave节点上，分别在/home/dolphinscheduler/data/zookeeper/data/myid写入节点标记：
ds1
1
ds2
2
ds3
3
启动Zookeeper
在ds1、ds2、ds3上，分别启动zkServer。
zkServer.sh start
2.3 Postgresql
这里数据库选择Postgresql，将其安装在ds1
yum install -y https://download.postgresql.org/pub/repos/yum/9.6/redhat/rhel-7-x86_64/pgdg-centos96-9.6-3.noarch.rpm # 安装yum源
yum install -y postgresql96-server postgresql96-contrib # 安装postgresql
/usr/pgsql-9.6/bin/postgresql96-setup initdb # 初始化数据库
配置远程访问。
vi /var/lib/pgsql/9.6/data/postgresql.conf
找到listen_addresses，按i键进入插入编辑模式，如果想对所有IP开放，则将localhost改为*即可，如果想仅对部分IP开放，多个IP之间用,（逗号+空格）隔开。改完之后去掉“listen_address”前面的#。
vi /var/lib/pgsql/9.6/data/pg_hba.conf
按i键进入插入编辑模式，在IPv4 local connections下方添加允许连接的IP。 如果想允许所有IPv4地址，则加入一行host all all 0.0.0.0/0 md5。IPv6方法类似。
启动服务
systemctl start postgresql-9.6
systemctl enable postgresql-9.6
3. 安装DS
3.1 初始化数据库
进入数据库。
su postgres
psql
创建用户并配置密码和权限。
create user ds with password 'ds1234';
create database dolphinscheduler owner ds;
grant all privileges on database dolphinscheduler to ds;
修改conf/application.properties中下面三项
spring.datasource.url=jdbc:postgresql://ds1:5432/dolphinscheduler?characterEncoding=UTF-8
spring.datasource.username=ds
spring.datasource.password=ds1234
执行脚本script/create-dolphinscheduler.sh初始化数据库。
sh create-dolphinscheduler.sh
3.2 配置install.sh
修改以下配置项。
# for example postgresql or mysql ...
dbtype="postgresql"
# db config
# db address and port
dbhost="ds1:5432"
# db name
dbname="dolphinscheduler"
# db username
username="ds"
# db passwprd
# Note: if there are special characters, please use the transfer character to transfer
passowrd="ds1234"
# conf/config/install_config.conf config
# Note: the installation path is not the same as the current path (pwd)
installPath="/opt/dolphinscheduler_inst"
# deployment user
# Note: the deployment user needs to have sudo privileges and permissions to operate hdfs. If hdfs is enabled, the root directory needs to be created by itself
deployUser="dolphinscheduler"
# zk cluster
zkQuorum="ds1:2181,ds2:2181,ds3:2181"
# install hosts
# Note: install the scheduled hostname list. If it is pseudo-distributed, just write a pseudo-distributed hostname
ips="ds1,ds2,ds3,ds4"
# ssh port, default 22
# Note: if ssh port is not default, modify here
sshPort=22
# run master machine
# Note: list of hosts hostname for deploying master
masters="ds1,ds2"
# run worker machine
# note: list of machine hostnames for deploying workers
workers="ds3,ds4"
# run alert machine
# note: list of machine hostnames for deploying alert server
alertServer="ds2"
# run api machine
# note: list of machine hostnames for deploying api server
apiServers="ds1"
# alert config
# mail protocol
mailProtocol="SMTP"
# mail server host
mailServerHost="smtp.163.com"
# mail server port
mailServerPort="465"
# sender
mailSender="akalayu@163.com"
# user
mailUser="akalayu@163.com"
# sender password
mailPassword="abcabc123"
# TLS mail protocol support
starttlsEnable="false"
sslTrust="smtp.163.com"
# SSL mail protocol support
# note: The SSL protocol is enabled by default.
# only one of TLS and SSL can be in the true state.
sslEnable="true"
# resource Center upload and select storage method：HDFS,S3,NONE
resUploadStartupType="HDFS"
# if resUploadStartupType is HDFS，defaultFS write namenode address，HA you need to put core-site.xml and hdfs-site.xml in the conf directory.
# if S3，write S3 address，HA，for example ：s3a://dolphinscheduler，
# Note，s3 be sure to create the root directory /dolphinscheduler
defaultFS="hdfs://10.45.154.209:8020"
# if it is a single resourcemanager, you only need to configure one host name. If it is resourcemanager HA, the default configuration is fine.
singleYarnIp="10.45.154.209"
3.3 一键部署DS
su dolphinscheduler
sh install.sh
3.4 安装Nginx
将nginx放到yum repro库中：
rpm -ivh http://nginx.org/packages/centos/7/noarch/RPMS/nginx-release-centos-7-0.el7.ngx.noarch.rpm
使用yum安装ngnix：
yum install nginx
在/etc/nginx/conf.d/目录下添加一个名为dolphinscheduler.conf的配置文件，内容如下：
server {
listen 8888;
server_name ds1;
#charset koi8-r;
#access_log /var/log/nginx/host.access.log main;
client_max_body_size 1024M;
location / {
root /opt/dolphinscheduler_inst/ui;
index index.html index.html;
}
location /dolphinscheduler {
proxy_pass http://ds1:12345;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header x_real_ipP $remote_addr;
proxy_set_header remote_addr $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_connect_timeout 4s;
proxy_read_timeout 30s;
proxy_send_timeout 12s;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
#error_page 404 /404.html;
# redirect server error pages to the static page /50x.html
#
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
启动nginx：
systemctl start nginx.service
通过安装ds1时指定的映射端口访问DS：
