第三十章 Centos7下Docker安装Airflow

一、拉取Airflow镜像文件

[root@prd-airflow001 ~]$ docker pull puckel/docker-airflow

二、创建挂载目录并授权

[root@prd-airflow001 ~]$ mkdir -p /data/airflow_data
[root@prd-airflow001 ~]$ chmod 777 /data/airflow_data -R

三、创建并启动Airflow容器

[root@prd-airflow001 ~]$ docker run \
-itd \
-p 8080:8080 \
-v /data/airflow_data:/usr/local/airflow \
-v /etc/localtime:/etc/localtime \
--restart always \
--privileged=true \
--name prd-airflow \
puckel/docker-airflow

四、查看Airflow是否启动成功

[root@m1 ~]# docker ps
CONTAINER ID   IMAGE             COMMAND                  CREATED         STATUS         PORTS                                       NAMES
d52953b79600   puckel/docker-airflow                                          "/entrypoint.sh webs…"   7 second ago    Up 7 second               5555/tcp, 8793/tcp, 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp                    prd-airflow

五、配置airflow

#1.进入容器
[root@prd-airflow001 ~]$ docker exec -it d52953b79600 bash

#2.初始化数据库
airflow@d52953b79600:~$ airflow initdb  

出现错误：
airflow.exceptions.AirflowException: Could not create Fernet object: Incorrect padding

解决办法：
python -c "from cryptography.fernet import Fernet;print(Fernet.generate_key().decode())"
oNu9XwewQNyx9mAJT2vZvtm3qzPRZIWRqwk9hSVch4A=

export AIRFLOW__CORE__FERNET_KEY=oNu9XwewQNyx9mAJT2vZvtm3qzPRZIWRqwk9hSVch4A=

#3.安装相关使用插件
airflow@d52953b79600:~$ pip install pymysql
airflow@d52953b79600:~$ pip install clickhouse_driver

#4.启动调度服务
airflow@d52953b79600:~$ airflow scheduler
airflow@d52953b79600:~$ nohup airflow scheduler >>airflow-scheduler.log 2>& 1 &
airflow@d52953b79600:~$ exit

六、验证连接

#1.登录ip+port 发现调度可以正常启动。