==背景==
给公司搭建了一个建议的大数据平台,组件包括zookeeper、hadoop、flink、influxdb、redis、mysql、rocketmq
因为zookeeper、hadoop是平台的基础组件,打算配置systemctl守护进程,用来挂掉自动重启,以减少运维负担。
==规划==
1、每台机器设置service文件的管理文件夹:/home/radmin/rexel-dev
2、文件组成,为每个service生成两个文件:xxx.service及xxx.service.run。.run是.service需要执行的脚本,负责环境依赖
3、软连接:在systemctl目录下建立软连接,连接到rexel-dev下的具体文件
4、所有.run文件赋予777权限。
==准备==
1、创建pid文件存放路径
for i in {1..3}; do ssh vm$i mkdir -p /home/radmin/data/pids/hadoop; done
2、创建service文件存放路径
for i in {1..3}; do ssh vm$i mkdir -p /home/radmin/rexel-dev; done
3、配置系统环境变量
export HADOOP_PID_DIR=/home/radmin/data/pids/hadoop
export YARN_PID_DIR=/home/radmin/data/pids/hadoop
==zookeeper==
1、文件规划
rexel-zookeeper.service
rexel-zookeeper.service.run
2、配置run文件
文件:rexel-zookeeper.service.run
#!/bin/bash source /etc/profile /home/radmin/zookeeper-3.5.6/bin/zkServer.sh start
3、配置service文件
文件:rexel-zookeeper.service
[Unit] Description=Zookeeper Service After=network.target ConditionPathExists=/home/radmin/zookeeper-3.5.6/conf/zoo.cfg [Service] Type=forking PIDFile=/home/radmin/data/zk/dataDir/zookeeper_server.pid ExecStart=/home/radmin/rexel-dev/rexel-zookeeper.service.run Restart=always [Install] WantedBy=multi-user.target
4、配置软连接
命令:
ln -s /home/radmin/rexel-dev/rexel-zookeeper.service /usr/lib/systemd/system/rexel-zookeeper.service
5、加载生效
命令:
systemctl daemon-reload
6、启动服务
命令:
systemctl status rexel-zookeeper.service
【小贴士】
任何增加、修改或删除systemctl,都需要执行systemctl daemon-reload命令,使修改生效。
【小插曲】
service启动失败,通过journalctl -xe命令查看日志,发现提示
Can't open PID file /home/radmin/data/pid/zookeeper.pid (yet?) after start: No such file or directory
发现原因是配置的pid文件路径不正确,将pid文件的路径和zoo.cfg中的dataDir路径一致,重新启动service之后,恢复正常。
==hadoop==
1、文件规划
rexel-hdfs-journalnode.service rexel-hdfs-journalnode.service.run rexel-hdfs-namenode.service rexel-hdfs-namenode.service.run rexel-hdfs-zkfc.service rexel-hdfs-zkfc.service.run rexel-hdfs-datanode.service rexel-hdfs-datanode.service.run rexel-yarn-nodemanager.service rexel-yarn-nodemanager.service.run rexel-yarn-resourcemanager.service rexel-yarn-resourcemanager.service.run
2、配置run文件
文件1:rexel-hdfs-journalnode.service.run
#!/bin/bash source /etc/profile /home/radmin/hadoop-2.8.3/sbin/hadoop-daemon.sh start journalnode
文件2:rexel-hdfs-namenode.service.run
#!/bin/bash source /etc/profile /home/radmin/hadoop-2.8.3/sbin/hadoop-daemon.sh start namenode
文件3:rexel-hdfs-zkfc.service.run
#!/bin/bash source /etc/profile /home/radmin/hadoop-2.8.3/sbin/hadoop-daemon.sh start zkfc
文件4:rexel-hdfs-datanode.service.run
#!/bin/bash source /etc/profile /home/radmin/hadoop-2.8.3/sbin/hadoop-daemon.sh start datanode
文件5:rexel-yarn-resourcemanager.service.run
#!/bin/bash source /etc/profile /home/radmin/hadoop-2.8.3/sbin/yarn-daemon.sh start resourcemanager
文件6:rexel-yarn-nodemanager.service.run
#!/bin/bash source /etc/profile /home/radmin/hadoop-2.8.3/sbin/yarn-daemon.sh start nodemanager
3、配置service文件
文件1:rexel-hdfs-journalnode.service
[Unit] Description=HDFS Journalnode Service After=network.target [Service] Type=forking PIDFile=/home/radmin/data/pids/hadoop/hadoop-root-journalnode.pid ExecStart=/home/radmin/rexel-dev/rexel-hdfs-journalnode.service.run Restart=always [Install] WantedBy=multi-user.target
文件2:rexel-hdfs-namenode.service
[Unit] Description=HDFS Namenode Service After=network.target [Service] Type=forking PIDFile=/home/radmin/data/pids/hadoop/hadoop-root-namenode.pid ExecStart=/home/radmin/rexel-dev/rexel-hdfs-namenode.service.run Restart=always [Install] WantedBy=multi-user.target
文件3:rexel-hdfs-datanode.service
[Unit] Description=HDFS Datanode Service After=network.target [Service] Type=forking PIDFile=/home/radmin/data/pids/hadoop/hadoop-root-datanode.pid ExecStart=/home/radmin/rexel-dev/rexel-hdfs-datanode.service.run Restart=always [Install] WantedBy=multi-user.target
文件4:rexel-hdfs-zkfc.service
[Unit] Description=HDFS ZKFC Service After=network.target [Service] Type=forking PIDFile=/home/radmin/data/pids/hadoop/hadoop-root-zkfc.pid ExecStart=/home/radmin/rexel-dev/rexel-hdfs-zkfc.service.run Restart=always [Install] WantedBy=multi-user.target
文件5:rexel-yarn-resourcemanager.service
[Unit] Description=Yarn Resourcemanager Service After=network.target [Service] Type=forking PIDFile=/home/radmin/data/pids/hadoop/yarn-root-resourcemanager.pid ExecStart=/home/radmin/rexel-dev/rexel-yarn-resourcemanager.service.run Restart=always [Install] WantedBy=multi-user.target
文件6:rexel-yarn-nodemanager.service
[Unit] Description=Yarn Nodemanager Service After=network.target [Service] Type=forking PIDFile=/home/radmin/data/pids/hadoop/yarn-root-nodemanager.pid ExecStart=/home/radmin/rexel-dev/rexel-yarn-nodemanager.service.run Restart=always [Install] WantedBy=multi-user.target
4、配置软连接
命令:
ln -s /home/radmin/rexel-dev/rexel-hdfs-journalnode.service /usr/lib/systemd/system/rexel-hdfs-journalnode.service ln -s /home/radmin/rexel-dev/rexel-hdfs-namenode.service /usr/lib/systemd/system/rexel-hdfs-namenode.service ln -s /home/radmin/rexel-dev/rexel-hdfs-zkfc.service /usr/lib/systemd/system/rexel-hdfs-zkfc.service ln -s /home/radmin/rexel-dev/rexel-hdfs-datanode.service /usr/lib/systemd/system/rexel-hdfs-datanode.service ln -s /home/radmin/rexel-dev/rexel-yarn-resourcemanager.service /usr/lib/systemd/system/rexel-yarn-resourcemanager.service ln -s /home/radmin/rexel-dev/rexel-yarn-nodemanager.service /usr/lib/systemd/system/rexel-yarn-nodemanager.service
5、加载生效
命令:
systemctl daemon-reload
6、启动服务
命令:
systemctl start rexel-hdfs-journalnode.service systemctl start rexel-hdfs-namenode.service systemctl start rexel-hdfs-zkfc.service systemctl start rexel-hdfs-datanode.service systemctl start rexel-yarn-resourcemanager.service systemctl start rexel-yarn-nodemanager.service
【小插曲】
和zookeeper一样,提示了找不到pid文件的错误。
解决办法:在系统环境变量中配置HADOOP_PID_DIR,YARN_PID_DIR然后修改一下service文件中pid文件的路径,重新启动服务即可。
==Redis==
1、文件规划
rexel-redis.service
rexel-redis.service.run
2、配置run文件
文件:rexel-redis.service.run
#!/bin/bash source /etc/profile /home/radmin/redis-5.0.8/src/redis-server /home/radmin/redis-5.0.8/redis.conf
3、配置service文件
文件:rexel-redis.service
[Unit] Description=Redis Service After=network.target [Service] Type=forking PIDFile=/home/radmin/data/pids/redis_8793.pid ExecStart=/home/radmin/rexel-dev/rexel-redis.service.run Restart=always [Install] WantedBy=multi-user.target
【小贴士】
redis设置了默认的pid文件路径:/var/run/redis_6379.pid,为了方便管理,我重新定义了pidfile的配置。
相应的在rexel-redis.service文件中的
4、配置软连接
命令:
ln -s /home/radmin/rexel-dev/rexel-redis.service /usr/lib/systemd/system/rexel-redis.service
5、加载生效
命令:
systemctl daemon-reload
6、启动服务
命令:
systemctl start rexel-redis.service
--END--