CDH6.3.1企业离线部署

Cloudera（CDH）简介

官方文档https://www.cloudera.com/documentation/enterprise/latest.html

CDH(Clouder's Distribution Including Apache hadoop)，基于web的用户界面，支持大多数Hadoop组件，包括HDFS、MapReduce、Hive、pig、Hbase、Zookeeper、Sqoop,简化了大数据平台的安装和使用难度。

一、准备环境

1、4台虚拟机，8核32G，1T硬盘

node1的IP地址为：192.168.2.180

node2的IP地址为：192.168.2.181

node3的IP地址为：192.168.2.182

node4的IP地址为：192.168.2.183

2、关闭所有节点的selinux

vi /etc/selinux/config
将SELINUX=enforcing改为SELINUX=disabled 
设置后需要重启才能生效，现在先不要重启机器

 3、设置所有节点的hosts文件

[root@localhost ~]# echo "192.168.2.180 node0">> /etc/hosts
[root@localhost ~]# echo "192.168.2.181 node1">> /etc/hosts
[root@localhost ~]# echo "192.168.2.182 node2">> /etc/hosts
[root@localhost ~]# echo "192.168.2.183 node3">> /etc/hosts
[root@node0 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.2.180 node0
192.168.2.181 node1
192.168.2.182 node2
192.168.2.183 node3

 4、关闭所有节点的防火墙和清空防火墙规则链

# systemctl stop firewalld

# systemctl disable firewalld

# iptables -F

5、关闭所有节点的selinux

[root@localhost ~]# cat /etc/selinux/config

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

 重启服务器

# reboot

6、设置所有节点的时区一致及时钟同步

所有节点安装ntp

# yum install -y ntp

选取node0为ntp的主节点

# vi /etc/ntp.conf 

添加内容如下：

#添加一个网段
restrict 192.168.2.254 mask 255.255.255.0 nomodify notrap
#time    设置时间同步服务器
server 0.asia.pool.ntp.org
server 1.asia.pool.ntp.org
server 2.asia.pool.ntp.org
server 3.asia.pool.ntp.org
#当外部时间不可用时，可使用本地硬件时间
server 127.127.1.0 iburst #local clock 

 开启ntpd及查看状态

[root@node0 ~]# systemctl start ntpd
[root@node0 ~]# systemctl status ntpd
● ntpd.service - Network Time Service
Loaded: loaded (/usr/lib/systemd/system/ntpd.service; disabled; vendor preset: disabled)
Active: active (running) since Tue 2020-07-21 23:08:36 CST; 4s ago
Process: 3209 ExecStart=/usr/sbin/ntpd -u ntp:ntp $OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 3210 (ntpd)
CGroup: /system.slice/ntpd.service
└─3210 /usr/sbin/ntpd -u ntp:ntp -g

Jul 21 23:08:36 node0 ntpd[3210]: proto: precision = 0.105 usec
Jul 21 23:08:36 node0 ntpd[3210]: 0.0.0.0 c01d 0d kern kernel time sync enabled
Jul 21 23:08:36 node0 ntpd[3210]: ntp_io: estimated max descriptors: 1024, initial socket boundary: 16
Jul 21 23:08:36 node0 ntpd[3210]: Listen and drop on 0 v4wildcard 0.0.0.0 UDP 123
Jul 21 23:08:36 node0 ntpd[3210]: Listen and drop on 1 v6wildcard :: UDP 123
Jul 21 23:08:36 node0 ntpd[3210]: Listen normally on 2 lo 127.0.0.1 UDP 123
Jul 21 23:08:36 node0 ntpd[3210]: Listen normally on 3 ens192 192.168.2.180 UDP 123
Jul 21 23:08:36 node0 ntpd[3210]: Listen normally on 4 lo ::1 UDP 123
Jul 21 23:08:36 node0 ntpd[3210]: Listen normally on 5 ens192 fe80::664e:b20e:244f:99bd UDP 123
Jul 21 23:08:36 node0 ntpd[3210]: Listening on routing socket on fd #22 for interface updates

验证

# ntpq -p

[root@node0 ~]# ntpq -p
     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
 y.ns.gin.ntt.ne .INIT.          16 u    -   64    0    0.000    0.000   0.000
 ntp.hkg10.hk.le .INIT.          16 u    -   64    0    0.000    0.000   0.000
 111.230.189.174 .INIT.          16 u    -   64    0    0.000    0.000   0.000
 LOCAL(0)        .LOCL.           5 l    -   64    0    0.000    0.000   0.000

设置开机自启动

# systemctl enable ntpd

其他从节点启用ntpd服务

由于是新安装的服务，默认是不启动的。

# systemctl start ntpd
# systemctl enable ntpd

node1、node2、node3分别同步node0的时间

# ntpdate node0

设置node1、node2、node3每天凌晨同步node0节点时间的计划任务

# echo "00 00 * * * /usr/sbin/ntpdate node0 >/dev/null &" >> /var/spool/cron/root

# crontab -l

7、每个节点部署JDK【Java的路径必须放在/usr/java下】

# mkdir /usr/java

# tar -xzvf jdk-8u181-linux-x64.tar.gz -C /usr/java/

# chown -R root:root /usr/java/jdk1.8.0_181

# echo "export JAVA_HOME=/usr/java/jdk1.8.0_181" >> /etc/profile

# echo "export PATH=/usr/java/jdk1.8.0_181/bin:${PATH}" >> /etc/profile

# source /etc/profile

# which java

8、MySQL数据库相关操作

创建CDH的元数据库和用户、amon服务的数据库及用户

[root@mysql5 ~]# mysql -uroot -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or g.
Your MySQL connection id is 1800190
Server version: 5.7.27 MySQL Community Server (GPL)

Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or 'h' for help. Type 'c' to clear the current input statement.

mysql> create database cmf DEFAULT CHARACTER SET utf8;
Query OK, 1 row affected (0.00 sec)

mysql> create database amon DEFAULT CHARACTER SET utf8;
Query OK, 1 row affected (0.00 sec)

mysql> create user 'cmf' identified by 'Cnbisoft@123';
Query OK, 0 rows affected (0.00 sec)

mysql> create user 'amon' identified by 'Cnbisoft@123';
Query OK, 0 rows affected (0.00 sec)

mysql> grant all on cmf.* to 'cmf';
Query OK, 0 rows affected (0.00 sec)

mysql> grant all on amon.* to 'amon';
Query OK, 0 rows affected (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

 9、node0节点部署MySQL jdbc jar

# mkdir -p /usr/share/java/

# mv mysql-connector-java-5.1.47.jar /usr/share/java/mysql-connector-java.jar

二、CDH部署

1、离线部署cm server 及agent

所有节点创建目录及解压

# mkdir /opt/cloudera-manager

# tar -xzvf cm6.3.1-redhat7.tar.gz -C /opt/cloudera-manager/

2、选择node0为cm server，不下载依赖包直接部署

# cd /opt/cloudera-manager/cm6.3.1/RPMS/x86_64

# rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm --nodeps --force

# rpm -ivh cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm --nodeps --force

[root@node1 x86_64]# rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
warning: cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID b0b19c9f: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:cloudera-manager-daemons-6.3.1-14################################# [100%]                  ( 10%)
[root@node1 x86_64]#
[root@node1 x86_64]#
[root@node1 x86_64]#
[root@node1 x86_64]# rpm -ivh  cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm --nodeps --force
warning: cloudera-manager-server-6.3.1-1466458.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID b0b19c9f: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:cloudera-manager-server-6.3.1-146################################# [100%]
Created symlink from /etc/systemd/system/multi-user.target.wants/cloudera-scm-server.service to /usr/lib/systemd/system/cloudera-scm-server.service.

3、所有节点（包含node0）为cm agent，不下载依赖包直接部署

# cd /opt/cloudera-manager/cm6.3.1/RPMS/x86_64

# rpm -ivh cloudera-manager-daemons-6.3.1-1466458.el7.x86_64.rpm --nodeps --force

# rpm -ivh cloudera-manager-agent-6.3.1-1466458.el7.x86_64.rpm --nodeps --force

4、所有节点修改agent的配置，指向server的节点node0

# sed -i "s/server_host=localhost/server_host=node0/g" /etc/cloudera-scm-agent/config.ini

5、主节点修改server的配置

# vi /etc/cloudera-scm-server/db.properties

# Copyright (c) 2012 Cloudera, Inc. All rights reserved.
#
# This file describes the database connection.
#

# The database type
# Currently 'mysql', 'postgresql' and 'oracle' are valid databases.
com.cloudera.cmf.db.type=mysql

# The database host
# If a non standard port is needed, use 'hostname:port'
#com.cloudera.cmf.db.host=localhost
com.cloudera.cmf.db.host=192.168.2.240

# The database name
#com.cloudera.cmf.db.name=cmf
com.cloudera.cmf.db.name=cmf

# The database user
#com.cloudera.cmf.db.user=cmf
com.cloudera.cmf.db.user=cmf

# The database user's password
#com.cloudera.cmf.db.password=
com.cloudera.cmf.db.password=Cnbisoft@123

# The db setup type
# After fresh install it is set to INIT
# and will be changed post config.
# If scm-server uses Embedded DB then it is set to EMBEDDED
# If scm-server uses External DB then it is set to EXTERNAL
#com.cloudera.cmf.db.setupType=INIT
com.cloudera.cmf.db.setupType=EXTERNAL

6、node1节点部署离线parcel源

安装httpd服务

# yum install -y httpd

部署离线parcel源

# mkdir -p /var/www/html/cdh6_parcel

# mv CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel /var/www/html/cdh6_parcel/

# mv CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1 /var/www/html/cdh6_parcel/CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha

# mv manifest.json /var/www/html/cdh6_parcel/

启动httpd，window查看

# systemctl start httpd

# systemctl enable httpd

[root@node1 ~]# mv CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel /var/www/html/cdh6_parcel/
[root@node1 ~]# mv CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha1 /var/www/html/cdh6_parcel/CDH-6.3.1-1.cdh6.3.1.p0.1470567-el7.parcel.sha
[root@node1 ~]# mv manifest.json /var/www/html/cdh6_parcel/
[root@node1 ~]# systemctl start httpd
[root@node1 ~]# systemctl enable httpd
Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.

 http://192.168.2.180/cdh6_parcel/

7、node0节点启动Server

# systemctl start cloudera-scm-server

# cd /var/log/cloudera-scm-server/

[root@node1 ~]# cd /var/log/cloudera-scm-server/
[root@node1 cloudera-scm-server]# ll
total 456
-rw-r----- 1 cloudera-scm cloudera-scm 238988 Jul 31 18:55 cloudera-scm-server.log
-rw-r----- 1 cloudera-scm cloudera-scm   1297 Jul 31 18:54 cmf-server-nio.log
-rw-r----- 1 cloudera-scm cloudera-scm    623 Jul 31 18:54 cmf-server-perf.log

 有错误就根据错误解决，没有错误，等待1min，出现7180端口，表明是成功的。

# systemctl status cloudera-scm-server

# netstat -tunlp | grep 7180

8、所有节点启动Agent

# systemctl start cloudera-scm-agent

# systemctl status cloudera-scm-agent

三、接下来，全部Web界面操作

访问地址：http://192.168.2.181:7180

账号密码：admin/admin 

欢迎使用Cloudera Manager

 最终用户许可条款与条件。勾选

 欢迎使用Cloudera Manager--您想要部署哪个版本？选择Cloudera Express免费版本

 

 继续

 BigData

 

 管理当前的主机，继续

 选择更多选项

 添加本地URL库

 自动刷新页面，显示CDH

 安装完成

 注意：安装失败的情况时有发生，经常会出现主机异常错误，这是因为虚拟机性能不佳导致的。

我把虚拟机放置在固态硬盘中运行，解决了此问题。

如果没有条件提高虚拟机的性能，也可以修改最大超时时间。

# vi /etc/cloudera-scm-agent/config.ini

# Maximum time to wait (in seconds) for all metric collectors to finish
# collecting data.
max_collection_wait_seconds=10.0

# Maximum time to wait (in seconds) when connecting to a local role's
# webserver to fetch metrics.
metrics_url_timeout_seconds=30.0

# Maximum time to wait (in seconds) when connecting to a local TaskTracker
# to fetch task attempt data.
task_metrics_timeout_seconds=5.0

检查群集

继续

自定义服务，选择HDFS、Hive、Zookeeper

 

自定义角色分配

数据库设置，测试连接

"审核更改"不做任何修改直接继续

 end