使用ansible离线部署CDH 5.15.1大数据集群
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
在此之前，我之前分享过使用shell自定义脚本部署大数据集群，不管是部署CDH集群还是部署HDP集群基本上套路都一样，调优的方式也都大同小异。只是操作界面上可能存在一定的差异。最近赶上公司搞自动化运维项目，我们用到了ansible的一些功能，本篇博客主要记录ansible部署CDH服务。
一.安装ansible
1>.操作环境介绍
注意，请检查你的虚拟机是否可以正常联网，如果不能联网的话，本篇博客估计也就只能成为你的一个参考部署方案了！由于Mac内存有限，我只开了3台虚拟机，使用的操作系统是CentOS 7.2发行版的，内存分配每个节点4G，大家根据自己的机器适当调大或调小内存。详细信息如下：
[root@node101 ~]# hostname node101.yinzhengjie.org.cn #这是本篇博客所说的主节点，该节点配置必须高于其他节点配置或者配置不能低于2G内存！否则你在后期安装服务时，可能会遇到各种坑！（我测试时，三台主机分配过4，1，1，或者3，3，3基本上都可以正常运行！） [root@node101 ~]# [root@node101 ~]# hostname -i 172.30.1.101 [root@node101 ~]# [root@node101 ~]# cat /etc/redhat-release CentOS Linux release 7.2.1511 (Core) [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# free -h total used free shared buff/cache available Mem: 5.9G 120M 5.5G 6.7M 213M 713M Swap: 2.0G 0B 2.0G [root@node101 ~]# [root@node101 ~]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/VolGroup-lv_root 50G 3.8G 43G 9% / devtmpfs 480M 0 480M 0% /dev tmpfs 495M 0 495M 0% /dev/shm tmpfs 495M 6.7M 488M 2% /run tmpfs 495M 0 495M 0% /sys/fs/cgroup /dev/sda1 477M 133M 315M 30% /boot /dev/mapper/VolGroup-lv_home 12G 41M 11G 1% /home Home 234G 156G 79G 67% /media/psf/Home tmpfs 99M 0 99M 0% /run/user/0 [root@node101 ~]# [root@node101 ~]# cat /etc/hosts | grep yinzhengjie 172.30.1.101 node101.yinzhengjie.org.cn 172.30.1.102 node102.yinzhengjie.org.cn 172.30.1.103 node103.yinzhengjie.org.cn [root@node101 ~]# [root@node101 ~]#
2>.在主节点（node101.yinzhengjie.org.cn）配置至各个节点的免密钥登录
[root@node101 ~]# ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa Generating public/private rsa key pair. Created directory '/root/.ssh'. Your identification has been saved in /root/.ssh/id_rsa. Your public key has been saved in /root/.ssh/id_rsa.pub. The key fingerprint is: 7b:9a:3a:cd:0b:7b:b5:5e:da:ad:b8:dd:aa:0a:b1:fb root@node101.yinzhengjie.org.cn The key's randomart image is: +--[ RSA 2048]----+ | | | | | | | | | . S | | o .. | | +o..... | | .=++.* o | | +=E+*o=oo | +-----------------+ [root@node101 ~]#
[root@node101 ~]# ssh-copy-id root@node101.yinzhengjie.org.cn The authenticity of host 'node101.yinzhengjie.org.cn (172.30.1.101)' can't be established. ECDSA key fingerprint is 72:2c:15:15:05:ba:4a:73:6d:5b:88:c4:bd:ab:c2:66. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@node101.yinzhengjie.org.cn's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@node101.yinzhengjie.org.cn'" and check to make sure that only the key(s) you wanted were added. [root@node101 ~]# [root@node101 ~]# who root pts/0 2019-01-10 16:19 (172.30.1.2) [root@node101 ~]# [root@node101 ~]# ssh node101.yinzhengjie.org.cn Last login: Thu Jan 10 16:19:18 2019 from 172.30.1.2 [root@node101 ~]# [root@node101 ~]# who root pts/0 2019-01-10 16:19 (172.30.1.2) root pts/1 2019-01-10 16:30 (node101.yinzhengjie.org.cn) [root@node101 ~]# [root@node101 ~]# exit logout Connection to node101.yinzhengjie.org.cn closed. [root@node101 ~]# [root@node101 ~]# who root pts/0 2019-01-10 16:19 (172.30.1.2) [root@node101 ~]# [root@node101 ~]#
[root@node101 ~]# ssh-copy-id root@node102.yinzhengjie.org.cn The authenticity of host 'node102.yinzhengjie.org.cn (172.30.1.102)' can't be established. ECDSA key fingerprint is 85:f4:84:e1:7a:13:45:31:22:7b:12:49:5d:1b:56:ec. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@node102.yinzhengjie.org.cn's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@node102.yinzhengjie.org.cn'" and check to make sure that only the key(s) you wanted were added. [root@node101 ~]# [root@node101 ~]# ssh node102.yinzhengjie.org.cn Last login: Thu Jan 10 15:46:07 2019 from 172.30.1.2 [root@node102 ~]# [root@node102 ~]# who root pts/0 2019-01-10 15:46 (172.30.1.2) root pts/1 2019-01-10 16:31 (172.30.1.101) [root@node102 ~]# [root@node102 ~]# exit logout Connection to node102.yinzhengjie.org.cn closed. [root@node101 ~]# [root@node101 ~]#
[root@node101 ~]# ssh-copy-id root@node103.yinzhengjie.org.cn The authenticity of host 'node103.yinzhengjie.org.cn (172.30.1.103)' can't be established. ECDSA key fingerprint is 62:e7:04:30:ce:10:0d:55:9d:e5:24:9f:19:ab:4c:e7. Are you sure you want to continue connecting (yes/no)? yes /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys root@node103.yinzhengjie.org.cn's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'root@node103.yinzhengjie.org.cn'" and check to make sure that only the key(s) you wanted were added. [root@node101 ~]# [root@node101 ~]# ssh node103.yinzhengjie.org.cn Last login: Thu Jan 10 15:46:08 2019 from 172.30.1.2 [root@node103 ~]# [root@node103 ~]# who root pts/0 2019-01-10 15:46 (172.30.1.2) root pts/1 2019-01-10 16:32 (172.30.1.101) [root@node103 ~]# [root@node103 ~]# exit logout Connection to node103.yinzhengjie.org.cn closed. [root@node101 ~]# [root@node101 ~]#
3>.使用yum工具包安装ansible
安装ansible依赖python2-pyasn1，PyYAML，python-jinja2等模块，使用yum方式安装，他会自动帮我们处理依赖关系，我们配置好yum源即可。
[root@node101 ~]# yum -y install wget Loaded plugins: fastestmirror base | 3.6 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 updates/7/x86_64/primary_db FAILED http://mirrors.cqu.edu.cn/CentOS/7.6.1810/updates/x86_64/repodata/1c8b4fadfe3ee3539b33616064daaae4a96b2d659235607042e44c7cc389e0dc-primary.sqlite.bz2: [Errno 14] curl#56 - "Recv failure: Connection reset by peer" Trying other mirror. (1/4): extras/7/x86_64/primary_db | 156 kB 00:00:00 (2/4): base/7/x86_64/group_gz | 166 kB 00:00:02 (3/4): updates/7/x86_64/primary_db | 2.4 MB 00:00:03 (4/4): base/7/x86_64/primary_db | 6.0 MB 00:00:08 Determining fastest mirrors * base: mirrors.tuna.tsinghua.edu.cn * extras: mirrors.aliyun.com * updates: mirrors.tuna.tsinghua.edu.cn Resolving Dependencies --> Running transaction check ---> Package wget.x86_64 0:1.14-18.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved =================================================================================================================================================================== Package Arch Version Repository Size =================================================================================================================================================================== Installing: wget x86_64 1.14-18.el7 base 547 k Transaction Summary =================================================================================================================================================================== Install 1 Package Total download size: 547 k Installed size: 2.0 M Downloading packages: warning: /var/cache/yum/x86_64/7/base/packages/wget-1.14-18.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY Public key for wget-1.14-18.el7.x86_64.rpm is not installed wget-1.14-18.el7.x86_64.rpm | 547 kB 00:00:00 Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 Importing GPG key 0xF4A80EB5: Userid : "CentOS-7 Key (CentOS 7 Official Signing Key) <security@centos.org>" Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5 Package : centos-release-7-2.1511.el7.centos.2.10.x86_64 (@anaconda) From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : wget-1.14-18.el7.x86_64 1/1 Verifying : wget-1.14-18.el7.x86_64 1/1 Installed: wget.x86_64 0:1.14-18.el7 Complete! [root@node101 ~]#
[root@node101 ~]# wget http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm --2019-01-10 17:14:45-- http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm Resolving dl.fedoraproject.org (dl.fedoraproject.org)... 209.132.181.23, 209.132.181.24 Connecting to dl.fedoraproject.org (dl.fedoraproject.org)|209.132.181.23|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 15080 (15K) [application/x-rpm] Saving to: ‘epel-release-latest-7.noarch.rpm’ 100%[================================================================================================================================>] 15,080 78.2KB/s in 0.2s 2019-01-10 17:14:46 (78.2 KB/s) - ‘epel-release-latest-7.noarch.rpm’ saved [15080/15080] [root@node101 ~]#
[root@node101 ~]# ll total 16 -rw-r--r--. 1 root root 15080 Jan 10 17:00 epel-release-latest-7.noarch.rpm [root@node101 ~]# [root@node101 ~]# rpm -ivh epel-release-latest-7.noarch.rpm warning: epel-release-latest-7.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY Preparing... ################################# [100%] Updating / installing... 1:epel-release-7-11 ################################# [100%] [root@node101 ~]# [root@node101 ~]# ll /etc/yum.repos.d/ | grep epel -rw-r--r--. 1 root root 951 Oct 3 2017 epel.repo -rw-r--r--. 1 root root 1050 Oct 3 2017 epel-testing.repo [root@node101 ~]#
[root@node101 ~]# cat /etc/yum.conf | grep -v ^# |grep -v ^$ [main] cachedir=/var/cache/yum/$basearch/$releasever keepcache=0 debuglevel=2 logfile=/var/log/yum.log exactarch=1 obsoletes=1 gpgcheck=1 plugins=1 installonly_limit=5 bugtracker_url=http://bugs.centos.org/set_project.php?project_id=23&ref=http://bugs.centos.org/bug_report_page.php?category=yum distroverpkg=centos-release [root@node101 ~]# [root@node101 ~]# sed -i 's#keepcache=0#keepcache=1#' /etc/yum.conf [root@node101 ~]# [root@node101 ~]# cat /etc/yum.conf | grep -v ^# |grep -v ^$ [main] cachedir=/var/cache/yum/$basearch/$releasever keepcache=1 debuglevel=2 logfile=/var/log/yum.log exactarch=1 obsoletes=1 gpgcheck=1 plugins=1 installonly_limit=5 bugtracker_url=http://bugs.centos.org/set_project.php?project_id=23&ref=http://bugs.centos.org/bug_report_page.php?category=yum distroverpkg=centos-release [root@node101 ~]#
[root@node101 ~]# yum clean all Loaded plugins: fastestmirror, langpacks Cleaning repos: epel Cleaning up everything [root@node101 ~]#
[root@node101 ~]# yum makecache Loaded plugins: fastestmirror, langpacks base | 3.6 kB 00:00:00 epel/x86_64/metalink | 8.1 kB 00:00:00 epel | 3.2 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 (1/18): epel/x86_64/group_gz | 88 kB 00:00:00 (2/18): base/7/x86_64/other_db | 2.6 MB 00:00:06 (3/18): base/7/x86_64/group_gz | 166 kB 00:00:08 (4/18): epel/x86_64/prestodelta | 803 B 00:00:11 (5/18): epel/x86_64/updateinfo | 948 kB 00:00:13 (6/18): epel/x86_64/filelists | 10 MB 00:00:20 (7/18): epel/x86_64/other | 2.3 MB 00:00:06 (8/18): base/7/x86_64/filelists_db | 7.1 MB 00:00:27 (9/18): extras/7/x86_64/primary_db | 156 kB 00:00:01 (10/18): extras/7/x86_64/other_db | 107 kB 00:00:01 (11/18): base/7/x86_64/primary_db | 6.0 MB 00:00:33 (12/18): updates/7/x86_64/prestodelta | 183 kB 00:00:03 (13/18): extras/7/x86_64/filelists_db | 189 kB 00:00:22 (14/18): updates/7/x86_64/filelists_db | 1.4 MB 00:00:14 (15/18): updates/7/x86_64/primary_db | 1.3 MB 00:00:14 extras/7/x86_64/prestodelta FAILED ========================================================-] 200 kB/s | 36 MB 00:00:01 ETA http://mirrors.njupt.edu.cn/centos/7.6.1810/extras/x86_64/repodata/9718255a5fa57675311c5ac73719bee470f94b10d38b5aa988511b9ddd860025-prestodelta.xml.gz: [Errno 12] Timeout on http://10.10.254.10/cache/4/02/mirrors.njupt.edu.cn/4f9b5de52f938c8378fba16c668a0a55/9718255a5fa57675311c5ac73719bee470f94b10d38b5aa988511b9ddd860025-prestodelta.xml.gz: (28, 'Connection timed out after 30884 milliseconds') Trying other mirror. (16/18): extras/7/x86_64/prestodelta | 36 kB 00:00:00 (17/18): epel/x86_64/primary | 3.6 MB 00:00:38 updates/7/x86_64/other_db FAILED http://mirrors.njupt.edu.cn/centos/7.6.1810/updates/x86_64/repodata/1fab468ed066d5469823091845179165040deb4951923d854a025e3b5c903051-other.sqlite.bz2: [Errno 12] Timeout on http://10.10.254.10/cache/5/02/mirrors.njupt.edu.cn/08d3e890295133ac7140ae59fc9dee3f/1fab468ed066d5469823091845179165040deb4951923d854a025e3b5c903051-other.sqlite.bz2: (28, 'Connection timed out after 30866 milliseconds') Trying other mirror. (18/18): updates/7/x86_64/other_db | 191 kB 00:00:00 Determining fastest mirrors * base: mirror.jdcloud.com * epel: mirrors.yun-idc.com * extras: mirror.jdcloud.com * updates: mirrors.tuna.tsinghua.edu.cn epel 12852/12852 epel 12852/12852 epel 12852/12852 Metadata Cache Created [root@node101 ~]#
[root@node101 ~]# yum -y install ansible Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile * base: mirror.jdcloud.com * epel: mirrors.yun-idc.com * extras: mirror.jdcloud.com * updates: mirrors.tuna.tsinghua.edu.cn Resolving Dependencies --> Running transaction check ---> Package ansible.noarch 0:2.7.5-1.el7 will be installed --> Processing Dependency: sshpass for package: ansible-2.7.5-1.el7.noarch --> Processing Dependency: python2-jmespath for package: ansible-2.7.5-1.el7.noarch --> Processing Dependency: python-paramiko for package: ansible-2.7.5-1.el7.noarch --> Processing Dependency: python-keyczar for package: ansible-2.7.5-1.el7.noarch --> Processing Dependency: python-jinja2 for package: ansible-2.7.5-1.el7.noarch --> Processing Dependency: python-httplib2 for package: ansible-2.7.5-1.el7.noarch --> Processing Dependency: python-crypto for package: ansible-2.7.5-1.el7.noarch --> Processing Dependency: PyYAML for package: ansible-2.7.5-1.el7.noarch --> Running transaction check ---> Package PyYAML.x86_64 0:3.10-11.el7 will be installed --> Processing Dependency: libyaml-0.so.2()(64bit) for package: PyYAML-3.10-11.el7.x86_64 ---> Package python-httplib2.noarch 0:0.9.2-1.el7 will be installed ---> Package python-jinja2.noarch 0:2.7.2-2.el7 will be installed --> Processing Dependency: python-babel >= 0.8 for package: python-jinja2-2.7.2-2.el7.noarch --> Processing Dependency: python-markupsafe for package: python-jinja2-2.7.2-2.el7.noarch ---> Package python-keyczar.noarch 0:0.71c-2.el7 will be installed ---> Package python-paramiko.noarch 0:2.1.1-9.el7 will be installed --> Processing Dependency: python2-pyasn1 for package: python-paramiko-2.1.1-9.el7.noarch ---> Package python2-crypto.x86_64 0:2.6.1-15.el7 will be installed --> Processing Dependency: libtomcrypt.so.0()(64bit) for package: python2-crypto-2.6.1-15.el7.x86_64 ---> Package python2-jmespath.noarch 0:0.9.0-3.el7 will be installed ---> Package sshpass.x86_64 0:1.06-2.el7 will be installed --> Running transaction check ---> Package libtomcrypt.x86_64 0:1.17-26.el7 will be installed --> Processing Dependency: libtommath >= 0.42.0 for package: libtomcrypt-1.17-26.el7.x86_64 --> Processing Dependency: libtommath.so.0()(64bit) for package: libtomcrypt-1.17-26.el7.x86_64 ---> Package libyaml.x86_64 0:0.1.4-11.el7_0 will be installed ---> Package python-babel.noarch 0:0.9.6-8.el7 will be installed ---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed ---> Package python-pyasn1.noarch 0:0.1.6-2.el7 will be obsoleted ---> Package python2-pyasn1.noarch 0:0.1.9-7.el7 will be obsoleting --> Running transaction check ---> Package libtommath.x86_64 0:0.42.0-6.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ========================================================================================================================================================================== Package Arch Version Repository Size ========================================================================================================================================================================== Installing: ansible noarch 2.7.5-1.el7 epel 11 M python2-pyasn1 noarch 0.1.9-7.el7 base 100 k replacing python-pyasn1.noarch 0.1.6-2.el7 Installing for dependencies: PyYAML x86_64 3.10-11.el7 base 153 k libtomcrypt x86_64 1.17-26.el7 extras 224 k libtommath x86_64 0.42.0-6.el7 extras 36 k libyaml x86_64 0.1.4-11.el7_0 base 55 k python-babel noarch 0.9.6-8.el7 base 1.4 M python-httplib2 noarch 0.9.2-1.el7 extras 115 k python-jinja2 noarch 2.7.2-2.el7 base 515 k python-keyczar noarch 0.71c-2.el7 epel 218 k python-markupsafe x86_64 0.11-10.el7 base 25 k python-paramiko noarch 2.1.1-9.el7 updates 269 k python2-crypto x86_64 2.6.1-15.el7 extras 477 k python2-jmespath noarch 0.9.0-3.el7 extras 39 k sshpass x86_64 1.06-2.el7 extras 21 k Transaction Summary ========================================================================================================================================================================== Install 2 Packages (+13 Dependent packages) Total download size: 15 M Downloading packages: (1/15): PyYAML-3.10-11.el7.x86_64.rpm | 153 kB 00:00:00 (2/15): libtomcrypt-1.17-26.el7.x86_64.rpm | 224 kB 00:00:00 (3/15): python-httplib2-0.9.2-1.el7.noarch.rpm | 115 kB 00:00:00 (4/15): python-jinja2-2.7.2-2.el7.noarch.rpm | 515 kB 00:00:00 (5/15): libtommath-0.42.0-6.el7.x86_64.rpm | 36 kB 00:00:01 (6/15): python-markupsafe-0.11-10.el7.x86_64.rpm | 25 kB 00:00:00 (7/15): python-paramiko-2.1.1-9.el7.noarch.rpm | 269 kB 00:00:00 (8/15): python2-crypto-2.6.1-15.el7.x86_64.rpm | 477 kB 00:00:00 (9/15): python2-jmespath-0.9.0-3.el7.noarch.rpm | 39 kB 00:00:00 (10/15): python2-pyasn1-0.1.9-7.el7.noarch.rpm | 100 kB 00:00:00 (11/15): sshpass-1.06-2.el7.x86_64.rpm | 21 kB 00:00:00 (12/15): python-babel-0.9.6-8.el7.noarch.rpm | 1.4 MB 00:00:04 (13/15): libyaml-0.1.4-11.el7_0.x86_64.rpm | 55 kB 00:00:05 warning: /var/cache/yum/x86_64/7/epel/packages/python-keyczar-0.71c-2.el7.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID 352c64e5: NOKEY7 kB/s | 9.4 MB 00:00:06 ETA Public key for python-keyczar-0.71c-2.el7.noarch.rpm is not installed (14/15): python-keyczar-0.71c-2.el7.noarch.rpm | 218 kB 00:00:09 (15/15): ansible-2.7.5-1.el7.noarch.rpm | 11 MB 00:00:17 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 870 kB/s | 15 MB 00:00:17 Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7 Importing GPG key 0x352C64E5: Userid : "Fedora EPEL (7) <epel@fedoraproject.org>" Fingerprint: 91e9 7d7c 4a5e 96f1 7f3e 888f 6a2f aea2 352c 64e5 Package : epel-release-7-11.noarch (installed) From : /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7 Running transaction check Running transaction test Transaction test succeeded Running transaction Warning: RPMDB altered outside of yum. Installing : python2-pyasn1-0.1.9-7.el7.noarch 1/16 Installing : python-paramiko-2.1.1-9.el7.noarch 2/16 Installing : python-babel-0.9.6-8.el7.noarch 3/16 Installing : python-httplib2-0.9.2-1.el7.noarch 4/16 Installing : sshpass-1.06-2.el7.x86_64 5/16 Installing : libtommath-0.42.0-6.el7.x86_64 6/16 Installing : libtomcrypt-1.17-26.el7.x86_64 7/16 Installing : python2-crypto-2.6.1-15.el7.x86_64 8/16 Installing : python-keyczar-0.71c-2.el7.noarch 9/16 Installing : python-markupsafe-0.11-10.el7.x86_64 10/16 Installing : python-jinja2-2.7.2-2.el7.noarch 11/16 Installing : python2-jmespath-0.9.0-3.el7.noarch 12/16 Installing : libyaml-0.1.4-11.el7_0.x86_64 13/16 Installing : PyYAML-3.10-11.el7.x86_64 14/16 Installing : ansible-2.7.5-1.el7.noarch 15/16 Erasing : python-pyasn1-0.1.6-2.el7.noarch 16/16 Verifying : python-keyczar-0.71c-2.el7.noarch 1/16 Verifying : libyaml-0.1.4-11.el7_0.x86_64 2/16 Verifying : python-jinja2-2.7.2-2.el7.noarch 3/16 Verifying : python2-jmespath-0.9.0-3.el7.noarch 4/16 Verifying : python-paramiko-2.1.1-9.el7.noarch 5/16 Verifying : python-markupsafe-0.11-10.el7.x86_64 6/16 Verifying : python2-crypto-2.6.1-15.el7.x86_64 7/16 Verifying : libtommath-0.42.0-6.el7.x86_64 8/16 Verifying : sshpass-1.06-2.el7.x86_64 9/16 Verifying : python-httplib2-0.9.2-1.el7.noarch 10/16 Verifying : ansible-2.7.5-1.el7.noarch 11/16 Verifying : PyYAML-3.10-11.el7.x86_64 12/16 Verifying : python-babel-0.9.6-8.el7.noarch 13/16 Verifying : libtomcrypt-1.17-26.el7.x86_64 14/16 Verifying : python2-pyasn1-0.1.9-7.el7.noarch 15/16 Verifying : python-pyasn1-0.1.6-2.el7.noarch 16/16 Installed: ansible.noarch 0:2.7.5-1.el7 python2-pyasn1.noarch 0:0.1.9-7.el7 Dependency Installed: PyYAML.x86_64 0:3.10-11.el7 libtomcrypt.x86_64 0:1.17-26.el7 libtommath.x86_64 0:0.42.0-6.el7 libyaml.x86_64 0:0.1.4-11.el7_0 python-babel.noarch 0:0.9.6-8.el7 python-httplib2.noarch 0:0.9.2-1.el7 python-jinja2.noarch 0:2.7.2-2.el7 python-keyczar.noarch 0:0.71c-2.el7 python-markupsafe.x86_64 0:0.11-10.el7 python-paramiko.noarch 0:2.1.1-9.el7 python2-crypto.x86_64 0:2.6.1-15.el7 python2-jmespath.noarch 0:0.9.0-3.el7 sshpass.x86_64 0:1.06-2.el7 Replaced: python-pyasn1.noarch 0:0.1.6-2.el7 Complete! [root@node101 ~]#
4>.配置ansible的配置文件
我根据我的需求，将主机分为2个角色，大家都知道，hdfs集群又NameNode和DataNode之分，因此我将配置相对较高的那台机器配置为NameNode节点，然后把DataNode服务安置在所有的节点上。
[root@node101 ~]# tail -8 /etc/ansible/hosts #Add by yinzhengjie [namenode] node101.yinzhengjie.org.cn [datanode] node101.yinzhengjie.org.cn node102.yinzhengjie.org.cn node103.yinzhengjie.org.cn [root@node101 ~]# [root@node101 ~]#
5>.安装相关依赖插件，解决cryptography 0.8.2 版本兼容性（如果该步骤不做的话，在你执行“ansible all -m ping”命令时可能会报错！）
5.1>.安装pip
[root@node101 ~]# wget https://files.pythonhosted.org/packages/45/ae/8a0ad77defb7cc903f09e551d88b443304a9bd6e6f124e75c0fbbf6de8f7/pip-18.1.tar.gz --2019-01-10 18:11:31-- https://files.pythonhosted.org/packages/45/ae/8a0ad77defb7cc903f09e551d88b443304a9bd6e6f124e75c0fbbf6de8f7/pip-18.1.tar.gz Resolving files.pythonhosted.org (files.pythonhosted.org)... 151.101.109.63, 2a04:4e42:36::319 Connecting to files.pythonhosted.org (files.pythonhosted.org)|151.101.109.63|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1259370 (1.2M) [binary/octet-stream] Saving to: ‘pip-18.1.tar.gz’ 100%[================================================================================================================================>] 1,259,370 64.7KB/s in 22s 2019-01-10 18:11:55 (55.4 KB/s) - ‘pip-18.1.tar.gz’ saved [1259370/1259370] [root@node101 ~]#
[root@node101 ~]# tar -zxf pip-18.1.tar.gz [root@node101 ~]# ll total 1252 -rw-r--r--. 1 root root 15080 Oct 3 2017 epel-release-latest-7.noarch.rpm drwxr-xr-x. 6 501 games 4096 Jan 10 18:13 pip-18.1 -rw-r--r--. 1 root root 1259370 Oct 5 19:20 pip-18.1.tar.gz [root@node101 ~]# [root@node101 ~]# cd pip-18.1/ [root@node101 pip-18.1]# [root@node101 pip-18.1]# python setup.py install .........(内容输出过多，我这里就统一复制了！) Installing pip2.7 script to /usr/bin Installing pip2 script to /usr/bin Installed /usr/lib/python2.7/site-packages/pip-18.1-py2.7.egg Processing dependencies for pip==18.1 Finished processing dependencies for pip==18.1 [root@node101 pip-18.1]#
5.2>.安装wheel
[root@node101 ~]# wget https://files.pythonhosted.org/packages/d8/55/221a530d66bf78e72996453d1e2dedef526063546e131d70bed548d80588/wheel-0.32.3.tar.gz --2019-01-10 18:16:37-- https://files.pythonhosted.org/packages/d8/55/221a530d66bf78e72996453d1e2dedef526063546e131d70bed548d80588/wheel-0.32.3.tar.gz Resolving files.pythonhosted.org (files.pythonhosted.org)... 151.101.109.63, 2a04:4e42:1a::319 Connecting to files.pythonhosted.org (files.pythonhosted.org)|151.101.109.63|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 18585 (18K) [binary/octet-stream] Saving to: ‘wheel-0.32.3.tar.gz’ 100%[================================================================================================================================>] 18,585 --.-K/s in 0.1s 2019-01-10 18:16:38 (162 KB/s) - ‘wheel-0.32.3.tar.gz’ saved [18585/18585] [root@node101 ~]# ll total 1272 -rw-r--r--. 1 root root 15080 Oct 3 2017 epel-release-latest-7.noarch.rpm drwxr-xr-x. 6 501 games 4096 Jan 10 18:13 pip-18.1 -rw-r--r--. 1 root root 1259370 Oct 5 19:20 pip-18.1.tar.gz -rw-r--r--. 1 root root 18585 Nov 19 08:26 wheel-0.32.3.tar.gz [root@node101 ~]# [root@node101 ~]#
[root@node101 ~]# ll total 1272 -rw-r--r--. 1 root root 15080 Oct 3 2017 epel-release-latest-7.noarch.rpm drwxr-xr-x. 6 501 games 4096 Jan 10 18:13 pip-18.1 -rw-r--r--. 1 root root 1259370 Oct 5 19:20 pip-18.1.tar.gz -rw-r--r--. 1 root root 18585 Nov 19 08:26 wheel-0.32.3.tar.gz [root@node101 ~]# [root@node101 ~]# tar -zxf wheel-0.32.3.tar.gz [root@node101 ~]# [root@node101 ~]# ll total 1276 -rw-r--r--. 1 root root 15080 Oct 3 2017 epel-release-latest-7.noarch.rpm drwxr-xr-x. 6 501 games 4096 Jan 10 18:13 pip-18.1 -rw-r--r--. 1 root root 1259370 Oct 5 19:20 pip-18.1.tar.gz drwxrwxr-x. 4 2000 2000 4096 Nov 19 08:25 wheel-0.32.3 -rw-r--r--. 1 root root 18585 Nov 19 08:26 wheel-0.32.3.tar.gz [root@node101 ~]# [root@node101 ~]# cd wheel-0.32.3/ [root@node101 wheel-0.32.3]# ll total 32 -rw-rw-r--. 1 2000 2000 1125 Nov 19 08:25 LICENSE.txt -rw-rw-r--. 1 2000 2000 244 Nov 19 08:25 MANIFEST.in -rw-rw-r--. 1 2000 2000 2207 Nov 19 08:25 PKG-INFO -rw-rw-r--. 1 2000 2000 799 Nov 19 08:25 README.rst -rw-rw-r--. 1 2000 2000 276 Nov 19 08:25 setup.cfg -rw-rw-r--. 1 2000 2000 2133 Nov 19 08:25 setup.py drwxrwxr-x. 3 2000 2000 4096 Nov 19 08:25 wheel drwxrwxr-x. 2 2000 2000 4096 Nov 19 08:25 wheel.egg-info [root@node101 wheel-0.32.3]# [root@node101 wheel-0.32.3]# python setup.py install /usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'project_urls' warnings.warn(msg) /usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'python_requires' warnings.warn(msg) running install running bdist_egg running egg_info writing requirements to wheel.egg-info/requires.txt writing wheel.egg-info/PKG-INFO writing top-level names to wheel.egg-info/top_level.txt writing dependency_links to wheel.egg-info/dependency_links.txt writing entry points to wheel.egg-info/entry_points.txt reading manifest file 'wheel.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching 'wheel/*.txt' warning: no files found matching '*.sh' warning: no files found matching '*.py' under directory 'wheel/test' warning: no files found matching 'wheel/test/test-1.0-py2.py3-none-win32.whl' warning: no files found matching 'wheel/test/headers.dist/header.h' warning: no files found matching 'wheel/test/pydist-schema.json' no previously-included directories found matching 'wheel/test/*/dist' no previously-included directories found matching 'wheel/test/*/build' writing manifest file 'wheel.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib creating build/lib/wheel copying wheel/wheelfile.py -> build/lib/wheel copying wheel/pep425tags.py -> build/lib/wheel copying wheel/__init__.py -> build/lib/wheel copying wheel/util.py -> build/lib/wheel copying wheel/metadata.py -> build/lib/wheel copying wheel/__main__.py -> build/lib/wheel copying wheel/bdist_wheel.py -> build/lib/wheel copying wheel/pkginfo.py -> build/lib/wheel creating build/lib/wheel/cli copying wheel/cli/install.py -> build/lib/wheel/cli copying wheel/cli/pack.py -> build/lib/wheel/cli copying wheel/cli/__init__.py -> build/lib/wheel/cli copying wheel/cli/convert.py -> build/lib/wheel/cli copying wheel/cli/unpack.py -> build/lib/wheel/cli creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/wheel copying build/lib/wheel/wheelfile.py -> build/bdist.linux-x86_64/egg/wheel copying build/lib/wheel/pep425tags.py -> build/bdist.linux-x86_64/egg/wheel copying build/lib/wheel/__init__.py -> build/bdist.linux-x86_64/egg/wheel copying build/lib/wheel/util.py -> build/bdist.linux-x86_64/egg/wheel copying build/lib/wheel/metadata.py -> build/bdist.linux-x86_64/egg/wheel copying build/lib/wheel/__main__.py -> build/bdist.linux-x86_64/egg/wheel copying build/lib/wheel/bdist_wheel.py -> build/bdist.linux-x86_64/egg/wheel copying build/lib/wheel/pkginfo.py -> build/bdist.linux-x86_64/egg/wheel creating build/bdist.linux-x86_64/egg/wheel/cli copying build/lib/wheel/cli/install.py -> build/bdist.linux-x86_64/egg/wheel/cli copying build/lib/wheel/cli/pack.py -> build/bdist.linux-x86_64/egg/wheel/cli copying build/lib/wheel/cli/__init__.py -> build/bdist.linux-x86_64/egg/wheel/cli copying build/lib/wheel/cli/convert.py -> build/bdist.linux-x86_64/egg/wheel/cli copying build/lib/wheel/cli/unpack.py -> build/bdist.linux-x86_64/egg/wheel/cli byte-compiling build/bdist.linux-x86_64/egg/wheel/wheelfile.py to wheelfile.pyc byte-compiling build/bdist.linux-x86_64/egg/wheel/pep425tags.py to pep425tags.pyc byte-compiling build/bdist.linux-x86_64/egg/wheel/__init__.py to __init__.pyc byte-compiling build/bdist.linux-x86_64/egg/wheel/util.py to util.pyc byte-compiling build/bdist.linux-x86_64/egg/wheel/metadata.py to metadata.pyc byte-compiling build/bdist.linux-x86_64/egg/wheel/__main__.py to __main__.pyc byte-compiling build/bdist.linux-x86_64/egg/wheel/bdist_wheel.py to bdist_wheel.pyc byte-compiling build/bdist.linux-x86_64/egg/wheel/pkginfo.py to pkginfo.pyc byte-compiling build/bdist.linux-x86_64/egg/wheel/cli/install.py to install.pyc byte-compiling build/bdist.linux-x86_64/egg/wheel/cli/pack.py to pack.pyc byte-compiling build/bdist.linux-x86_64/egg/wheel/cli/__init__.py to __init__.pyc byte-compiling build/bdist.linux-x86_64/egg/wheel/cli/convert.py to convert.pyc byte-compiling build/bdist.linux-x86_64/egg/wheel/cli/unpack.py to unpack.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO copying wheel.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO copying wheel.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying wheel.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying wheel.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying wheel.egg-info/not-zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO copying wheel.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying wheel.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO creating dist creating 'dist/wheel-0.32.3-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) Processing wheel-0.32.3-py2.7.egg creating /usr/lib/python2.7/site-packages/wheel-0.32.3-py2.7.egg Extracting wheel-0.32.3-py2.7.egg to /usr/lib/python2.7/site-packages Adding wheel 0.32.3 to easy-install.pth file Installing wheel script to /usr/bin Installed /usr/lib/python2.7/site-packages/wheel-0.32.3-py2.7.egg Processing dependencies for wheel==0.32.3 Finished processing dependencies for wheel==0.32.3 [root@node101 wheel-0.32.3]#
5.3>.安装cryptography
[root@node101 ~]# wget https://files.pythonhosted.org/packages/7f/ba/383b51cc26e3141c689ce988814385c7659f5ba01c4b5f2de38233010b5f/cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl --2019-01-10 18:23:46-- https://files.pythonhosted.org/packages/7f/ba/383b51cc26e3141c689ce988814385c7659f5ba01c4b5f2de38233010b5f/cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl Resolving files.pythonhosted.org (files.pythonhosted.org)... 151.101.73.63, 2a04:4e42:11::319 Connecting to files.pythonhosted.org (files.pythonhosted.org)|151.101.73.63|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 2138565 (2.0M) [binary/octet-stream] Saving to: ‘cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl’ 100%[================================================================================================================================>] 2,138,565 47.2KB/s in 24s 2019-01-10 18:24:11 (85.8 KB/s) - ‘cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl’ saved [2138565/2138565] [root@node101 ~]#
[root@node101 ~]# pip install cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl Processing ./cryptography-2.4.2-cp27-cp27mu-manylinux1_x86_64.whl Collecting idna>=2.1 (from cryptography==2.4.2) Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB) 100% |████████████████████████████████| 61kB 248kB/s Requirement already satisfied: enum34; python_version < "3" in /usr/lib/python2.7/site-packages (from cryptography==2.4.2) (1.0.4) Collecting cffi!=1.11.3,>=1.7 (from cryptography==2.4.2) Downloading https://files.pythonhosted.org/packages/14/dd/3e7a1e1280e7d767bd3fa15791759c91ec19058ebe31217fe66f3e9a8c49/cffi-1.11.5-cp27-cp27mu-manylinux1_x86_64.whl (407kB) 100% |████████████████████████████████| 409kB 147kB/s Requirement already satisfied: six>=1.4.1 in /usr/lib/python2.7/site-packages (from cryptography==2.4.2) (1.9.0) Collecting ipaddress; python_version < "3" (from cryptography==2.4.2) Downloading https://files.pythonhosted.org/packages/fc/d0/7fc3a811e011d4b388be48a0e381db8d990042df54aa4ef4599a31d39853/ipaddress-1.0.22-py2.py3-none-any.whl Collecting asn1crypto>=0.21.0 (from cryptography==2.4.2) Downloading https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl (101kB) 100% |████████████████████████████████| 102kB 292kB/s Requirement already satisfied: pycparser in /usr/lib/python2.7/site-packages (from cffi!=1.11.3,>=1.7->cryptography==2.4.2) (2.14) Installing collected packages: idna, cffi, ipaddress, asn1crypto, cryptography Found existing installation: cffi 0.8.6 Uninstalling cffi-0.8.6: Successfully uninstalled cffi-0.8.6 Found existing installation: cryptography 0.8.2 Uninstalling cryptography-0.8.2: Successfully uninstalled cryptography-0.8.2 Successfully installed asn1crypto-0.24.0 cffi-1.11.5 cryptography-2.4.2 idna-2.8 ipaddress-1.0.22 [root@node101 ~]#
6>.使用ansible同步文件到其他节点
[root@node101 ~]# ansible all -m ping node102.yinzhengjie.org.cn | SUCCESS => { "changed": false, "ping": "pong" } node103.yinzhengjie.org.cn | SUCCESS => { "changed": false, "ping": "pong" } node101.yinzhengjie.org.cn | SUCCESS => { "changed": false, "ping": "pong" } [root@node101 ~]#
[root@node101 ~]# ansible datanode -m copy -a 'src=/etc/hosts dest=/etc/hosts' node101.yinzhengjie.org.cn | SUCCESS => { "changed": false, "checksum": "6b35ffc3c04d3ec2d3464b74ea0f78c20f27131a", "dest": "/etc/hosts", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/hosts", "secontext": "system_u:object_r:net_conf_t:s0", "size": 279, "state": "file", "uid": 0 } node102.yinzhengjie.org.cn | CHANGED => { "changed": true, "checksum": "6b35ffc3c04d3ec2d3464b74ea0f78c20f27131a", "dest": "/etc/hosts", "gid": 0, "group": "root", "md5sum": "494c828fc321572368ec771a390f90c1", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:net_conf_t:s0", "size": 279, "src": "/root/.ansible/tmp/ansible-tmp-1547116266.5-263369209847793/source", "state": "file", "uid": 0 } node103.yinzhengjie.org.cn | CHANGED => { "changed": true, "checksum": "6b35ffc3c04d3ec2d3464b74ea0f78c20f27131a", "dest": "/etc/hosts", "gid": 0, "group": "root", "md5sum": "494c828fc321572368ec771a390f90c1", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:net_conf_t:s0", "size": 279, "src": "/root/.ansible/tmp/ansible-tmp-1547116266.52-44859007354579/source", "state": "file", "uid": 0 } [root@node101 ~]# [root@node101 ~]# ansible all -m shell -a 'cat /etc/hosts|grep yinzhengjie' node103.yinzhengjie.org.cn | CHANGED | rc=0 >> 172.30.1.101 node101.yinzhengjie.org.cn 172.30.1.102 node102.yinzhengjie.org.cn 172.30.1.103 node103.yinzhengjie.org.cn node101.yinzhengjie.org.cn | CHANGED | rc=0 >> 172.30.1.101 node101.yinzhengjie.org.cn 172.30.1.102 node102.yinzhengjie.org.cn 172.30.1.103 node103.yinzhengjie.org.cn node102.yinzhengjie.org.cn | CHANGED | rc=0 >> 172.30.1.101 node101.yinzhengjie.org.cn 172.30.1.102 node102.yinzhengjie.org.cn 172.30.1.103 node103.yinzhengjie.org.cn [root@node101 ~]#
二.安装jdk环境
1>.卸载jdk1.7版本
[root@node101 ~]# ansible all -m shell -a 'rpm -qa | grep java' [WARNING]: Consider using the yum, dnf or zypper module rather than running rpm. If you need to use command because yum, dnf or zypper is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node102.yinzhengjie.org.cn | CHANGED | rc=0 >> javapackages-tools-3.4.1-11.el7.noarch tzdata-java-2015g-1.el7.noarch java-1.7.0-openjdk-headless-1.7.0.91-2.6.2.3.el7.x86_64 python-javapackages-3.4.1-11.el7.noarch java-1.7.0-openjdk-1.7.0.91-2.6.2.3.el7.x86_64 node103.yinzhengjie.org.cn | CHANGED | rc=0 >> javapackages-tools-3.4.1-11.el7.noarch tzdata-java-2015g-1.el7.noarch java-1.7.0-openjdk-headless-1.7.0.91-2.6.2.3.el7.x86_64 python-javapackages-3.4.1-11.el7.noarch java-1.7.0-openjdk-1.7.0.91-2.6.2.3.el7.x86_64 node101.yinzhengjie.org.cn | CHANGED | rc=0 >> javapackages-tools-3.4.1-11.el7.noarch tzdata-java-2015g-1.el7.noarch java-1.7.0-openjdk-headless-1.7.0.91-2.6.2.3.el7.x86_64 python-javapackages-3.4.1-11.el7.noarch java-1.7.0-openjdk-1.7.0.91-2.6.2.3.el7.x86_64 [root@node101 ~]#
[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps javapackages-tools-3.4.1-11.el7.noarch' [WARNING]: Consider using the yum, dnf or zypper module rather than running rpm. If you need to use command because yum, dnf or zypper is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> node103.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 ~]#
[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps tzdata-java-2015g-1.el7.noarch' [WARNING]: Consider using the yum, dnf or zypper module rather than running rpm. If you need to use command because yum, dnf or zypper is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node103.yinzhengjie.org.cn | CHANGED | rc=0 >> node101.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 ~]#
[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps java-1.7.0-openjdk-headless-1.7.0.91-2.6.2.3.el7.x86_64' [WARNING]: Consider using the yum, dnf or zypper module rather than running rpm. If you need to use command because yum, dnf or zypper is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> node103.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 ~]#
[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps python-javapackages-3.4.1-11.el7.noarch' [WARNING]: Consider using the yum, dnf or zypper module rather than running rpm. If you need to use command because yum, dnf or zypper is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> node103.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 ~]#
[root@node101 ~]# ansible all -m shell -a 'rpm -e --nodeps java-1.7.0-openjdk-1.7.0.91-2.6.2.3.el7.x86_64' [WARNING]: Consider using the yum, dnf or zypper module rather than running rpm. If you need to use command because yum, dnf or zypper is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> node103.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 ~]#
[root@node101 ~]# ansible all -m shell -a 'rpm -qa | grep java' [WARNING]: Consider using the yum, dnf or zypper module rather than running rpm. If you need to use command because yum, dnf or zypper is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node103.yinzhengjie.org.cn | FAILED | rc=1 >> non-zero return code node102.yinzhengjie.org.cn | FAILED | rc=1 >> non-zero return code node101.yinzhengjie.org.cn | FAILED | rc=1 >> non-zero return code [root@node101 ~]#
2>.安装jdk1.8版本几以上版本
大家可以自行去Oracle官网下载相应的jdk，我这里给出下载地址：https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html（推荐下载tar.gz的安装包）。
[root@node101 ~]# ansible all -m shell -a 'mkdir -pv /yinzhengjie/softwares' [WARNING]: Consider using the file module with state=directory rather than running mkdir. If you need to use command because file is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node103.yinzhengjie.org.cn | CHANGED | rc=0 >> mkdir: created directory ‘/yinzhengjie’ mkdir: created directory ‘/yinzhengjie/softwares’ node101.yinzhengjie.org.cn | CHANGED | rc=0 >> mkdir: created directory ‘/yinzhengjie’ mkdir: created directory ‘/yinzhengjie/softwares’ node102.yinzhengjie.org.cn | CHANGED | rc=0 >> mkdir: created directory ‘/yinzhengjie’ mkdir: created directory ‘/yinzhengjie/softwares’ [root@node101 ~]#
[root@node101 ~]# ansible namenode -m shell -a 'tar -zxf ~/jdk-8u131-linux-x64.tar.gz -C /yinzhengjie/softwares' [WARNING]: Consider using the unarchive module rather than running tar. If you need to use command because unarchive is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 ~]# ls /yinzhengjie/softwares/ jdk1.8.0_131 [root@node101 ~]# [root@node101 ~]# ll /yinzhengjie/softwares/jdk1.8.0_131/ total 25884 drwxr-xr-x. 2 10 143 4096 Mar 15 2017 bin -r--r--r--. 1 10 143 3244 Mar 15 2017 COPYRIGHT drwxr-xr-x. 4 10 143 4096 Mar 15 2017 db drwxr-xr-x. 3 10 143 4096 Mar 15 2017 include -rwxr-xr-x. 1 10 143 5097105 Mar 15 2017 javafx-src.zip drwxr-xr-x. 5 10 143 4096 Mar 15 2017 jre drwxr-xr-x. 5 10 143 4096 Mar 15 2017 lib -r--r--r--. 1 10 143 40 Mar 15 2017 LICENSE drwxr-xr-x. 4 10 143 4096 Mar 15 2017 man -r--r--r--. 1 10 143 159 Mar 15 2017 README.html -rw-r--r--. 1 10 143 526 Mar 15 2017 release -rw-r--r--. 1 10 143 21115141 Mar 15 2017 src.zip -rwxr-xr-x. 1 10 143 63933 Mar 15 2017 THIRDPARTYLICENSEREADME-JAVAFX.txt -r--r--r--. 1 10 143 177094 Mar 15 2017 THIRDPARTYLICENSEREADME.txt [root@node101 ~]#
[root@node101 ~]# ansible datanode -m copy -a 'src=/yinzhengjie/softwares/jdk1.8.0_131 dest=/yinzhengjie/softwares' node101.yinzhengjie.org.cn | CHANGED => { "changed": true, "dest": "/yinzhengjie/softwares/", "src": "/yinzhengjie/softwares/jdk1.8.0_131" } node103.yinzhengjie.org.cn | CHANGED => { "changed": true, "dest": "/yinzhengjie/softwares/", "src": "/yinzhengjie/softwares/jdk1.8.0_131" } node102.yinzhengjie.org.cn | CHANGED => { "changed": true, "dest": "/yinzhengjie/softwares/", "src": "/yinzhengjie/softwares/jdk1.8.0_131" } [root@node101 ~]#
[root@node101 ~]# ansible datanode -m shell -a 'ln -s /yinzhengjie/softwares/jdk1.8.0_131 /yinzhengjie/softwares/jdk' [WARNING]: Consider using the file module with state=link rather than running ln. If you need to use command because file is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc=0 >> node103.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 ~]#
3>.配置环境java的系统环境变量
[root@node101 ~]# tail -3 /etc/profile #ADD JAVA_PATH JAVA_HOME=/yinzhengjie/softwares/jdk PATH=$PATH:$JAVA_HOME/bin [root@node101 ~]# [root@node101 ~]# ansible datanode -m copy -a 'src=/etc/profile dest=/etc/profile' node103.yinzhengjie.org.cn | CHANGED => { "changed": true, "checksum": "5b3a42ee16d78583b834dc0dea67d282b5fa5323", "dest": "/etc/profile", "gid": 0, "group": "root", "md5sum": "cd04b3ecdde83a7c250b923167dc0c01", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 1828, "src": "/root/.ansible/tmp/ansible-tmp-1547122009.37-153584791304684/source", "state": "file", "uid": 0 } node102.yinzhengjie.org.cn | CHANGED => { "changed": true, "checksum": "5b3a42ee16d78583b834dc0dea67d282b5fa5323", "dest": "/etc/profile", "gid": 0, "group": "root", "md5sum": "cd04b3ecdde83a7c250b923167dc0c01", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 1828, "src": "/root/.ansible/tmp/ansible-tmp-1547122009.34-130374655280496/source", "state": "file", "uid": 0 } node101.yinzhengjie.org.cn | SUCCESS => { "changed": false, "checksum": "5b3a42ee16d78583b834dc0dea67d282b5fa5323", "dest": "/etc/profile", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/profile", "secontext": "system_u:object_r:etc_t:s0", "size": 1828, "state": "file", "uid": 0 } [root@node101 ~]#
[root@node101 ~]# ansible datanode -m shell -a 'chmod -R 777 /yinzhengjie/softwares/jdk/bin' [WARNING]: Consider using the file module with mode rather than running chmod. If you need to use command because file is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node103.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> node101.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 ~]#
[root@node101 ~]# ansible datanode -m shell -a 'source /etc/profile;java -version' node101.yinzhengjie.org.cn | CHANGED | rc=0 >> java version "1.8.0_131" Java(TM) SE Runtime Environment (build 1.8.0_131-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode) node103.yinzhengjie.org.cn | CHANGED | rc=0 >> java version "1.8.0_131" Java(TM) SE Runtime Environment (build 1.8.0_131-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode) node102.yinzhengjie.org.cn | CHANGED | rc=0 >> java version "1.8.0_131" Java(TM) SE Runtime Environment (build 1.8.0_131-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode) [root@node101 ~]#
[root@node101 ~]# ansible datanode -m shell -a 'mkdir /usr/java' [WARNING]: Consider using the file module with state=directory rather than running mkdir. If you need to use command because file is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc=0 >> node103.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 ~]#
[root@node101 ~]# ansible datanode -m shell -a 'ln -s /yinzhengjie/softwares/jdk1.8.0_131/ /usr/java/jdk1.8' [WARNING]: Consider using the file module with state=link rather than running ln. If you need to use command because file is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc=0 >> node103.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 ~]#
[root@node101 ~]# ansible datanode -m shell -a 'ln -s /yinzhengjie/softwares/jdk1.8.0_131/ /usr/java/defalut' [WARNING]: Consider using the file module with state=link rather than running ln. If you need to use command because file is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node103.yinzhengjie.org.cn | CHANGED | rc=0 >> node101.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 ~]#
[root@node101 ~]# ansible datanode -m shell -a 'ls -l /usr/java' node101.yinzhengjie.org.cn | CHANGED | rc=0 >> total 0 lrwxrwxrwx. 1 root root 36 Jan 10 20:55 defalut -> /yinzhengjie/softwares/jdk1.8.0_131/ lrwxrwxrwx. 1 root root 36 Jan 10 20:55 jdk1.8 -> /yinzhengjie/softwares/jdk1.8.0_131/ node103.yinzhengjie.org.cn | CHANGED | rc=0 >> total 0 lrwxrwxrwx. 1 root root 36 Jan 10 20:55 defalut -> /yinzhengjie/softwares/jdk1.8.0_131/ lrwxrwxrwx. 1 root root 36 Jan 10 20:55 jdk1.8 -> /yinzhengjie/softwares/jdk1.8.0_131/ node102.yinzhengjie.org.cn | CHANGED | rc=0 >> total 0 lrwxrwxrwx. 1 root root 36 Jan 10 20:55 defalut -> /yinzhengjie/softwares/jdk1.8.0_131/ lrwxrwxrwx. 1 root root 36 Jan 10 20:55 jdk1.8 -> /yinzhengjie/softwares/jdk1.8.0_131/ [root@node101 ~]#
三.安装mysql
1>.下载mysql的yum源
[root@node101 ~]# wget http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm --2019-01-10 21:08:45-- http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm Resolving dev.mysql.com (dev.mysql.com)... 137.254.60.11 Connecting to dev.mysql.com (dev.mysql.com)|137.254.60.11|:80... connected. HTTP request sent, awaiting response... 301 Moved Permanently Location: https://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm [following] --2019-01-10 21:08:45-- https://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm Connecting to dev.mysql.com (dev.mysql.com)|137.254.60.11|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://repo.mysql.com//mysql-community-release-el7-5.noarch.rpm [following] --2019-01-10 21:08:48-- https://repo.mysql.com//mysql-community-release-el7-5.noarch.rpm Resolving repo.mysql.com (repo.mysql.com)... 104.76.8.253 Connecting to repo.mysql.com (repo.mysql.com)|104.76.8.253|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 6140 (6.0K) [application/x-redhat-package-manager] Saving to: ‘mysql-community-release-el7-5.noarch.rpm’ 100%[================================================================================================================================>] 6,140 --.-K/s in 0s 2019-01-10 21:08:50 (529 MB/s) - ‘mysql-community-release-el7-5.noarch.rpm’ saved [6140/6140] [root@node101 ~]# [root@node101 ~]# ll total 8 -rw-r--r--. 1 root root 6140 Nov 12 2015 mysql-community-release-el7-5.noarch.rpm [root@node101 ~]#
2>.安装mysql的yum源
[root@node101 ~]# ll total 8 -rw-r--r--. 1 root root 6140 Nov 12 2015 mysql-community-release-el7-5.noarch.rpm [root@node101 ~]# [root@node101 ~]# rpm -ivh mysql-community-release-el7-5.noarch.rpm Preparing... ################################# [100%] Updating / installing... 1:mysql-community-release-el7-5 ################################# [100%] [root@node101 ~]# [root@node101 ~]# ll /etc/yum.repos.d/ | grep mysql -rw-r--r--. 1 root root 1209 Jan 29 2014 mysql-community.repo -rw-r--r--. 1 root root 1060 Jan 29 2014 mysql-community-source.repo [root@node101 ~]#
3>.安装mysql服务
[root@node101 ~]# yum -y install mysql-community-server Loaded plugins: fastestmirror, langpacks mysql-connectors-community | 2.5 kB 00:00:00 mysql-tools-community | 2.5 kB 00:00:01 mysql56-community | 2.5 kB 00:00:02 (1/3): mysql-tools-community/x86_64/primary_db | 48 kB 00:00:00 (2/3): mysql-connectors-community/x86_64/primary_db | 29 kB 00:00:00 (3/3): mysql56-community/x86_64/primary_db | 209 kB 00:00:03 Loading mirror speeds from cached hostfile * base: mirror.jdcloud.com * epel: mirrors.yun-idc.com * extras: mirror.jdcloud.com * updates: mirrors.tuna.tsinghua.edu.cn Resolving Dependencies --> Running transaction check ---> Package mysql-community-server.x86_64 0:5.6.42-2.el7 will be installed --> Processing Dependency: mysql-community-common(x86-64) = 5.6.42-2.el7 for package: mysql-community-server-5.6.42-2.el7.x86_64 --> Processing Dependency: mysql-community-client(x86-64) >= 5.6.10 for package: mysql-community-server-5.6.42-2.el7.x86_64 --> Running transaction check ---> Package mysql-community-client.x86_64 0:5.6.42-2.el7 will be installed --> Processing Dependency: mysql-community-libs(x86-64) >= 5.6.10 for package: mysql-community-client-5.6.42-2.el7.x86_64 ---> Package mysql-community-common.x86_64 0:5.6.42-2.el7 will be installed --> Running transaction check ---> Package mariadb-libs.x86_64 1:5.5.44-2.el7.centos will be obsoleted ---> Package mysql-community-libs.x86_64 0:5.6.42-2.el7 will be obsoleting --> Finished Dependency Resolution Dependencies Resolved ========================================================================================================================================================================== Package Arch Version Repository Size ========================================================================================================================================================================== Installing: mysql-community-libs x86_64 5.6.42-2.el7 mysql56-community 2.0 M replacing mariadb-libs.x86_64 1:5.5.44-2.el7.centos mysql-community-server x86_64 5.6.42-2.el7 mysql56-community 59 M Installing for dependencies: mysql-community-client x86_64 5.6.42-2.el7 mysql56-community 20 M mysql-community-common x86_64 5.6.42-2.el7 mysql56-community 257 k Transaction Summary ========================================================================================================================================================================== Install 2 Packages (+2 Dependent packages) Total download size: 81 M Downloading packages: warning: /var/cache/yum/x86_64/7/mysql56-community/packages/mysql-community-common-5.6.42-2.el7.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 5072e1f5: NOKEY02:24 ETA Public key for mysql-community-common-5.6.42-2.el7.x86_64.rpm is not installed (1/4): mysql-community-common-5.6.42-2.el7.x86_64.rpm | 257 kB 00:00:01 (2/4): mysql-community-libs-5.6.42-2.el7.x86_64.rpm | 2.0 MB 00:00:10 (3/4): mysql-community-client-5.6.42-2.el7.x86_64.rpm | 20 MB 00:03:29 (4/4): mysql-community-server-5.6.42-2.el7.x86_64.rpm | 59 MB 00:05:01 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 264 kB/s | 81 MB 00:05:14 Retrieving key from file:/etc/pki/rpm-gpg/RPM-GPG-KEY-mysql Importing GPG key 0x5072E1F5: Userid : "MySQL Release Engineering <mysql-build@oss.oracle.com>" Fingerprint: a4a9 4068 76fc bd3c 4567 70c8 8c71 8d3b 5072 e1f5 Package : mysql-community-release-el7-5.noarch (installed) From : file:/etc/pki/rpm-gpg/RPM-GPG-KEY-mysql Running transaction check Running transaction test Transaction test succeeded Running transaction Warning: RPMDB altered outside of yum. ** Found 2 pre-existing rpmdb problem(s), 'yum check' output follows: 1:libreoffice-core-4.3.7.2-5.el7.x86_64 has missing requires of java-headless >= ('1', '1.6', None) 1:libreoffice-ure-4.3.7.2-5.el7.x86_64 has missing requires of libjvm.so()(64bit) Installing : mysql-community-common-5.6.42-2.el7.x86_64 1/5 Installing : mysql-community-libs-5.6.42-2.el7.x86_64 2/5 Installing : mysql-community-client-5.6.42-2.el7.x86_64 3/5 Installing : mysql-community-server-5.6.42-2.el7.x86_64 4/5 Erasing : 1:mariadb-libs-5.5.44-2.el7.centos.x86_64 5/5 Verifying : mysql-community-libs-5.6.42-2.el7.x86_64 1/5 Verifying : mysql-community-common-5.6.42-2.el7.x86_64 2/5 Verifying : mysql-community-client-5.6.42-2.el7.x86_64 3/5 Verifying : mysql-community-server-5.6.42-2.el7.x86_64 4/5 Verifying : 1:mariadb-libs-5.5.44-2.el7.centos.x86_64 5/5 Installed: mysql-community-libs.x86_64 0:5.6.42-2.el7 mysql-community-server.x86_64 0:5.6.42-2.el7 Dependency Installed: mysql-community-client.x86_64 0:5.6.42-2.el7 mysql-community-common.x86_64 0:5.6.42-2.el7 Replaced: mariadb-libs.x86_64 1:5.5.44-2.el7.centos Complete! [root@node101 ~]#
4>.启动mysql服务
[root@node101 ~]# systemctl enable mysqld [root@node101 ~]# [root@node101 ~]# systemctl start mysqld [root@node101 ~]# [root@node101 ~]# [root@node101 ~]# systemctl status mysqld ● mysqld.service - MySQL Community Server Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled) Active: active (running) since Thu 2019-01-10 21:40:29 HKT; 8s ago Process: 10810 ExecStartPost=/usr/bin/mysql-systemd-start post (code=exited, status=0/SUCCESS) Process: 10730 ExecStartPre=/usr/bin/mysql-systemd-start pre (code=exited, status=0/SUCCESS) Main PID: 10809 (mysqld_safe) CGroup: /system.slice/mysqld.service ├─10809 /bin/sh /usr/bin/mysqld_safe --basedir=/usr └─10974 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib64/mysql/plugin --log-error=/var/log/mysqld.log --pid-file=/var/run/my... Jan 10 21:40:28 node101.yinzhengjie.org.cn mysql-systemd-start[10730]: PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER ! Jan 10 21:40:28 node101.yinzhengjie.org.cn mysql-systemd-start[10730]: To do so, start the server, then issue the following commands: Jan 10 21:40:28 node101.yinzhengjie.org.cn mysql-systemd-start[10730]: /usr/bin/mysqladmin -u root password 'new-password' Jan 10 21:40:28 node101.yinzhengjie.org.cn mysql-systemd-start[10730]: /usr/bin/mysqladmin -u root -h node101.yinzhengjie.org.cn password 'new-password' Jan 10 21:40:28 node101.yinzhengjie.org.cn mysql-systemd-start[10730]: Alternatively you can run: Jan 10 21:40:28 node101.yinzhengjie.org.cn mysql-systemd-start[10730]: /usr/bin/mysql_secure_installation Jan 10 21:40:28 node101.yinzhengjie.org.cn mysql-systemd-start[10730]: which will also give you the option of removing the test Jan 10 21:40:28 node101.yinzhengjie.org.cn mysqld_safe[10809]: 190110 21:40:28 mysqld_safe Logging to '/var/log/mysqld.log'. Jan 10 21:40:28 node101.yinzhengjie.org.cn mysqld_safe[10809]: 190110 21:40:28 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql Jan 10 21:40:29 node101.yinzhengjie.org.cn systemd[1]: Started MySQL Community Server. [root@node101 ~]# [root@node101 ~]#
5>.为mysql配置管理用户
[root@node101 ~]# mysql -uroot #安装后的操作系统，默认是密钥root密码的，我们直接可以用root用户登陆即可。 Welcome to the MySQL monitor. Commands end with ; or g. Your MySQL connection id is 2 Server version: 5.6.42 MySQL Community Server (GPL) Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. mysql> set password for 'root'@'localhost' = password('yinzhengjie'); #修改root本地用户的登陆密码为“yinzhengjie” Query OK, 0 rows affected (0.00 sec) mysql> grant all privileges on *.* to 'root'@'172.30.1.10%' identified by 'yinzhengjie'; #我这里值允许172.30.1.100～172.30.1.109这10个服务器IP使用root用户登陆我这台mysql。其他的机器不允许。 Query OK, 0 rows affected (0.00 sec) mysql> FLUSH PRIVILEGES; #别忘记刷新权限哟！ Query OK, 0 rows affected (0.00 sec) mysql> quit Bye [root@node101 ~]#
6>.配置一个cdh超级用户权限（这个步骤如果你调过了，在初始化数据库时可能与报错哟！）
[root@node101 mysql]# systemctl start mysqld [root@node101 mysql]# [root@node101 mysql]# systemctl enable mysqld Created symlink from /etc/systemd/system/mysql.service to /usr/lib/systemd/system/mysqld.service. Created symlink from /etc/systemd/system/multi-user.target.wants/mysqld.service to /usr/lib/systemd/system/mysqld.service. [root@node101 mysql]# [root@node101 mysql]# mysql Welcome to the MySQL monitor. Commands end with ; or g. Your MySQL connection id is 2 Server version: 5.6.38 MySQL Community Server (GPL) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or 'h' for help. Type 'c' to clear the current input statement. mysql> GRANT ALL PRIVILEGES ON *.* TO 'cdh'@'172.30.1.10%' IDENTIFIED BY 'yinzhengjie' WITH GRANT OPTION; Query OK, 0 rows affected (0.00 sec) mysql> GRANT ALL PRIVILEGES ON *.* TO 'cdh'@'node101.yinzhengjie.org.cn' IDENTIFIED BY 'yinzhengjie' WITH GRANT OPTION; Query OK, 0 rows affected (0.00 sec) mysql> FLUSH PRIVILEGES; Query OK, 0 rows affected (0.00 sec) mysql> quit Bye [root@node101 mysql]#
四.部署cloudrea manager软件
1>.安装相应的依赖包（每个节点都需要安装哟～）
[root@node101 ~]# ansible datanode -m shell -a 'yum -y install chkconfig python bind-utils psmisc libxslt zlib sqlite cyrus-sasl-plain cyrus-sasl-gssapi fuse fuse-libs redhat-lsb'
2>.安装Cloudera Manager Server，Agent（在所有机器）
2.1>.CDH的选择
下载地址：http://archive.cloudera.com/cdh5/parcels/5.15.1/
2.2>.CM的选择
下载地址：http://archive.cloudera.com/cm5/cm/5/
2.3>.温馨提示：
如果网速较慢的小伙伴可以用不建议使用wget下载，推荐使用迅雷下载再上传到服务器，如果有百度云超级会员的话也可以使用我给大家体现下载好的软件。百度云链接：https://pan.baidu.com/s/148t70GenClGQRtJYX8WjvQ 密码：i2yd
[root@node101 CDH-5.15.1]# ll total 3071760 -rw-r--r--. 1 root root 2120090032 Jan 10 18:55 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel -rw-r--r--. 1 root root 41 Jan 10 18:54 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1 -rw-r--r--. 1 root root 838894986 Jan 10 18:55 cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz -rw-r--r--. 1 root root 185540433 Jan 10 18:53 jdk-8u131-linux-x64.tar.gz -rw-r--r--. 1 root root 73767 Jan 10 18:55 manifest.json -rw-r--r--. 1 root root 855946 Jan 10 18:55 mysql-connector-java-5.1.26.jar [root@node101 CDH-5.15.1]#
[root@node101 CDH-5.15.1]# ansible all -m shell -a 'ls /yinzhengjie/softwares' node103.yinzhengjie.org.cn | CHANGED | rc=0 >> jdk jdk1.8.0_131 node101.yinzhengjie.org.cn | CHANGED | rc=0 >> CDH-5.15.1 jdk jdk1.8.0_131 node102.yinzhengjie.org.cn | CHANGED | rc=0 >> jdk jdk1.8.0_131 [root@node101 CDH-5.15.1]# ansible all -m copy -a 'src=/yinzhengjie/softwares/CDH-5.15.1/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz dest=/yinzhengjie/softwares/' node101.yinzhengjie.org.cn | CHANGED => { "changed": true, "checksum": "4227988280e0f09164bbfc6c59f291c6a814b8be", "dest": "/yinzhengjie/softwares/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz", "gid": 0, "group": "root", "md5sum": "ca0be616f62668915d9ceeb22bdcdd4b", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:default_t:s0", "size": 838894986, "src": "/root/.ansible/tmp/ansible-tmp-1547172382.51-139484190159118/source", "state": "file", "uid": 0 } node103.yinzhengjie.org.cn | CHANGED => { "changed": true, "checksum": "4227988280e0f09164bbfc6c59f291c6a814b8be", "dest": "/yinzhengjie/softwares/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz", "gid": 0, "group": "root", "md5sum": "ca0be616f62668915d9ceeb22bdcdd4b", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:default_t:s0", "size": 838894986, "src": "/root/.ansible/tmp/ansible-tmp-1547172382.57-61195539797705/source", "state": "file", "uid": 0 } node102.yinzhengjie.org.cn | CHANGED => { "changed": true, "checksum": "4227988280e0f09164bbfc6c59f291c6a814b8be", "dest": "/yinzhengjie/softwares/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz", "gid": 0, "group": "root", "md5sum": "ca0be616f62668915d9ceeb22bdcdd4b", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:default_t:s0", "size": 838894986, "src": "/root/.ansible/tmp/ansible-tmp-1547172382.54-248038987426177/source", "state": "file", "uid": 0 } [root@node101 CDH-5.15.1]# [root@node101 CDH-5.15.1]# [root@node101 CDH-5.15.1]# [root@node101 CDH-5.15.1]# ansible all -m shell -a 'ls /yinzhengjie/softwares' node101.yinzhengjie.org.cn | CHANGED | rc=0 >> CDH-5.15.1 cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz jdk jdk1.8.0_131 node103.yinzhengjie.org.cn | CHANGED | rc=0 >> cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz jdk jdk1.8.0_131 node102.yinzhengjie.org.cn | CHANGED | rc=0 >> cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz jdk jdk1.8.0_131 [root@node101 CDH-5.15.1]#
[root@node101 CDH-5.15.1]# ansible all -m shell -a 'mkdir /opt/cloudera-manager' [WARNING]: Consider using the file module with state=directory rather than running mkdir. If you need to use command because file is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node103.yinzhengjie.org.cn | CHANGED | rc=0 >> node101.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 CDH-5.15.1]#
[root@node101 softwares]# ansible all -m shell -a 'tar -zxf /yinzhengjie/softwares/cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz -C /opt/cloudera-manager' [WARNING]: Consider using the unarchive module rather than running tar. If you need to use command because unarchive is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> node103.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 softwares]# [root@node101 softwares]# ansible all -m shell -a 'ls -l /opt/cloudera-manager' node101.yinzhengjie.org.cn | CHANGED | rc=0 >> total 8 drwxr-xr-x. 4 1106 4001 4096 Jul 31 18:28 cloudera drwxr-xr-x. 9 1106 4001 4096 Jul 31 18:28 cm-5.15.1 node102.yinzhengjie.org.cn | CHANGED | rc=0 >> total 8 drwxr-xr-x. 4 1106 4001 4096 Jul 31 18:28 cloudera drwxr-xr-x. 9 1106 4001 4096 Jul 31 18:28 cm-5.15.1 node103.yinzhengjie.org.cn | CHANGED | rc=0 >> total 8 drwxr-xr-x. 4 1106 4001 4096 Jul 31 18:28 cloudera drwxr-xr-x. 9 1106 4001 4096 Jul 31 18:28 cm-5.15.1 [root@node101 softwares]#
[root@node101 softwares]# ansible all -m shell -a 'useradd --system --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm' node102.yinzhengjie.org.cn | CHANGED | rc=0 >> node101.yinzhengjie.org.cn | CHANGED | rc=0 >> node103.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 softwares]# [root@node101 softwares]# [root@node101 softwares]# ansible all -m shell -a 'id cloudera-scm' node101.yinzhengjie.org.cn | CHANGED | rc=0 >> uid=986(cloudera-scm) gid=981(cloudera-scm) groups=981(cloudera-scm) node102.yinzhengjie.org.cn | CHANGED | rc=0 >> uid=986(cloudera-scm) gid=981(cloudera-scm) groups=981(cloudera-scm) node103.yinzhengjie.org.cn | CHANGED | rc=0 >> uid=986(cloudera-scm) gid=981(cloudera-scm) groups=981(cloudera-scm) [root@node101 softwares]#
3>.配置CM的Agent端
[root@node101 softwares]# ansible all -m shell -a 'grep server_host /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini' node103.yinzhengjie.org.cn | CHANGED | rc=0 >> server_host=localhost node102.yinzhengjie.org.cn | CHANGED | rc=0 >> server_host=localhost node101.yinzhengjie.org.cn | CHANGED | rc=0 >> server_host=localhost [root@node101 softwares]# [root@node101 softwares]# ansible all -m shell -a 'sed -i 's/server_host=localhost/server_host=node101.yinzhengjie.org.cn/g' /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini' [WARNING]: Consider using the replace, lineinfile or template module rather than running sed. If you need to use command because replace, lineinfile or template is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node103.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> node101.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 softwares]# [root@node101 softwares]# ansible all -m shell -a 'grep server_host /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini' node101.yinzhengjie.org.cn | CHANGED | rc=0 >> server_host=node101.yinzhengjie.org.cn node103.yinzhengjie.org.cn | CHANGED | rc=0 >> server_host=node101.yinzhengjie.org.cn node102.yinzhengjie.org.cn | CHANGED | rc=0 >> server_host=node101.yinzhengjie.org.cn [root@node101 softwares]#
4>.初始化CM Server的数据库
[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'mkdir /usr/share/java' [WARNING]: Consider using the file module with state=directory rather than running mkdir. If you need to use command because file is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 CDH-5.15.1]#
[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'ls -l /usr/share/java' node101.yinzhengjie.org.cn | CHANGED | rc=0 >> total 0 [root@node101 CDH-5.15.1]# [root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'mv /yinzhengjie/softwares/CDH-5.15.1/mysql-connector-java-5.1.26.jar /usr/share/java/mysql-connector-java.jar' node101.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'ls -l /usr/share/java' node101.yinzhengjie.org.cn | CHANGED | rc=0 >> total 836 -rw-r--r--. 1 root root 855946 Jan 10 18:55 mysql-connector-java.jar [root@node101 CDH-5.15.1]#
[root@node101 CDH-5.15.1]# ansible namenode -m shell -a '/opt/cloudera-manager/cm-5.15.1/share/cmf/schema/scm_prepare_database.sh mysql cdh -h node101.yinzhengjie.org.cn -ucdh -pyinzhengjie --scm-host node101.yinzhengjie.org.cn scm scm scm' node101.yinzhengjie.org.cn | CHANGED | rc=0 >> JAVA_HOME=/usr/java/jdk1.8 Verifying that we can write to /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-server Creating SCM configuration file in /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-server Executing: /usr/java/jdk1.8/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/opt/cloudera-manager/cm-5.15.1/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db. [ main] DbCommandExecutor INFO Successfully connected to database. All done, your SCM database is configured correctly! [root@node101 CDH-5.15.1]#
5>.制作CDH本地源
[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'mkdir -p /opt/cloudera/parcel-repo' [WARNING]: Consider using the file module with state=directory rather than running mkdir. If you need to use command because file is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 CDH-5.15.1]#
[root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'chown cloudera-scm:cloudera-scm /opt/cloudera/parcel-repo/' [WARNING]: Consider using the file module with owner rather than running chown. If you need to use command because file is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 CDH-5.15.1]# [root@node101 CDH-5.15.1]# ansible namenode -m shell -a 'ls -ld /opt/cloudera/parcel-repo/' node101.yinzhengjie.org.cn | CHANGED | rc=0 >> drwxr-xr-x. 2 cloudera-scm cloudera-scm 4096 Jan 11 10:42 /opt/cloudera/parcel-repo/ [root@node101 CDH-5.15.1]#
[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'mkdir -p /opt/cloudera/parcels' [WARNING]: Consider using the file module with state=directory rather than running mkdir. If you need to use command because file is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> node103.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 CDH-5.15.1]# [root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'ls -ld /opt/cloudera/parcels' node102.yinzhengjie.org.cn | CHANGED | rc=0 >> drwxr-xr-x. 2 root root 4096 Jan 11 10:48 /opt/cloudera/parcels node103.yinzhengjie.org.cn | CHANGED | rc=0 >> drwxr-xr-x. 2 root root 4096 Jan 11 10:48 /opt/cloudera/parcels node101.yinzhengjie.org.cn | CHANGED | rc=0 >> drwxr-xr-x. 2 root root 4096 Jan 11 10:48 /opt/cloudera/parcels [root@node101 CDH-5.15.1]#
[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'chown cloudera-scm:cloudera-scm /opt/cloudera/parcels' [WARNING]: Consider using the file module with owner rather than running chown. If you need to use command because file is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc=0 >> node103.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 CDH-5.15.1]# [root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'ls -ld /opt/cloudera/parcels' node101.yinzhengjie.org.cn | CHANGED | rc=0 >> drwxr-xr-x. 2 cloudera-scm cloudera-scm 4096 Jan 11 10:48 /opt/cloudera/parcels node103.yinzhengjie.org.cn | CHANGED | rc=0 >> drwxr-xr-x. 2 cloudera-scm cloudera-scm 4096 Jan 11 10:48 /opt/cloudera/parcels node102.yinzhengjie.org.cn | CHANGED | rc=0 >> drwxr-xr-x. 2 cloudera-scm cloudera-scm 4096 Jan 11 10:48 /opt/cloudera/parcels [root@node101 CDH-5.15.1]#
[root@node101 CDH-5.15.1]# [root@node101 CDH-5.15.1]# cp CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel /opt/cloudera/parcel-repo/ [root@node101 CDH-5.15.1]# [root@node101 CDH-5.15.1]# cp manifest.json /opt/cloudera/parcel-repo/ #别小瞧这个文件，尽管它不是很大，但是它却记录着CDH和Hadoop生态圈组件的版本依赖关系！ [root@node101 CDH-5.15.1]# [root@node101 CDH-5.15.1]# cp CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha1 /opt/cloudera/parcel-repo/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha #注意，我在拷贝到时候重命名该文件了！ [root@node101 CDH-5.15.1]# [root@node101 CDH-5.15.1]# ll /opt/cloudera/parcel-repo/ total 2070484 -rw-r--r-- 1 root root 2120090032 Sep 13 07:40 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel -rw-r--r-- 1 root root 41 Sep 13 07:41 CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha -rw-r--r-- 1 root root 73767 Sep 13 07:41 manifest.json [root@node101 CDH-5.15.1]# 温馨提示： 如果你没有下载到“CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.sha”文件是，可以找到“manifest.json”文件中"parcelName": "CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel"对应的"hash": "deff00898e410a34cf0a1e66c5dbe87546608f0c"复制到该文件即可。这个方法也适用于其他的版本！
五.启动集群
1>.启动cloudera manager的Server 端
[root@node101 CDH-5.15.1]# ansible namenode -m shell -a '/opt/cloudera-manager/cm-5.15.1/etc/init.d/cloudera-scm-server start' node101.yinzhengjie.org.cn | CHANGED | rc=0 >> Starting cloudera-scm-server: [ OK ] [root@node101 CDH-5.15.1]# [root@node101 CDH-5.15.1]# netstat -untalp | grep 7180 tcp 0 0 0.0.0.0:7180 0.0.0.0:* LISTEN 30064/java [root@node101 CDH-5.15.1]#
............（信息会有很多，你会看到日志中启动了很多端口，如果7180默认端口启动成功，那么你就可以访问WebUI界面了，当然这个端口咱们是可以修改的！） 2019-01-11 10:57:52,253 INFO WebServerImpl:com.cloudera.server.web.cmon.JobDetailGatekeeper: ActivityMonitor configured to allow job details for all jobs. 2019-01-11 10:57:53,956 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Discovered parcel on CM server: CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel 2019-01-11 10:57:53,957 INFO ParcelUpdateService:com.cloudera.parcel.components.LocalParcelManagerImpl: Created torrent file: /opt/cloudera/parcel-repo/CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel.torrent 2019-01-11 10:57:53,963 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Creating single-file torrent for CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel... 2019-01-11 10:57:53,965 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: Hashing data from CDH-5.15.1-1.cdh5.15.1.p0.4-el7.parcel with 2 threads (4044 pieces)... 2019-01-11 10:57:54,522 INFO WebServerImpl:com.cloudera.server.web.cmf.AggregatorController: AggregateSummaryScheduler started. 2019-01-11 10:57:55,568 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... 10% complete 2019-01-11 10:57:55,957 ERROR SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: The server storage directory [/var/lib/cloudera-scm-server] doesn't exist. 2019-01-11 10:57:55,965 ERROR SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: No read permission to the server storage directory [/var/lib/cloudera-scm-server] 2019-01-11 10:57:55,965 ERROR SearchRepositoryManager-0:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: No write permission to the server storage directory [/var/lib/cloudera-scm-server] 2019-01-11 10:57:57,025 INFO WebServerImpl:org.mortbay.log: jetty-6.1.26.cloudera.4 2019-01-11 10:57:57,030 INFO WebServerImpl:org.mortbay.log: Started SelectChannelConnector@0.0.0.0:7180 2019-01-11 10:57:57,030 INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server. 2019-01-11 10:57:57,297 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... 20% complete 2019-01-11 10:57:58,330 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... 30% complete 2019-01-11 10:57:59,348 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... 40% complete 2019-01-11 10:58:00,082 INFO ParcelUpdateService:com.turn.ttorrent.common.Torrent: ... 50% complete
2>.启动cloudera manager的Agent 端
[root@node101 CDH-5.15.1]# ansible datanode -m shell -a '/opt/cloudera-manager/cm-5.15.1/etc/init.d/cloudera-scm-agent start' node102.yinzhengjie.org.cn | CHANGED | rc=0 >> Starting cloudera-scm-agent: [ OK ] node101.yinzhengjie.org.cn | CHANGED | rc=0 >> Starting cloudera-scm-agent: [ OK ] node103.yinzhengjie.org.cn | CHANGED | rc=0 >> Starting cloudera-scm-agent: [ OK ] [root@node101 CDH-5.15.1]# [root@node101 CDH-5.15.1]# ansible datanode -m shell -a '/opt/cloudera-manager/cm-5.15.1/etc/init.d/cloudera-scm-agent status' node103.yinzhengjie.org.cn | CHANGED | rc=0 >> cloudera-scm-agent (pid 25342) is running... node102.yinzhengjie.org.cn | CHANGED | rc=0 >> cloudera-scm-agent (pid 25506) is running... node101.yinzhengjie.org.cn | CHANGED | rc=0 >> cloudera-scm-agent (pid 31929) is running... [root@node101 CDH-5.15.1]#
3>.关闭防火墙和selinux
温馨提示：
在实际生产环境中，如果是内网环境咱们可以关闭防火墙和selinux，风向毕竟很小。而且很多公司都是有硬件防火墙的，如果咱们在开启防火墙会影响服务器性能。由于我是测试机环境，为了实验方便就直接禁用这2个服务了，实际生产中，根据你的需求选择是否开启防火墙，到时候你只需要定义一些列端口开放规则即可。
[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'systemctl status firewalld' node101.yinzhengjie.org.cn | CHANGED | rc=0 >> ● firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2019-01-10 17:58:21 HKT; 17h ago Main PID: 937 (firewalld) CGroup: /system.slice/firewalld.service └─937 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid Jan 10 17:58:19 node101.yinzhengjie.org.cn systemd[1]: Starting firewalld - dynamic firewall daemon... Jan 10 17:58:21 node101.yinzhengjie.org.cn systemd[1]: Started firewalld - dynamic firewall daemon. node103.yinzhengjie.org.cn | CHANGED | rc=0 >> ● firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2019-01-10 17:59:02 HKT; 17h ago Main PID: 952 (firewalld) CGroup: /system.slice/firewalld.service └─952 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid Jan 10 17:59:01 node103.yinzhengjie.org.cn systemd[1]: Starting firewalld - dynamic firewall daemon... Jan 10 17:59:02 node103.yinzhengjie.org.cn systemd[1]: Started firewalld - dynamic firewall daemon. node102.yinzhengjie.org.cn | CHANGED | rc=0 >> ● firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2019-01-10 17:58:22 HKT; 17h ago Main PID: 913 (firewalld) CGroup: /system.slice/firewalld.service └─913 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid Jan 10 17:58:21 node102.yinzhengjie.org.cn systemd[1]: Starting firewalld - dynamic firewall daemon... Jan 10 17:58:22 node102.yinzhengjie.org.cn systemd[1]: Started firewalld - dynamic firewall daemon. [root@node101 CDH-5.15.1]#
[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'systemctl stop firewalld' node102.yinzhengjie.org.cn | CHANGED | rc=0 >> node103.yinzhengjie.org.cn | CHANGED | rc=0 >> node101.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 CDH-5.15.1]#
[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'systemctl disable firewalld' node103.yinzhengjie.org.cn | CHANGED | rc=0 >> Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service. node102.yinzhengjie.org.cn | CHANGED | rc=0 >> Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service. Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. node101.yinzhengjie.org.cn | CHANGED | rc=0 >> Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service. Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service. [root@node101 CDH-5.15.1]#
[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'setenforce 0' node103.yinzhengjie.org.cn | CHANGED | rc=0 >> node101.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 CDH-5.15.1]#
[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'getenforce' node101.yinzhengjie.org.cn | CHANGED | rc=0 >> Permissive node102.yinzhengjie.org.cn | CHANGED | rc=0 >> Permissive node103.yinzhengjie.org.cn | CHANGED | rc=0 >> Permissive [root@node101 CDH-5.15.1]#
[root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'sed -i 's#SELINUX=enforcing#SELINUX=disabled#' /etc/selinux/config' [WARNING]: Consider using the replace, lineinfile or template module rather than running sed. If you need to use command because replace, lineinfile or template is insufficient you can add warn=False to this command task or set command_warnings=False in ansible.cfg to get rid of this message. node101.yinzhengjie.org.cn | CHANGED | rc=0 >> node103.yinzhengjie.org.cn | CHANGED | rc=0 >> node102.yinzhengjie.org.cn | CHANGED | rc=0 >> [root@node101 CDH-5.15.1]# [root@node101 CDH-5.15.1]# [root@node101 CDH-5.15.1]# ansible datanode -m shell -a 'cat /etc/selinux/config | grep SELINUX= | grep -v ^#' node103.yinzhengjie.org.cn | CHANGED | rc=0 >> SELINUX=disabled node101.yinzhengjie.org.cn | CHANGED | rc=0 >> SELINUX=disabled node102.yinzhengjie.org.cn | CHANGED | rc=0 >> SELINUX=disabled [root@node101 CDH-5.15.1]#
4>.如下图所示，访问webUI界面（http://node101.yinzhengjie.org.cn:7180/）
我们安装CM的过程通过WebUI的安装向导来进行安装，推荐使用谷歌浏览器，不要使用容易崩溃的浏览器，这样会影响你安装进度的！如果你也出现了以上界面，恭喜你CM部署成功，关于CDH的部署详情请参考：https://www.cnblogs.com/yinzhengjie/p/9638360.html。