大数据技术之Hadoop3.1.2版本HA模式
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.Hadoop的HA特点
1>.主备NameNode
2>.解决单点故障（属性，位置）
1.主NameNode对外提供服务，备NameNode同步主NameNode元数据一带切换； 2.所有DataNode同时向两个NameNode汇报数据块信息（位置）； 3.JNN：集群（属性）； 4.Standby：备，完成了edits.log文件的合并产生新的image，推送回ANN；
3>.两种切换选择
1.手动切换 通过命令实现主备之间的切换，可以用HDFS升级等场合。 2.自动切换 基于Zookeeper实现。
4>.基于zookeeper自动切换方案
1.Zookeeper Failover Controller进程用于监控NameNode监控状态； 2.并向zookeeper注册NameNode； 3.NameNode挂掉后，ZKFC为NameNode竞争锁，获得ZKFC锁的NameNode变为active；
5>.配置Hadoop3.1.2的官方文档（http://hadoop.apache.org/docs/r3.1.2/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html）
二.zookeeper完全分布式集群搭建
1>.下载zookeeper软件（下载地址：https://www.apache.org/dyn/closer.cgi/zookeeper/）
[root@node101.yinzhengjie.org.cn ~]# wget https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz --2019-04-12 08:48:40-- https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.4.14/zookeeper-3.4.14.tar.gz Resolving mirrors.tuna.tsinghua.edu.cn (mirrors.tuna.tsinghua.edu.cn)... 101.6.8.193, 2402:f000:1:408:8100::1 Connecting to mirrors.tuna.tsinghua.edu.cn (mirrors.tuna.tsinghua.edu.cn)|101.6.8.193|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 37676320 (36M) [application/x-gzip] Saving to: ‘zookeeper-3.4.14.tar.gz’ 100%[============================================================================================================================================>] 37,676,320 7.57MB/s in 4.5s 2019-04-12 08:48:45 (7.94 MB/s) - ‘zookeeper-3.4.14.tar.gz’ saved [37676320/37676320] [root@node101.yinzhengjie.org.cn ~]#
2>.加压zookeeper并配置环境变量
[root@node101.yinzhengjie.org.cn ~]# tar -zxf zookeeper-3.4.14.tar.gz -C /yinzhengjie/softwares/ [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/softwares/zookeeper-3.4.14/ total 1672 drwxr-xr-x 2 2002 2002 202 Mar 6 09:09 bin -rw-rw-r-- 1 2002 2002 97426 Mar 6 08:50 build.xml drwxr-xr-x 2 2002 2002 77 Mar 6 09:09 conf drwxr-xr-x 2 2002 2002 4096 Mar 6 09:10 dist-maven -rw-rw-r-- 1 2002 2002 1709 Mar 6 08:50 ivysettings.xml -rw-rw-r-- 1 2002 2002 10742 Mar 6 08:50 ivy.xml drwxr-xr-x 4 2002 2002 308 Mar 6 09:09 lib -rw-rw-r-- 1 2002 2002 11970 Mar 6 08:50 LICENSE.txt -rw-rw-r-- 1 2002 2002 3132 Mar 6 08:50 NOTICE.txt -rw-rw-r-- 1 2002 2002 31622 Mar 6 08:50 pom.xml -rw-rw-r-- 1 2002 2002 1765 Mar 6 08:50 README.md -rw-rw-r-- 1 2002 2002 1770 Mar 6 08:50 README_packaging.txt drwxr-xr-x 3 2002 2002 22 Mar 6 08:50 src -rw-rw-r-- 1 2002 2002 1515359 Mar 6 08:50 zookeeper-3.4.14.jar -rw-rw-r-- 1 2002 2002 836 Mar 6 09:10 zookeeper-3.4.14.jar.asc -rw-rw-r-- 1 2002 2002 33 Mar 6 08:50 zookeeper-3.4.14.jar.md5 -rw-rw-r-- 1 2002 2002 41 Mar 6 08:50 zookeeper-3.4.14.jar.sha1 drwxr-xr-x 3 2002 2002 47 Mar 6 09:09 zookeeper-client drwxr-xr-x 12 2002 2002 4096 Mar 6 09:09 zookeeper-contrib drwxr-xr-x 7 2002 2002 4096 Mar 6 09:09 zookeeper-docs drwxr-xr-x 3 2002 2002 35 Mar 6 09:09 zookeeper-it drwxr-xr-x 4 2002 2002 46 Mar 6 09:09 zookeeper-jute drwxr-xr-x 5 2002 2002 176 Mar 6 09:09 zookeeper-recipes drwxr-xr-x 3 2002 2002 32 Mar 6 09:09 zookeeper-server [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# tail -3 /etc/profile #ADD ZOOKEEPER_HOME PATH ZOOKEEPER_HOME=/yinzhengjie/softwares/zookeeper-3.4.14 PATH=$PATH:$ZOOKEEPER_HOME/bin [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# source /etc/profile [root@node101.yinzhengjie.org.cn ~]#
3>.配置zookeeper的配置文件
[root@node101.yinzhengjie.org.cn ~]# cp /yinzhengjie/softwares/zookeeper-3.4.14/conf/zoo_sample.cfg /yinzhengjie/softwares/zookeeper-3.4.14/conf/zoo.cfg [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/softwares/zookeeper-3.4.14/conf/ total 16 -rw-rw-r-- 1 2002 2002 535 Mar 6 08:50 configuration.xsl -rw-rw-r-- 1 2002 2002 2161 Mar 6 08:50 log4j.properties -rw-r--r-- 1 root root 922 Apr 12 08:55 zoo.cfg -rw-rw-r-- 1 2002 2002 922 Mar 6 08:50 zoo_sample.cfg [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/zookeeper-3.4.14/conf/zoo.cfg # 滴答，计时的基本单位，默认是2000毫秒，即2秒。它是zookeeper最小的时间单位，用于丈量心跳时间和超时时间等，通常设置成默认2秒即可。 tickTime=2000 # 初始化限制是10滴答，默认是10个滴答，即默认是20秒。指定follower节点初始化是链接leader节点的最大tick次数。 initLimit=5 # 数据同步的时间限制，默认是5个滴答，即默认时间是10秒。设定了follower节点与leader节点进行同步的最大时间。与initLimit类似，它也是以tickTime为单位进行指定的。 syncLimit=2 # 指定zookeeper的工作目录，这是一个非常重要的参数，zookeeper会在内存中在内存只能中保存系统快照，并定期写入该路径指定的文件夹中。生产环境中需要注意该文件夹的磁盘占用情况。 dataDir=/home/yinzhengjie/zookeeper # 监听zookeeper的默认端口。zookeeper监听客户端链接的端口，一般设置成默认2181即可。 clientPort=2181 # 这个操作将限制连接到 ZooKeeper 的客户端的数量，限制并发连接的数量，它通过 IP 来区分不同的客户端。此配置选项可以用来阻止某些类别的 Dos 攻击。将它设置为 0 或者忽略而不进行设置将会取消对并发连接的限制。 #maxClientCnxns=60 # 在上文中已经提到，3.4.0及之后版本，ZK提供了自动清理事务日志和快照文件的功能，这个参数指定了清理频率，单位是小时，需要配置一个1或更大的整数，默认是0，表示不开启自动清理功能。 #autopurge.purgeInterval=1 # 这个参数和上面的参数搭配使用，这个参数指定了需要保留的文件数目。默认是保留3个。 #autopurge.snapRetainCount=3 #server.x=[hostname]:nnnnn[:nnnnn]，这里的x是一个数字，与myid文件中的id是一致的。右边可以配置两个端口，第一个端口用于F和L之间的数据同步和其它通信，第二个端口用于Leader选举过程中投票通信。 server.101=node101.yinzhengjie.org.cn:2888:3888 server.102=node102.yinzhengjie.org.cn:2888:3888 server.103=node103.yinzhengjie.org.cn:2888:3888 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/zookeeper-3.4.14/conf/java.env #!/bin/bash #@author :yinzhengjie #blog:http://www.cnblogs.com/yinzhengjie #EMAIL:y1053419035@qq.com #指定JDK的安装路径 export JAVA_HOME=/yinzhengjie/softwares/jdk #指定zookeeper的heap内存大小 export JVMFLAGS="-Xms256m -Xmx256m $JVMFLAGS" [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
4>.编写zookeeper的启动脚本
[root@node101.yinzhengjie.org.cn ~]# vi /usr/local/bin/xzk.sh [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# cat /usr/local/bin/xzk.sh #!/bin/bash #@author :yinzhengjie #blog:http://www.cnblogs.com/yinzhengjie #EMAIL:y1053419035@qq.com #判断用户是否传参 if [ $# -ne 1 ];then echo "无效参数，用法为: $0 {start|stop|restart|status}" exit fi #获取用户输入的命令 cmd=$1 #定义函数功能 function zookeeperManger(){ case $cmd in start) echo "启动服务" remoteExecution start ;; stop) echo "停止服务" remoteExecution stop ;; restart) echo "重启服务" remoteExecution restart ;; status) echo "查看状态" remoteExecution status ;; *) echo "无效参数，用法为: $0 {start|stop|restart|status}" ;; esac } #定义执行的命令 function remoteExecution(){ for (( i=101 ; i<=103 ; i++ )) ; do tput setaf 2 echo ========== node${i}.yinzhengjie.org.cn zkServer.sh $1 ================ tput setaf 9 ssh node${i}.yinzhengjie.org.cn "source /etc/profile ; zkServer.sh $1" done } #调用函数 zookeeperManger [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ll /usr/local/bin/xzk.sh -rw-r--r-- 1 root root 1125 Apr 12 09:03 /usr/local/bin/xzk.sh [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# chmod +x /usr/local/bin/xzk.sh [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /usr/local/bin/xzk.sh -rwxr-xr-x 1 root root 1125 Apr 12 09:03 /usr/local/bin/xzk.sh [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
5>.创建myid文件并写入服务器编号
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'mkdir /home/yinzhengjie/zookeeper/' [WARNING]: Consider using file module with state=directory rather than running mkdir node101.yinzhengjie.org.cn | SUCCESS | rc=0 >> node103.yinzhengjie.org.cn | SUCCESS | rc=0 >> node102.yinzhengjie.org.cn | SUCCESS | rc=0 >> [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# for (( i=101;i<=103;i++ )) do ssh node${i}.yinzhengjie.org.cn "echo -n $i > /home/yinzhengjie/zookeeper/myid" ;done [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'cat /home/yinzhengjie/zookeeper/myid' node103.yinzhengjie.org.cn | SUCCESS | rc=0 >> 103 node102.yinzhengjie.org.cn | SUCCESS | rc=0 >> 102 node101.yinzhengjie.org.cn | SUCCESS | rc=0 >> 101 [root@node101.yinzhengjie.org.cn ~]#
6>.分发配置文件
[root@node101.yinzhengjie.org.cn ~]# scp /etc/profile node102.yinzhengjie.org.cn:/etc/ profile 100% 2127 1.2MB/s 00:00 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# scp /etc/profile node103.yinzhengjie.org.cn:/etc/ profile 100% 2127 1.4MB/s 00:00 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# scp -r /yinzhengjie/softwares/zookeeper-3.4.14/ node102.yinzhengjie.org.cn:/yinzhengjie/softwares/ ........ log.42 100% 184 128.3KB/s 00:00 log.63b 100% 48KB 22.7MB/s 00:00 snapshot.0 100% 296 277.6KB/s 00:00 snapshot.272 100% 55KB 26.5MB/s 00:00 snapshot.273 100% 55KB 24.3MB/s 00:00 snapshot.639 100% 140KB 27.2MB/s 00:00 snapshot.83f 100% 4824 3.5MB/s 00:00 minikdc-krb5.conf 100% 1103 1.3MB/s 00:00 minikdc.ldiff 100% 1630 2.2MB/s 00:00 log.100000001 100% 2401KB 50.2MB/s 00:00 log.100001bf0 100% 1001KB 59.6MB/s 00:00 snapshot.100000000 100% 73 17.1KB/s 00:00 snapshot.100001bec 100% 2507KB 61.6MB/s 00:00 check_compatibility.py 100% 7212 4.7MB/s 00:00 checkstyle-noframes-sorted.xsl 100% 5489 6.5MB/s 00:00 checkstyle.xml 100% 7489 7.3MB/s 00:00 findbugsExcludeFile.xml 100% 5027 6.6MB/s 00:00 log4j.properties 100% 2712 1.9MB/s 00:00 test-patch.properties 100% 855 930.7KB/s 00:00 test-github-pr.sh 100% 24KB 9.7MB/s 00:00 test-patch.sh 100% 24KB 19.4MB/s 00:00 pom.xml 100% 10KB 7.6MB/s 00:00 LICENSE.txt 100% 12KB 5.9MB/s 00:00 NOTICE.txt 100% 3132 1.2MB/s 00:00 README.md 100% 1765 1.4MB/s 00:00 README_packaging.txt 100% 1770 2.5MB/s 00:00 build.xml 100% 95KB 39.5MB/s 00:00 ivy.xml 100% 10KB 8.2MB/s 00:00 ivysettings.xml 100% 1709 1.3MB/s 00:00 pom.xml 100% 31KB 8.9MB/s 00:00 zookeeper-3.4.14.jar 100% 1480KB 41.5MB/s 00:00 zookeeper-3.4.14.jar.md5 100% 33 25.6KB/s 00:00 zookeeper-3.4.14.jar.sha1 100% 41 60.7KB/s 00:00 zookeeper-3.4.14.jar.asc 100% 836 1.2MB/s 00:00 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# scp -r /yinzhengjie/softwares/zookeeper-3.4.14/ node103.yinzhengjie.org.cn:/yinzhengjie/softwares/ ...... snapshot.273 100% 55KB 14.8MB/s 00:00 snapshot.639 100% 140KB 21.7MB/s 00:00 snapshot.83f 100% 4824 4.4MB/s 00:00 minikdc-krb5.conf 100% 1103 1.0MB/s 00:00 minikdc.ldiff 100% 1630 1.4MB/s 00:00 log.100000001 100% 2401KB 53.2MB/s 00:00 log.100001bf0 100% 1001KB 46.5MB/s 00:00 snapshot.100000000 100% 73 40.2KB/s 00:00 snapshot.100001bec 100% 2507KB 57.4MB/s 00:00 check_compatibility.py 100% 7212 1.4MB/s 00:00 checkstyle-noframes-sorted.xsl 100% 5489 4.4MB/s 00:00 checkstyle.xml 100% 7489 5.0MB/s 00:00 findbugsExcludeFile.xml 100% 5027 4.2MB/s 00:00 log4j.properties 100% 2712 2.5MB/s 00:00 test-patch.properties 100% 855 376.7KB/s 00:00 test-github-pr.sh 100% 24KB 6.3MB/s 00:00 test-patch.sh 100% 24KB 13.5MB/s 00:00 pom.xml 100% 10KB 6.5MB/s 00:00 LICENSE.txt 100% 12KB 4.9MB/s 00:00 NOTICE.txt 100% 3132 2.8MB/s 00:00 README.md 100% 1765 1.9MB/s 00:00 README_packaging.txt 100% 1770 1.6MB/s 00:00 build.xml 100% 95KB 27.3MB/s 00:00 ivy.xml 100% 10KB 10.1MB/s 00:00 ivysettings.xml 100% 1709 1.3MB/s 00:00 pom.xml 100% 31KB 13.6MB/s 00:00 zookeeper-3.4.14.jar 100% 1480KB 42.5MB/s 00:00 zookeeper-3.4.14.jar.md5 100% 33 38.8KB/s 00:00 zookeeper-3.4.14.jar.sha1 100% 41 29.3KB/s 00:00 zookeeper-3.4.14.jar.asc 100% 836 690.2KB/s 00:00 [root@node101.yinzhengjie.org.cn ~]#
7>.启动zookeeper集群
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'jps' node101.yinzhengjie.org.cn | SUCCESS | rc=0 >> 10062 Jps node103.yinzhengjie.org.cn | SUCCESS | rc=0 >> 9720 Jps node102.yinzhengjie.org.cn | SUCCESS | rc=0 >> 9759 Jps [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# xzk.sh start 启动服务 ========== node101.yinzhengjie.org.cn zkServer.sh start ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED ========== node102.yinzhengjie.org.cn zkServer.sh start ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED ========== node103.yinzhengjie.org.cn zkServer.sh start ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Starting zookeeper ... STARTED [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# xzk.sh status 查看状态 ========== node101.yinzhengjie.org.cn zkServer.sh status ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Mode: follower ========== node102.yinzhengjie.org.cn zkServer.sh status ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Mode: leader ========== node103.yinzhengjie.org.cn zkServer.sh status ================ ZooKeeper JMX enabled by default Using config: /yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf/zoo.cfg Mode: follower [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'jps' node103.yinzhengjie.org.cn | SUCCESS | rc=0 >> 10177 QuorumPeerMain 10279 Jps node102.yinzhengjie.org.cn | SUCCESS | rc=0 >> 10209 QuorumPeerMain 10317 Jps node101.yinzhengjie.org.cn | SUCCESS | rc=0 >> 10765 QuorumPeerMain 10925 Jps [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh Connecting to localhost:2181 2019-04-12 09:23:04,270 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT 2019-04-12 09:23:04,272 [myid:] - INFO [main:Environment@100] - Client environment:host.name=node101.yinzhengjie.org.cn 2019-04-12 09:23:04,273 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_201 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/yinzhengjie/softwares/jdk1.8.0_201/jre 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf: 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2019-04-12 09:23:04,275 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.el7.x86_64 2019-04-12 09:23:04,276 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2019-04-12 09:23:04,276 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2019-04-12 09:23:04,276 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/root 2019-04-12 09:23:04,277 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5ce65a89 Welcome to ZooKeeper! 2019-04-12 09:23:04,303 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2019-04-12 09:23:04,365 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session 2019-04-12 09:23:04,395 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x6500007f14d00000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: localhost:2181(CONNECTED) 0] ls / [zookeeper] [zk: localhost:2181(CONNECTED) 1] ls /zookeeper [quota] [zk: localhost:2181(CONNECTED) 2] ls /zookeeper/quota [] [zk: localhost:2181(CONNECTED) 3] quit Quitting... 2019-04-12 09:23:13,101 [myid:] - INFO [main:ZooKeeper@693] - Session: 0x6500007f14d00000 closed 2019-04-12 09:23:13,102 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x6500007f14d00000 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
三.部署hadoop 3.1.2 HA案例实操（切记，3个namenode之间应该相互配置免密登录哟！）
1>.备份原有的hadoop完全分布式的配置目录
[root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/softwares/hadoop-3.1.2/etc/ total 4 drwxr-xr-x 3 1001 1002 4096 Apr 11 09:27 hadoop [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# cp -r /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/ /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop-full [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# ll /yinzhengjie/softwares/hadoop-3.1.2/etc/ total 8 drwxr-xr-x 3 1001 1002 4096 Apr 11 09:27 hadoop drwxr-xr-x 3 root root 4096 Apr 12 07:15 hadoop-full [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
2>.编辑hadoop的环境变量配置文件，主要修改各个服务的所属用户
[root@node101.yinzhengjie.org.cn ~]# vi /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/hadoop-env.sh [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# tail -6 /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/hadoop-env.sh #Add Hadoop Path by yinzhengjie export JAVA_HOME=/yinzhengjie/softwares/jdk export HDFS_NAMENODE_USER=root export HDFS_DATANODE_USER=root export HDFS_ZKFC_USER=root export HDFS_JOURNALNODE_USER=root [root@node101.yinzhengjie.org.cn ~]#
3>.编辑core-site.xml配置文件
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/core-site.xml <?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://yinzhengjie-hdfs</value> </property> <property> <name>hadoop.tmp.dir</name> <value>/data/hadoop/ha</value> </property> <property> <name>hadoop.http.staticuser.user</name> <value>root</value> </property> <property> <name>ha.zookeeper.quorum</name> <value>node101.yinzhengjie.org.cn:2181,node102.yinzhengjie.org.cn:2181,node103.yinzhengjie.org.cn:2181</value> </property> </configuration> <!-- core-site.xml配置文件的作用： 用于定义系统级别的参数，如HDFS URL、Hadoop的临时目录以及用于rack-aware集群中的配置文件的配置等，此中的参数定义会覆盖core-default.xml文件中的默认配置。 fs.defaultFS 参数的作用： #声明namenode的地址，相当于声明hdfs文件系统。我们可以指定某个ip地址，在ha模式中我们通常指定hdfs集群的逻辑名称 hadoop.tmp.dir 参数的作用： #声明hadoop工作目录的地址。 hadoop.http.staticuser.user 参数的作用： #在网页界面访问数据使用的用户名。默认值是一个不真实存在的用户，此用户权限很小，不能访问不同用户的数据。这保证了数据安全。也可以设置为hdfs和hadoop等具有较高权限的用户，但会导致能够登陆网页界面的人能看到其它用户数据。实际设置请综合考虑。如无特殊需求。使用默认值就好. ha.zookeeper.quorum 参数的作用： #指定zookeeper集群的地址，让zookeeper帮我们管理hdfs集群。千万别手动在后面加zookeeper的根目录哟，否则在使用hdfs进行zookeeper格式化会报错！ --> [root@node101.yinzhengjie.org.cn ~]#
4>.编辑hdfs-site.xml配置文件
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/hdfs-site.xml <?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>dfs.replication</name> <value>2</value> </property> <property> <name>dfs.nameservices</name> <value>yinzhengjie-hdfs</value> </property> <property> <name>dfs.ha.namenodes.yinzhengjie-hdfs</name> <value>namenode1,namenode2,namenode3</value> </property> <property> <name>dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode1</name> <value>node101.yinzhengjie.org.cn:8020</value> </property> <property> <name>dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode2</name> <value>node102.yinzhengjie.org.cn:8020</value> </property> <property> <name>dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode3</name> <value>node103.yinzhengjie.org.cn:8020</value> </property> <property> <name>dfs.namenode.http-address.yinzhengjie-hdfs.namenode1</name> <value>node101.yinzhengjie.org.cn:9870</value> </property> <property> <name>dfs.namenode.http-address.yinzhengjie-hdfs.namenode2</name> <value>node102.yinzhengjie.org.cn:9870</value> </property> <property> <name>dfs.namenode.http-address.yinzhengjie-hdfs.namenode3</name> <value>node103.yinzhengjie.org.cn:9870</value> </property> <property> <name>dfs.namenode.shared.edits.dir</name> <value>qjournal://node101.yinzhengjie.org.cn:8485;node102.yinzhengjie.org.cn:8485;node103.yinzhengjie.org.cn:8485/yinzhengjie-hdfs</value> </property> <property> <name>dfs.client.failover.proxy.provider.yinzhengjie-hdfs</name> <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value> </property> <property> <name>dfs.ha.fencing.methods</name> <value>sshfence</value> </property> <property> <name>dfs.ha.fencing.ssh.private-key-files</name> <value>/root/.ssh/id_rsa</value> </property> <property> <name>dfs.journalnode.edits.dir</name> <value>/data/hadoop/ha/journalnode</value> </property> <property> <name>dfs.ha.automatic-failover.enabled</name> <value>true</value> </property> </configuration> <!-- hdfs-site.xml 配置文件的作用： #HDFS的相关设定，如文件副本的个数、块大小及是否使用强制权限等，此中的参数定义会覆盖hdfs-default.xml文件中的默认配置. dfs.replication 参数的作用： #为了数据可用性及冗余的目的，HDFS会在多个节点上保存同一个数据块的多个副本，其默认为3个。而只有一个节点的伪分布式环境中其仅用 保存一个副本即可，这可以通过dfs.replication属性进行定义。它是一个软件级备份。 dfs.nameservices 参数的作用： #指定一个逻辑名称，该名称会自动去帮我们找打真实NameNode节点，这和咱们的keepalive的VIP有点类似！这个名称咱们可以自定义！但是我们正在引用时千万别引用错了！ dfs.ha.namenodes.yinzhengjie-hdfs 参数的作用： #看到没，上面我们定义了dfs.nameservices的名称，现在我们就用到了，我们把定义的逻辑名称指向了各个namenode的别名 dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode1 参数的作用： #定义远程主机调用的服务器地址。 dfs.namenode.http-address.yinzhengjie-hdfs.namenode1 参数的作用： #上面我们定义了集群的逻辑名称，也定义节点的逻辑名称，但是真正的ip地址我们始终没有指定，因此我们这一步骤就是指定dfs.namenode.rpc-address.yinzhengjie-hdfs.namenode1对应的IP地址。当然你写主机名称也是可以的，只不过嘛你得在/etc/hosts文件做映射或者配置DNS中有对应的A记录哟！ dfs.namenode.shared.edits.dir 参数的作用： #配置JournalNode集群的地址。 dfs.client.failover.proxy.provider.yinzhengjie-hdfs 参数的作用： #配置故障转移的代理类，这是HDFS客户端找到active NameNode类名的一个代理类，如果你不配置这个项的话，那么客户端连接active namenode地址时会报错！实际上故障转移就是通过这个代理类来实现的，我们只需要默认官网的配置即可！不需要手动修改！ dfs.ha.fencing.methods 参数的作用： #指定ha出现故障时的隔离方法！ dfs.ha.fencing.ssh.private-key-files 参数的作用： #指定隔离主机的私钥PATH。 dfs.journalnode.edits.dir 参数的作用： #指定JN节点存放编辑日志的目录。 dfs.ha.automatic-failover.enabled 参数的作用： #开启自动故障转移功能. -> [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
5>.查看works配置文件
[root@node101.yinzhengjie.org.cn ~]# cat /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop/workers node101.yinzhengjie.org.cn node102.yinzhengjie.org.cn node103.yinzhengjie.org.cn [root@node101.yinzhengjie.org.cn ~]#
5>.将修改的配置文件分发到其他节点中
[root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# cd /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop [root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop]# [root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop]# scp core-site.xml hdfs-site.xml hadoop-env.sh node102.yinzhengjie.org.cn:`pwd` core-site.xml 100% 1804 1.0MB/s 00:00 hdfs-site.xml 100% 4635 5.4MB/s 00:00 hadoop-env.sh 100% 16KB 12.5MB/s 00:00 [root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop]#
[root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop]# scp core-site.xml hdfs-site.xml hadoop-env.sh node103.yinzhengjie.org.cn:`pwd` core-site.xml 100% 1804 227.6KB/s 00:00 hdfs-site.xml 100% 4635 2.2MB/s 00:00 hadoop-env.sh 100% 16KB 13.3MB/s 00:00 [root@node101.yinzhengjie.org.cn /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop]#
6>.启动journalnode服务
[root@node101.yinzhengjie.org.cn ~]# ansible all -m shell -a 'jps' node101.yinzhengjie.org.cn | SUCCESS | rc=0 >> 11082 Jps 10765 QuorumPeerMain node103.yinzhengjie.org.cn | SUCCESS | rc=0 >> 10336 Jps 10177 QuorumPeerMain node102.yinzhengjie.org.cn | SUCCESS | rc=0 >> 10209 QuorumPeerMain 10373 Jps [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ll /data/hadoop/ total 0 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# hdfs --daemon start journalnode [root@node101.yinzhengjie.org.cn ~]# jps 15733 JournalNode 10765 QuorumPeerMain 15775 Jps [root@node101.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# hdfs --daemon start journalnode [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]# jps 10209 QuorumPeerMain 13607 Jps 13566 JournalNode [root@node102.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# hdfs --daemon start journalnode [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]# jps 10177 QuorumPeerMain 13605 Jps 13564 JournalNode [root@node103.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# hdfs --daemon start journalnode ERROR: Cannot set priority of journalnode process 15496 [root@node101.yinzhengjie.org.cn ~]# 温馨提示： 如果遇到上述问题的话，请检查你的配置文件是否有误！多一个空格都不行，如果你用我的博客的配置文件的话，建议不要把注释拷贝进去，只拷贝有效的XML代码即可！
[root@node101.yinzhengjie.org.cn ~]# ll -R /data/ /data/: total 0 drwxr-xr-x 3 root root 16 Apr 15 07:34 hadoop /data/hadoop: total 0 drwxr-xr-x 3 root root 25 Apr 15 07:34 ha /data/hadoop/ha: total 0 drwxr-xr-x 2 root root 6 Apr 15 07:34 journalnode /data/hadoop/ha/journalnode: total 0 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# ll -R /data/ /data/: total 0 drwxr-xr-x 3 root root 16 Apr 15 07:45 hadoop /data/hadoop: total 0 drwxr-xr-x 3 root root 25 Apr 15 07:45 ha /data/hadoop/ha: total 0 drwxr-xr-x 2 root root 6 Apr 15 07:45 journalnode /data/hadoop/ha/journalnode: total 0 [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# ll -R /data/ /data/: total 0 drwxr-xr-x 3 root root 16 Apr 15 07:45 hadoop /data/hadoop: total 0 drwxr-xr-x 3 root root 25 Apr 15 07:45 ha /data/hadoop/ha: total 0 drwxr-xr-x 2 root root 6 Apr 15 07:45 journalnode /data/hadoop/ha/journalnode: total 0 [root@node103.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/journalnode/yinzhengjie-hdfs/current/VERSION #Mon Apr 15 07:53:26 PDT 2019 namespaceID=1130551675 clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6 cTime=1555340006352 storageType=JOURNAL_NODE layoutVersion=-64 [root@node101.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/journalnode/yinzhengjie-hdfs/current/VERSION #Mon Apr 15 07:53:26 PDT 2019 namespaceID=1130551675 clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6 cTime=1555340006352 storageType=JOURNAL_NODE layoutVersion=-64 [root@node102.yinzhengjie.org.cn ~]# [root@node102.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/journalnode/yinzhengjie-hdfs/current/VERSION #Mon Apr 15 07:53:26 PDT 2019 namespaceID=1130551675 clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6 cTime=1555340006352 storageType=JOURNAL_NODE layoutVersion=-64 [root@node103.yinzhengjie.org.cn ~]#
7>.格式化HDFS文件系统（目的是生成初始的元数据信息，生成的信息会存放在我们hdfs-core-site.xml配置文件中指定的工作目录下，注意生成的信息和上面的JN的VERSION之间的关系！）
[root@node101.yinzhengjie.org.cn ~]# hdfs namenode -format 2019-04-15 07:53:21,960 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = node101.yinzhengjie.org.cn/172.30.1.101 STARTUP_MSG: args = [-format] STARTUP_MSG: version = 3.1.2 STARTUP_MSG: classpath = /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-kms-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-ajax-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-all-4.0.52.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okio-1.6.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/dnsjava-2.1.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/fst-2.50.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-base-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/java-util-1.9.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-client-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/json-io-2.5.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/objenesis-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-registry-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-router-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-tests-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-core-3.1.2.jar STARTUP_MSG: build = https://github.com/apache/hadoop.git -r 1019dde65bcf12e05ef48ac71e84550d589e5d9a; compiled by 'sunilg' on 2019-01-29T01:39Z STARTUP_MSG: java = 1.8.0_201 ************************************************************/ 2019-04-15 07:53:21,972 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT] 2019-04-15 07:53:22,069 INFO namenode.NameNode: createNameNode [-format] Formatting using clusterid: CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6 2019-04-15 07:53:22,666 INFO namenode.FSEditLog: Edit logging is async:true 2019-04-15 07:53:22,682 INFO namenode.FSNamesystem: KeyProvider: null 2019-04-15 07:53:22,683 INFO namenode.FSNamesystem: fsLock is fair: true 2019-04-15 07:53:22,684 INFO namenode.FSNamesystem: Detailed lock hold time metrics enabled: false 2019-04-15 07:53:22,692 INFO namenode.FSNamesystem: fsOwner = root (auth:SIMPLE) 2019-04-15 07:53:22,692 INFO namenode.FSNamesystem: supergroup = supergroup 2019-04-15 07:53:22,692 INFO namenode.FSNamesystem: isPermissionEnabled = true 2019-04-15 07:53:22,693 INFO namenode.FSNamesystem: Determined nameservice ID: yinzhengjie-hdfs 2019-04-15 07:53:22,693 INFO namenode.FSNamesystem: HA Enabled: true 2019-04-15 07:53:22,739 INFO common.Util: dfs.datanode.fileio.profiling.sampling.percentage set to 0. Disabling file IO profiling 2019-04-15 07:53:22,755 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit: configured=1000, counted=60, effected=1000 2019-04-15 07:53:22,756 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true 2019-04-15 07:53:22,763 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000 2019-04-15 07:53:22,763 INFO blockmanagement.BlockManager: The block deletion will start around 2019 Apr 15 07:53:22 2019-04-15 07:53:22,765 INFO util.GSet: Computing capacity for map BlocksMap 2019-04-15 07:53:22,765 INFO util.GSet: VM type = 64-bit 2019-04-15 07:53:22,766 INFO util.GSet: 2.0% max memory 839.5 MB = 16.8 MB 2019-04-15 07:53:22,766 INFO util.GSet: capacity = 2^21 = 2097152 entries 2019-04-15 07:53:22,784 INFO blockmanagement.BlockManager: dfs.block.access.token.enable = false 2019-04-15 07:53:22,791 INFO Configuration.deprecation: No unit for dfs.namenode.safemode.extension(30000) assuming MILLISECONDS 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.threshold-pct = 0.9990000128746033 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.min.datanodes = 0 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManagerSafeMode: dfs.namenode.safemode.extension = 30000 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManager: defaultReplication = 2 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManager: maxReplication = 512 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManager: minReplication = 1 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManager: maxReplicationStreams = 2 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManager: redundancyRecheckInterval = 3000ms 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManager: encryptDataTransfer = false 2019-04-15 07:53:22,791 INFO blockmanagement.BlockManager: maxNumBlocksToLog = 1000 2019-04-15 07:53:22,841 INFO namenode.FSDirectory: GLOBAL serial map: bits=24 maxEntries=16777215 2019-04-15 07:53:22,856 INFO util.GSet: Computing capacity for map INodeMap 2019-04-15 07:53:22,856 INFO util.GSet: VM type = 64-bit 2019-04-15 07:53:22,856 INFO util.GSet: 1.0% max memory 839.5 MB = 8.4 MB 2019-04-15 07:53:22,856 INFO util.GSet: capacity = 2^20 = 1048576 entries 2019-04-15 07:53:22,857 INFO namenode.FSDirectory: ACLs enabled? false 2019-04-15 07:53:22,857 INFO namenode.FSDirectory: POSIX ACL inheritance enabled? true 2019-04-15 07:53:22,857 INFO namenode.FSDirectory: XAttrs enabled? true 2019-04-15 07:53:22,857 INFO namenode.NameNode: Caching file names occurring more than 10 times 2019-04-15 07:53:22,862 INFO snapshot.SnapshotManager: Loaded config captureOpenFiles: false, skipCaptureAccessTimeOnlyChange: false, snapshotDiffAllowSnapRootDescendant: true, maxSnapshotLimit: 65536 2019-04-15 07:53:22,867 INFO snapshot.SnapshotManager: SkipList is disabled 2019-04-15 07:53:22,873 INFO util.GSet: Computing capacity for map cachedBlocks 2019-04-15 07:53:22,873 INFO util.GSet: VM type = 64-bit 2019-04-15 07:53:22,873 INFO util.GSet: 0.25% max memory 839.5 MB = 2.1 MB 2019-04-15 07:53:22,873 INFO util.GSet: capacity = 2^18 = 262144 entries 2019-04-15 07:53:22,881 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10 2019-04-15 07:53:22,881 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10 2019-04-15 07:53:22,881 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25 2019-04-15 07:53:22,885 INFO namenode.FSNamesystem: Retry cache on namenode is enabled 2019-04-15 07:53:22,886 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis 2019-04-15 07:53:22,888 INFO util.GSet: Computing capacity for map NameNodeRetryCache 2019-04-15 07:53:22,889 INFO util.GSet: VM type = 64-bit 2019-04-15 07:53:22,889 INFO util.GSet: 0.029999999329447746% max memory 839.5 MB = 257.9 KB 2019-04-15 07:53:22,889 INFO util.GSet: capacity = 2^15 = 32768 entries 2019-04-15 07:53:26,352 INFO namenode.FSImage: Allocated new BlockPoolId: BP-692446373-172.30.1.101-1555340006352 2019-04-15 07:53:26,365 INFO common.Storage: Storage directory /data/hadoop/ha/dfs/name has been successfully formatted. 2019-04-15 07:53:26,657 INFO namenode.FSImageFormatProtobuf: Saving image file /data/hadoop/ha/dfs/name/current/fsimage.ckpt_0000000000000000000 using no compression 2019-04-15 07:53:26,779 INFO namenode.FSImageFormatProtobuf: Image file /data/hadoop/ha/dfs/name/current/fsimage.ckpt_0000000000000000000 of size 391 bytes saved in 0 seconds . 2019-04-15 07:53:26,792 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0 2019-04-15 07:53:26,976 INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at node101.yinzhengjie.org.cn/172.30.1.101 ************************************************************/ [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# ll -R /data/hadoop/ha/dfs /data/hadoop/ha/dfs: total 0 drwxr-xr-x 3 root root 21 Apr 15 07:53 name /data/hadoop/ha/dfs/name: total 0 drwxr-xr-x 2 root root 112 Apr 15 07:53 current /data/hadoop/ha/dfs/name/current: total 16 -rw-r--r-- 1 root root 391 Apr 15 07:53 fsimage_0000000000000000000 -rw-r--r-- 1 root root 62 Apr 15 07:53 fsimage_0000000000000000000.md5 -rw-r--r-- 1 root root 2 Apr 15 07:53 seen_txid -rw-r--r-- 1 root root 216 Apr 15 07:53 VERSION [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/dfs/name/current/VERSION #Mon Apr 15 07:53:26 PDT 2019 namespaceID=1130551675 clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6 cTime=1555340006352 storageType=NAME_NODE blockpoolID=BP-692446373-172.30.1.101-1555340006352 layoutVersion=-64 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/dfs/name/current/VERSION #Mon Apr 15 07:53:26 PDT 2019 namespaceID=1130551675 clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6 cTime=1555340006352 storageType=NAME_NODE blockpoolID=BP-692446373-172.30.1.101-1555340006352 layoutVersion=-64 [root@node101.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# ll /data/hadoop/ha/ total 0 drwxr-xr-x 3 root root 30 Apr 15 07:53 journalnode [root@node102.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# ll /data/hadoop/ha/ total 0 drwxr-xr-x 3 root root 30 Apr 15 07:53 journalnode [root@node103.yinzhengjie.org.cn ~]#
8>.启动NameNode并同步数据
[root@node101.yinzhengjie.org.cn ~]# jps Jps JournalNode QuorumPeerMain [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# hdfs --daemon start namenode [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# jps JournalNode NameNode Jps QuorumPeerMain [root@node101.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# hdfs namenode -bootstrapStandby 2019-04-15 08:08:00,871 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = node103.yinzhengjie.org.cn/172.30.1.103 STARTUP_MSG: args = [-bootstrapStandby] STARTUP_MSG: version = 3.1.2 STARTUP_MSG: classpath = /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-kms-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-ajax-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-all-4.0.52.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okio-1.6.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/dnsjava-2.1.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/fst-2.50.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-base-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/java-util-1.9.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-client-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/json-io-2.5.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/objenesis-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-registry-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-router-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-tests-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-core-3.1.2.jar STARTUP_MSG: build = https://github.com/apache/hadoop.git -r 1019dde65bcf12e05ef48ac71e84550d589e5d9a; compiled by 'sunilg' on 2019-01-29T01:39Z STARTUP_MSG: java = 1.8.0_201 ************************************************************/ 2019-04-15 08:08:00,890 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT] 2019-04-15 08:08:01,007 INFO namenode.NameNode: createNameNode [-bootstrapStandby] 2019-04-15 08:08:01,297 INFO ha.BootstrapStandby: Found nn: namenode1, ipc: node101.yinzhengjie.org.cn/172.30.1.101:8020 2019-04-15 08:08:01,297 INFO ha.BootstrapStandby: Found nn: namenode2, ipc: node102.yinzhengjie.org.cn/172.30.1.102:8020 ===================================================== About to bootstrap Standby ID namenode3 from: Nameservice ID: yinzhengjie-hdfs Other Namenode ID: namenode1 Other NN's HTTP address: http://node101.yinzhengjie.org.cn:9870 Other NN's IPC address: node101.yinzhengjie.org.cn/172.30.1.101:8020 Namespace ID: 1130551675 Block pool ID: BP-692446373-172.30.1.101-1555340006352 Cluster ID: CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6 Layout version: -64 isUpgradeFinalized: true ===================================================== 2019-04-15 08:08:02,595 INFO common.Storage: Storage directory /data/hadoop/ha/dfs/name has been successfully formatted. 2019-04-15 08:08:02,643 INFO namenode.FSEditLog: Edit logging is async:true 2019-04-15 08:08:02,756 INFO namenode.TransferFsImage: Opening connection to http://node101.yinzhengjie.org.cn:9870/imagetransfer?getimage=1&txid=0&storageInfo=-64:1130551675:1555340006352:CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6&bootstrapstandby=true 2019-04-15 08:08:03,014 INFO common.Util: Combined time for file download and fsync to all disks took 0.01s. The file download took 0.01s at 0.00 KB/s. Synchronous (fsync) write to disk of /data/hadoop/ha/dfs/name/current/fsimage.ckpt_0000000000000000000 took 0.00s. 2019-04-15 08:08:03,014 INFO namenode.TransferFsImage: Downloaded file fsimage.ckpt_0000000000000000000 size 391 bytes. 2019-04-15 08:08:03,076 INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at node103.yinzhengjie.org.cn/172.30.1.103 ************************************************************/ [root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# ll -R /data/hadoop/ha/dfs /data/hadoop/ha/dfs: total 0 drwxr-xr-x 3 root root 21 Apr 15 08:08 name /data/hadoop/ha/dfs/name: total 0 drwxr-xr-x 2 root root 112 Apr 15 08:08 current /data/hadoop/ha/dfs/name/current: total 16 -rw-r--r-- 1 root root 391 Apr 15 08:08 fsimage_0000000000000000000 -rw-r--r-- 1 root root 62 Apr 15 08:08 fsimage_0000000000000000000.md5 -rw-r--r-- 1 root root 2 Apr 15 08:08 seen_txid -rw-r--r-- 1 root root 216 Apr 15 08:08 VERSION [root@node103.yinzhengjie.org.cn ~]# [root@node103.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# cat /data/hadoop/ha/dfs/name/current/VERSION #Mon Apr 15 08:08:02 PDT 2019 namespaceID=1130551675 clusterID=CID-35f473ca-7c4e-4a0f-81a2-ee27c14989d6 cTime=1555340006352 storageType=NAME_NODE blockpoolID=BP-692446373-172.30.1.101-1555340006352 layoutVersion=-64 [root@node103.yinzhengjie.org.cn ~]#
四.zookeeper格式化、启动ha及测试
1>.初始化hdfs在zookeeper注册节点目录
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh Connecting to localhost:2181 2019-04-15 08:15:10,366 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT 2019-04-15 08:15:10,369 [myid:] - INFO [main:Environment@100] - Client environment:host.name=node101.yinzhengjie.org.cn 2019-04-15 08:15:10,369 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_201 2019-04-15 08:15:10,371 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2019-04-15 08:15:10,371 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/yinzhengjie/softwares/jdk1.8.0_201/jre 2019-04-15 08:15:10,371 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf: 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.el7.x86_64 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2019-04-15 08:15:10,372 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/root 2019-04-15 08:15:10,374 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5ce65a89 Welcome to ZooKeeper! 2019-04-15 08:15:10,406 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2019-04-15 08:15:10,476 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session 2019-04-15 08:15:10,497 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x65000f71aa200000, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: localhost:2181(CONNECTED) 0] ls / [zookeeper] [zk: localhost:2181(CONNECTED) 1] [zk: localhost:2181(CONNECTED) 1] quit Quitting... 2019-04-15 08:24:18,604 [myid:] - INFO [main:ZooKeeper@693] - Session: 0x65000f71aa200001 closed 2019-04-15 08:24:18,606 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x65000f71aa200001 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# hdfs zkfc -formatZK 2019-04-15 08:22:35,041 INFO tools.DFSZKFailoverController: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting DFSZKFailoverController STARTUP_MSG: host = node101.yinzhengjie.org.cn/172.30.1.101 STARTUP_MSG: args = [-formatZK] STARTUP_MSG: version = 3.1.2 STARTUP_MSG: classpath = /yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-kms-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-ajax-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-all-4.0.52.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okio-1.6.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/dnsjava-2.1.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/fst-2.50.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-base-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/java-util-1.9.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-client-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/json-io-2.5.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/objenesis-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-registry-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-router-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-tests-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-core-3.1.2.jar STARTUP_MSG: build = https://github.com/apache/hadoop.git -r 1019dde65bcf12e05ef48ac71e84550d589e5d9a; compiled by 'sunilg' on 2019-01-29T01:39Z STARTUP_MSG: java = 1.8.0_201 ************************************************************/ 2019-04-15 08:22:35,061 INFO tools.DFSZKFailoverController: registered UNIX signal handlers for [TERM, HUP, INT] 2019-04-15 08:22:35,387 INFO tools.DFSZKFailoverController: Failover controller configured for NameNode NameNode at node101.yinzhengjie.org.cn/172.30.1.101:8020 2019-04-15 08:22:35,535 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on 06/29/2018 00:39 GMT 2019-04-15 08:22:35,535 INFO zookeeper.ZooKeeper: Client environment:host.name=node101.yinzhengjie.org.cn 2019-04-15 08:22:35,535 INFO zookeeper.ZooKeeper: Client environment:java.version=1.8.0_201 2019-04-15 08:22:35,535 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation 2019-04-15 08:22:35,535 INFO zookeeper.ZooKeeper: Client environment:java.home=/yinzhengjie/softwares/jdk1.8.0_201/jre 2019-04-15 08:22:35,537 INFO zookeeper.ZooKeeper: Client environment:java.class.path=/yinzhengjie/softwares/hadoop-3.1.2/etc/hadoop:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsp-api-2.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/jul-to-slf4j-1.7.25.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/common/hadoop-kms-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-ajax-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-all-4.0.52.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okhttp-2.7.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/okio-1.6.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-auth-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-codec-1.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpclient-4.5.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/httpcore-4.4.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/nimbus-jose-jwt-4.41.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-smart-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/accessors-smart-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/asm-5.0.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/zookeeper-3.4.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/netty-3.10.5.Final.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-framework-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-client-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/guava-11.0.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-simplekdc-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-client-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-config-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-core-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-pkix-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-asn1-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-common-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-crypto-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-io-2.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-util-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/token-provider-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-admin-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-server-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerb-identity-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/kerby-xdr-1.0.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-core-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-server-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-server-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-http-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-util-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-io-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-webapp-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-xml-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-servlet-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jetty-security-9.3.24.v20180605.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/hadoop-annotations-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-math3-3.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-net-3.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-servlet-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jersey-json-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jettison-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-jaxrs-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-xc-1.9.13.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-beanutils-1.9.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-configuration2-2.1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-lang3-3.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/avro-1.7.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/paranamer-2.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/snappy-java-1.0.5.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/commons-compress-1.18.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/re2j-1.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/gson-2.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jsch-0.1.54.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/curator-recipes-2.13.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/htrace-core4-4.1.0-incubating.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-databind-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/jackson-core-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/stax2-api-3.1.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/lib/woodstox-core-5.0.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-nfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-native-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-rbf-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/lib/junit-4.11.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2-tests.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/HikariCP-java7-2.4.12.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/aopalliance-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/dnsjava-2.1.7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/fst-2.50.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/guice-servlet-4.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-base-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.7.8.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/java-util-1.9.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/javax.inject-1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-client-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/jersey-guice-1.19.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/json-io-2.5.1.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/metrics-core-3.2.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/objenesis-1.0.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/snakeyaml-1.16.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-client-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-registry-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-common-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-router-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-tests-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-api-3.1.2.jar:/yinzhengjie/softwares/hadoop-3.1.2/share/hadoop/yarn/hadoop-yarn-services-core-3.1.2.jar 2019-04-15 08:22:35,543 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/yinzhengjie/softwares/hadoop-3.1.2/lib/native 2019-04-15 08:22:35,544 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 2019-04-15 08:22:35,544 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA> 2019-04-15 08:22:35,544 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux 2019-04-15 08:22:35,544 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64 2019-04-15 08:22:35,544 INFO zookeeper.ZooKeeper: Client environment:os.version=3.10.0-957.el7.x86_64 2019-04-15 08:22:35,544 INFO zookeeper.ZooKeeper: Client environment:user.name=root 2019-04-15 08:22:35,544 INFO zookeeper.ZooKeeper: Client environment:user.home=/root 2019-04-15 08:22:35,544 INFO zookeeper.ZooKeeper: Client environment:user.dir=/root 2019-04-15 08:22:35,546 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=node101.yinzhengjie.org.cn:2181,node102.yinzhengjie.org.cn:2181,node103.yinzhengjie.org.cn:2181 sessionTimeout=10000 watcher=org.apache.hadoop.ha.ActiveStandbyElector$WatcherWithClientRef@3c407114 2019-04-15 08:22:35,585 INFO zookeeper.ClientCnxn: Opening socket connection to server node103.yinzhengjie.org.cn/172.30.1.103:2181. Will not attempt to authenticate using SASL (unknown error) 2019-04-15 08:22:35,590 INFO zookeeper.ClientCnxn: Socket connection established to node103.yinzhengjie.org.cn/172.30.1.103:2181, initiating session 2019-04-15 08:22:35,601 INFO zookeeper.ClientCnxn: Session establishment complete on server node103.yinzhengjie.org.cn/172.30.1.103:2181, sessionid = 0x67000f716ed80001, negotiated timeout = 10000 2019-04-15 08:22:35,613 INFO ha.ActiveStandbyElector: Session connected. 2019-04-15 08:22:35,633 INFO ha.ActiveStandbyElector: Successfully created /hadoop-ha/yinzhengjie-hdfs in ZK. 2019-04-15 08:22:35,638 INFO zookeeper.ZooKeeper: Session: 0x67000f716ed80001 closed 2019-04-15 08:22:35,696 INFO zookeeper.ClientCnxn: EventThread shut down for session: 0x67000f716ed80001 2019-04-15 08:22:35,698 INFO tools.DFSZKFailoverController: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down DFSZKFailoverController at node101.yinzhengjie.org.cn/172.30.1.101 ************************************************************/ [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh Connecting to localhost:2181 2019-04-15 08:24:07,198 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT 2019-04-15 08:24:07,201 [myid:] - INFO [main:Environment@100] - Client environment:host.name=node101.yinzhengjie.org.cn 2019-04-15 08:24:07,201 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_201 2019-04-15 08:24:07,204 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2019-04-15 08:24:07,204 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/yinzhengjie/softwares/jdk1.8.0_201/jre 2019-04-15 08:24:07,204 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf: 2019-04-15 08:24:07,204 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.el7.x86_64 2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2019-04-15 08:24:07,205 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2019-04-15 08:24:07,206 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/root 2019-04-15 08:24:07,207 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5ce65a89 Welcome to ZooKeeper! 2019-04-15 08:24:07,230 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2019-04-15 08:24:07,303 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session 2019-04-15 08:24:07,322 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x65000f71aa200001, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: localhost:2181(CONNECTED) 0] ls / [zookeeper, hadoop-ha] [zk: localhost:2181(CONNECTED) 1] ls /hadoop-ha [yinzhengjie-hdfs] [zk: localhost:2181(CONNECTED) 2] [zk: localhost:2181(CONNECTED) 2] ls /hadoop-ha/yinzhengjie-hdfs [] [zk: localhost:2181(CONNECTED) 3] [zk: localhost:2181(CONNECTED) 3] quit Quitting... 2019-04-15 08:24:18,604 [myid:] - INFO [main:ZooKeeper@693] - Session: 0x65000f71aa200001 closed 2019-04-15 08:24:18,606 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x65000f71aa200001 [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]#
2>.启动hadoop集群
[root@node101.yinzhengjie.org.cn ~]# start-dfs.sh Starting namenodes on [node101.yinzhengjie.org.cn node102.yinzhengjie.org.cn node103.yinzhengjie.org.cn] Last login: Mon Apr 15 08:15:05 PDT 2019 from 172.30.1.1 on pts/1 node101.yinzhengjie.org.cn: namenode is running as process 16023. Stop it first. Starting datanodes Last login: Mon Apr 15 08:28:26 PDT 2019 on pts/1 Starting journal nodes [node101.yinzhengjie.org.cn node102.yinzhengjie.org.cn node103.yinzhengjie.org.cn] Last login: Mon Apr 15 08:28:29 PDT 2019 on pts/1 node101.yinzhengjie.org.cn: journalnode is running as process 15733. Stop it first. node102.yinzhengjie.org.cn: journalnode is running as process 13735. Stop it first. node103.yinzhengjie.org.cn: journalnode is running as process 13723. Stop it first. Starting ZK Failover Controllers on NN hosts [node101.yinzhengjie.org.cn node102.yinzhengjie.org.cn node103.yinzhengjie.org.cn] Last login: Mon Apr 15 08:28:50 PDT 2019 on pts/1 [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# jps 16771 DataNode 15733 JournalNode 17174 DFSZKFailoverController 16023 NameNode 17228 Jps 10765 QuorumPeerMain [root@node101.yinzhengjie.org.cn ~]#
[root@node102.yinzhengjie.org.cn ~]# jps 10209 QuorumPeerMain 13939 DataNode 13735 JournalNode 14136 Jps [root@node102.yinzhengjie.org.cn ~]#
[root@node103.yinzhengjie.org.cn ~]# jps 10177 QuorumPeerMain 14209 Jps 13723 JournalNode 13995 DataNode 13917 NameNode [root@node103.yinzhengjie.org.cn ~]#
3>.启动hadoop集群成功后，我们再次查看zookeeper集群管理hadoop的节点信息
[root@node101.yinzhengjie.org.cn ~]# zkCli.sh Connecting to localhost:2181 2019-04-15 08:31:09,342 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.14-4c25d480e66aadd371de8bd2fd8da255ac140bcf, built on 03/06/2019 16:18 GMT 2019-04-15 08:31:09,346 [myid:] - INFO [main:Environment@100] - Client environment:host.name=node101.yinzhengjie.org.cn 2019-04-15 08:31:09,346 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_201 2019-04-15 08:31:09,361 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation 2019-04-15 08:31:09,361 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/yinzhengjie/softwares/jdk1.8.0_201/jre 2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/classes:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/target/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../build/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-log4j12-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/slf4j-api-1.7.25.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/netty-3.10.6.Final.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/log4j-1.2.17.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/jline-0.9.94.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../lib/audience-annotations-0.5.0.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-3.4.14.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../zookeeper-server/src/main/resources/lib/*.jar:/yinzhengjie/softwares/zookeeper-3.4.14/bin/../conf: 2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp 2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA> 2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux 2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64 2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:os.version=3.10.0-957.el7.x86_64 2019-04-15 08:31:09,362 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root 2019-04-15 08:31:09,363 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root 2019-04-15 08:31:09,363 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/root 2019-04-15 08:31:09,365 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=localhost:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@5ce65a89 Welcome to ZooKeeper! 2019-04-15 08:31:09,399 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1025] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error) JLine support is enabled 2019-04-15 08:31:09,507 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session 2019-04-15 08:31:09,531 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x65000f71aa200002, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null [zk: localhost:2181(CONNECTED) 0] ls /hadoop-ha/yinzhengjie-hdfs [ActiveBreadCrumb, ActiveStandbyElectorLock] [zk: localhost:2181(CONNECTED) 1] [zk: localhost:2181(CONNECTED) 1] ls /hadoop-ha/yinzhengjie-hdfs/ActiveBreadCrumb [] [zk: localhost:2181(CONNECTED) 2] [zk: localhost:2181(CONNECTED) 2] ls /hadoop-ha/yinzhengjie-hdfs/ActiveStandbyElectorLock [] [zk: localhost:2181(CONNECTED) 3] [zk: localhost:2181(CONNECTED) 3] get /hadoop-ha/yinzhengjie-hdfs/ActiveBreadCrumb yinzhengjie-hdfs namenode1node101.yinzhengjie.org.cn �>(�> #我们这里可以看到注册节点成功的为node101.yinzhengjie.org.cn
cZxid = 0x800000011 ctime = Mon Apr 15 08:28:56 PDT 2019 mZxid = 0x800000011 mtime = Mon Apr 15 08:28:56 PDT 2019 pZxid = 0x800000011 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 63 numChildren = 0 [zk: localhost:2181(CONNECTED) 4] [zk: localhost:2181(CONNECTED) 4] get /hadoop-ha/yinzhengjie-hdfs/ActiveStandbyElectorLock yinzhengjie-hdfs namenode1node101.yinzhengjie.org.cn �>(�> #注意，这里注册节点成功的为node101.yinzhengjie.org.cn cZxid = 0x800000010 ctime = Mon Apr 15 08:28:56 PDT 2019 mZxid = 0x800000010 mtime = Mon Apr 15 08:28:56 PDT 2019 pZxid = 0x800000010 cversion = 0 dataVersion = 0 aclVersion = 0 ephemeralOwner = 0x66000f71a2960001 dataLength = 63 numChildren = 0 [zk: localhost:2181(CONNECTED) 5]
4>.查看node101.yinzhengjie.org.cn节点的Web UI信息
5>.查看node103.yinzhengjie.org.cn节点的Web UI信息
6>.停用node101.yinzhengjie.org.cn的namenode进程停用，观察现象（注意检查多个namenode之间是否配置了免密码登录）
[root@node101.yinzhengjie.org.cn ~]# jps 16771 DataNode 15733 JournalNode 17174 DFSZKFailoverController 16023 NameNode 17255 ZooKeeperMain 17305 Jps 10765 QuorumPeerMain [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# hdfs --daemon stop namenode [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# jps 17361 Jps 16771 DataNode 15733 JournalNode 17174 DFSZKFailoverController 17255 ZooKeeperMain 10765 QuorumPeerMain [root@node101.yinzhengjie.org.cn ~]#
[root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# hdfs --daemon stop zkfc [root@node101.yinzhengjie.org.cn ~]# [root@node101.yinzhengjie.org.cn ~]# jps 18928 ZooKeeperMain 18370 DataNode 19106 Jps 18617 JournalNode 10765 QuorumPeerMain [root@node101.yinzhengjie.org.cn ~]#