- 准备虚拟机：linux-rhel-7.4-server，由于不使用虚拟机进行联网，所以选择host-only网络模式。此处，需要再VitralBox的管理菜单中的主机网络管理器新建一个虚拟网卡。安装完成虚拟机之后，默认网卡是关闭的，需要进行开启，指令如下：
[root@hadoop-01 network-scripts]# vi ifcfg-enp0s3 #默认网卡配置 TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=dhcp DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no IPV6_ADDR_GEN_MODE=stable-privacy NAME=enp0s3 UUID=9e448496-ecd5-4122-a91f-91f91bd15f5e DEVICE=enp0s3 ONBOOT=yes #修改为 yes,默认是no然后重启虚拟机
- 此时再来查看本机网络配置如下，就可以与宿主机的同网段的虚拟网卡进行通讯了，如果宿主机启用的网络共享，那么可以让虚拟机进行联网
[root@hadoop-01 network-scripts]# ifconfig enp0s3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 #已经有IP分配进来 inet 192.168.56.101 netmask 255.255.255.0 broadcast 192.168.56.255 inet6 fe80::bcf9:1d0d:e75d:500f prefixlen 64 scopeid 0x20<link> ether 08:00:27:fb:11:51 txqueuelen 1000 (Ethernet) RX packets 5763894 bytes 8204104505 (7.6 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 310622 bytes 23522131 (22.4 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1 (Local Loopback) RX packets 1698 bytes 134024 (130.8 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1698 bytes 134024 (130.8 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
- 上传准备好的程序包
ZBMAC-C03VQ091H:实验介质 hadoop$ ls ZooInspector.zip mysql-5.7.19-1.el7.x86_64.rpm-bundle.tar ZooViewer.zip mysql-connector-java-5.1.43-bin.jar apache-flume-1.7.0-bin.tar.gz pig-0.17.0.tar.gz apache-hive-2.3.0-bin.tar.gz sqoop-1.4.5.bin__hadoop-0.23.tar.gz hadoop-2.7.3.tar.gz virtualbox hbase-1.3.1-bin.tar.gz winscp513setup.exe hue-4.0.1.tgz zookeeper-3.4.10.tar.gz jdk-8u144-linux-x64.tar.gz #使用SCP命令： scp ./* hadoop-01@192.168.56.101:/home/hadoop-01/
- 安装JDK1.8并解压：
tar -zxvf jdk-8u144-linux-x64.tar.gz
- 设置当前用户的java_home 编辑~/.bash_profile
JAVA_HOME=/home/hadoop-02/sdk-home/jdk1.8.0_144 export JAVA_HOME PATH=$JAVA_HOME/bin:$PATH export PATH
- java环境变量设置成功后，使用 ：java -version查看版本是否正确
- 解压hadoop
tar -zxvf hadoop-2.7.3.tar.gz
- hadoop目录结构解释：
[hadoop-02@hadoop-02 ~]$ tree -L 3 /home/hadoop-02/sdk-home/hadoop-2.7.3/ /home/hadoop-02/sdk-home/hadoop-2.7.3/ |-- bin # 可执行命令 | |-- container-executor | |-- hadoop| |-- yarn | `-- yarn.cmd |-- etc | `-- hadoop # 配置文件目录 | |-- capacity-scheduler.xml | |-- configuration.xsl | | |-- yarn-env.sh | `-- yarn-site.xml |-- include | |-- hdfs.h | | `-- TemplateFactory.hh |-- lib | `-- native | |-- libhadoop.a | |-- libhadooppipes.a | | `-- libhdfs.so.0.0.0 |-- libexec | |-- hadoop-config.cmd | |-- hadoop-config.sh | |-- LICENSE.txt |-- logs | |-- hadoop-hadoop-02-datanode-hadoop-02.log | |-- hadoop-hadoop-02-datanode-hadoop-02.out | |-- |-- NOTICE.txt |-- README.txt |-- sbin # 启停脚本 | |-- distribute-exclude.sh | |-- hadoop-daemon.sh| `-- yarn-daemons.sh `-- share |-- doc #文档目录 | `-- hadoop `-- hadoop #所有jar包 |-- common |-- hdfs |-- httpfs |-- kms |-- mapreduce #内含示例jar包 |-- tools `-- yarn
- 设置hadoop的环境变量：
# /hadoop-2.7.3/etc/hadoop/hadoop-env.sh #修改JAVA_HOME为实际对应目录： # The java implementation to use. export JAVA_HOME=/home/hadoop-02/sdk-home/jdk1.8.0_144/
- 至此本机环境已经准备好找到hadoop的sbin目录执行start-all.sh
[hadoop-02@hadoop-02 sbin]$ ./start-all.sh This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh Incorrect configuration: namenode address dfs.namenode.servicerpc-address or dfs.namenode.rpc-address is not configured. Starting namenodes on [] hadoop-02@localhost's password: localhost: starting namenode, logging to /home/hadoop-02/sdk-home/hadoop-2.7.3/logs/hadoop-hadoop-02-namenode-hadoop-02.out hadoop-02@localhost's password: localhost: starting datanode, logging to /home/hadoop-02/sdk-home/hadoop-2.7.3/logs/hadoop-hadoop-02-datanode-hadoop-02.out Starting secondary namenodes [0.0.0.0] hadoop-02@0.0.0.0's password: 0.0.0.0: starting secondarynamenode, logging to /home/hadoop-02/sdk-home/hadoop-2.7.3/logs/hadoop-hadoop-02-secondarynamenode-hadoop-02.out 0.0.0.0: Exception in thread "main" java.lang.IllegalArgumentException: Invalid URI for NameNode address (check fs.defaultFS): file:/// has no authority. 0.0.0.0: at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:472) 0.0.0.0: at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:462) 0.0.0.0: at org.apache.hadoop.hdfs.server.namenode.NameNode.getServiceAddress(NameNode.java:455) 0.0.0.0: at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:229) 0.0.0.0: at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:192) 0.0.0.0: at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:671) starting yarn daemons starting resourcemanager, logging to /home/hadoop-02/sdk-home/hadoop-2.7.3/logs/yarn-hadoop-02-resourcemanager-hadoop-02.out hadoop-02@localhost's password: localhost: starting nodemanager, logging to /home/hadoop-02/sdk-home/hadoop-2.7.3/logs/yarn-hadoop-02-nodemanager-hadoop-02.out [hadoop-02@hadoop-02 sbin]$
- 中间如果没有配置免密登录，会出现四次输入密码，注意观察日志在启动对应的服务。
- 检查服务是否正常,包含如下服务：
[hadoop-02@hadoop-02 sbin]$ jps 6305 Jps 6178 NodeManager 5883 ResourceManager [hadoop-02@hadoop-02 sbin]$ - 运行wordcount 示例：
[hadoop-02@hadoop-02 sbin]$ hadoop jar /home/hadoop-02/sdk-home/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar wordcount /home/hadoop-02/test_hadoop/wordcount.txt /home/hadoop-02/test_hadoop/wordcount_output/
- 到输出目录就能看到结果文件如下：
[hadoop-02@hadoop-02 sbin]$ cd /home/hadoop-02/test_hadoop/wordcount_output [hadoop-02@hadoop-02 wordcount_output]$ ls _SUCCESS part-r-00000
- 至此本地环境搭建就介绍到这里