官网指南
Purpose
Using the CLI MiniCluster, users can simply start and stop a single-node Hadoop cluster with a single command, and without the need to set any environment variables or manage configuration files. The CLI MiniCluster starts both a YARN/MapReduce & HDFS clusters.
This is useful for cases where users want to quickly experiment with a real Hadoop cluster or test non-Java programs that rely on significant Hadoop functionality.
安装
protobuf
下载
安装
$ ./configure
$ make
$ make install
查看
$ protoc --version
libprotoc 2.5.0
minicluster
下载
安装
$ mvn clean install -DskipTests
$ mvn package -Pdist -Dtar -DskipTests -Dmaven.javadoc.skip
$ cd /hadoop-dist/target/hadoop-3.2.1
- 不知道为啥一直报ByteBuffer的position方法找不到的错，直接下载个二进制包，3.2.1二进制下载，解压后替换这个目录hadoop-3.2.1
- 确保目录权限没问题
- 第一次运行时要加上-format参数
$ bin/mapred minicluster -format
- 如果没指定端口，通过lsof -i|grep java查看LISTEN的端口