hadoop docker

Hadoop Docker项目

https://github.com/sequenceiq/hadoop-docker

sequenceiq提供2.3, 2.4.0, 2.4.1, 2.5.0版本的Hadoop，只需把下面2.5.0换成你需要的就可以

可以下载下来自己编译

docker build  -t sequenceiq/hadoop-docker:2.5.0 .

可以直接下载

docker pull sequenceiq/hadoop-docker:2.5.0

运行方法

docker run -i -t sequenceiq/hadoop-docker:2.5.0 /etc/bootstrap.sh -bash

测试运行

cd $HADOOP_PREFIX #进入Hadoop目录
bin/hadoop fs -ls #查看hdfs目录
bin/hadoop fs -mkdir wordcountinput #创建目录
bin/hadoop fs -ls #查看hdfs目录
bin/hadoop fs -put etc/hadoop/core-site.xml  /wordcountinput #上传文件
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.0.jar wordcount /wordcountinput /wordcountoutput
bin/hadoop fs -cat /wordcountoutput/* #view result

官方测试例子

cd $HADOOP_PREFIX
# run the mapreduce
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.0.jar grep input output 'dfs[a-z.]+'

# check the output
bin/hdfs dfs -cat output/*

将Hadoop在后台运行

后台运行Docker，需要加上  -d 参数

set hadoop port 50010 50070 to you outer server

docker run -d -p 50010  -p 50070 -i -t sequenceiq/hadoop-docker:2.5.0 /etc/bootstrap.sh –bash

use docker ps command, you get

docker set port 50070 to port 49154

you can use http://127.0.0.1:49154 to Browse HDFS.

the number 49154  is random generate.

docker run -d -p 50010:50010 -p 50070:50070 -i -t sequenceiq/hadoop-docker:2.5.0 /etc/bootstrap.sh –bash

use the up command, docker will set image’s 50070 to your machine’s 50070

docker run -d -p 9000:9000 -p 50020:50020 -p 50090:50090 -p 50070:50070 -p 50010:50010 -p 50075:50075 -p 8031:8031 -p 8032:8032 -p 8033:8033 -p 8040:8040 -p 8042:8042 -p 49707:49707 -p 8088:8088 -p 8030:8030 -i -t sequenceiq/hadoop-docker:2.5.0 /etc/bootstrap.sh –bash