• [Hadoop]单机尝试安装并测试Hadoop2.7.1(附带注释脚本)


    还算是比较顺利,在代码中注释写得已经比较详细了。附上脚本,有兴趣的朋友可以尝试一下。注意bashrc中添加的环境变量名的修改,否则会出现找不到主类等问题。

     1 ###ubuntu 14.04 LTS
     2 
     3 ##make hadoop account
     4 sudo addgroup hadoop    #make a group named hadoop
     5 sudo adduser -ingroup hadoop hadoop    #add an user named hadoop
     6 sudo vi /etc/sudoers    #add hadoop into sudoers(hadoop=root)
     7 
     8 ##change your account
     9 sudo su hadoop
    10 
    11 ##install ssh
    12 sudo apt-get install openssh-server    #install openssh-server
    13 sudo /etc/init.d/ssh start    #start ssh server
    14 ps -e | grep ssh    #check whether ssh is started or not
    15 
    16 ##log in without password
    17 ssh-keygen -t rsa -P ""    #log in without password generate rsa & rsa.pub
    18 cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys    #add authorized_keys to realize 'log in without password'
    19 ssh localhost    #check
    20 exit    #check
    21 
    22 ##install java environment
    23 sudo apt-get install openjdk-7-jdk    #install jdk
    24 java -version    #check whether java is installed or not
    25 
    26 ##get & install hadoop 2.7.1    (under user : hadoop)
    27 cd ~
    28 mkdir hadoop
    29 sudo wget http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.7.1/hadoop-2.7.1.tar.gz    #get hadoop2.7.1
    30 sudo tar xzf hadoop-2.7.1.tar.gz    #unzip the hadoop-*.*.*.tar.gz
    31 sudo rm hadoop-2.7.1.tar.gz    #remove the zip
    32 sudo mv hadoop-2.7.1 /usr/local/    #install hadoop at local
    33 sudo chmod 774 /usr/local/hadoop     #granted permissions to users (r&w)
    34 
    35 ##configure ~/.bashrc
    36 update-alternatives --config java    #get java's path (stared one. such as '/usr/lib/jvm/java-7-openjdk-amd64/jre/bin/java'. but we just need '/usr/lib/jvm/java-7-openjdk-amd64' this shorter path to set up JAVA_HOME environment variables)
    37 sudo vi ~/.bashrc    #edit bashrc file
    38 
    39 ##add the content below to the end of bashrc
    40 #--------------------------------------------------------------#
    41 
    42 #HADOOP VARIABLES START
    43 export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
    44 export HADOOP_INSTALL=/usr/local/hadoop-2.7.1
    45 export PATH=$PATH:$HADOOP_INSTALL/bin
    46 export PATH=$PATH:$HADOOP_INSTALL/sbin
    47 export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
    48 export HADOOP_COMMON_HOME=$HADOOP_INSTALL
    49 export HADOOP_HDFS_HOME=$HADOOP_INSTALL
    50 export YARN_HOME=$HADOOP_INSTALL
    51 export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native
    52 export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib"
    53 #HADOOP VARIABLES END
    54 
    55 #--------------------------------------------------------------#
    56 source ~/.bashrc    #make the environment variables come into effect
    57 
    58 ##configure hadoop
    59 sudo vi /usr/local/hadoop-2.7.1/etc/hadoop/hadoop-env.sh     #edit hadoop-env.sh
    60 ?JAVA_HOME    #(in vim) locate JAVA_HOME
    61 #change 'export JAVA_HOME=${JAVA_HOME}' into 'export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64'
    62 source /usr/local/hadoop-2.7.1/etc/hadoop/hadoop-env.sh #update
    63 
    64 ##test
    65 cd /usr/local/hadoop-2.7.1/    #go there
    66 sudo mkdir test
    67 sudo cp README.txt test
    68 bin/hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.7.1-sources.jar org.apache.hadoop.examples.WordCount input output
  • 相关阅读:
    2017-2018-1 20155338 《信息安全系统设计基础》 第三周学习总结
    2017-2018-1 20155338 《信息安全系统设计基础》 第二周课堂测试
    2017-2018-1 20155338 《信息安全系统设计基础》第1周学习总结
    20155338 2016-2017-2 《JAVA程序设计》课程总结
    20155338 《JAVA程序设计》实验五网络编程与安全实验报告
    20155338 2016-2017-2《Java程序设计》实验四Android程序开发实验报告
    20155338 《Java程序设计》实验三(敏捷开发与XP实践)实验报告
    20155338 2016-2017-2 《Java程序设计》第10周学习总结
    【私人向】Java复习笔记
    2017-2018-1 20155316 《信息安全系统设计基础》第2周学习总结
  • 原文地址:https://www.cnblogs.com/kirai/p/4875879.html
Copyright © 2020-2023  润新知