• hadoop 2.8.5安装步骤


    1、创建hadoop用户,作为haoop的运行用户

    2、配置JAVA_HOME环境变量,修改/etc/profile
    export JAVA_HOME=/usr/java/jdk1.8.0_51
    export PATH=${JAVA_HOME}/bin:$PATH

    3、配置hadoop用户的PATH环境变量,修改.bash_profile
    $HOME/hadoop/bin:$HOME/hadoop/bin

    4、配置免密登录,需要配置master免密登录slave和master自身
    a、namenode上执行 ssh-keygen -t rsa 生成key
    b、cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
    c、将生成的公钥复制到机器namenode上的~/.ssh/authorized_keys中
    ssh-copy-id -i ~/.ssh/id_rsa.pub hadoop@192.168.1.100
    d、测试能登录到自身

    5、配置hadoop-env.sh及yarn-env.sh中的javahome
    export JAVA_HOME=/usr/java/jdk1.8.0_51
    export HADOOP_CONF_DIR=/home/hadoop/hadoop/etc/hadoop
    6、修改slave 文件
    7、创建目录
    mkdir tmp
    mkdir hdfs
    mkdir hdfs/name
    mkdir hdfs/data
    8、修改core-site.xml配置文件
    <configuration>
    <configuration>
    <property>
    <name>hadoop.tmp.dir</name>
    <value>/home/hadoop/tmp</value>
    <final>true</final>
    <description>A base for other temporary directories.</description>
    </property>
    <property>
    <name>fs.default.name</name>
    <value>hdfs://rhel01:9000</value>
    <final>true</final>
    </property>
    </configuration>

    9、修改hdfs-site.xml文件
    <configuration>
    <property>
    <name>dfs.http.address</name>
    <value>rhel01:50070</value>
    </property>
    <property>
    <name>dfs.replication</name>
    <value>1</value>
    </property>
    <property>
    <name>dfs.namenode.secondary.http-address</name>
    <value>rhel01:50090</value>
    </property>
    </configuration>

    10、修改mapred-site.xml
    <configuration>
    <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
    </property>
    <property>
    <name>mapreduce.jobhistory.address</name>
    <value>rhel01:10020</value>
    </property>
    <property>
    <name>mapreduce.jobhistory.webapp.address</name>
    <value>rhel01:19888</value>
    </property>
    </configuration>

    11、修改yarn-site.xml文件
    <configuration>
    <property>
    <name>yarn.resourcemanager.hostname</name>
    <value>rhel01</value>
    </property>
    <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
    </property>
    <property>
    <name>yarn.resourcemanager.address</name>
    <value>rhel01:8032</value>
    </property>
    <property>
    <name>yarn.resourcemanager.scheduler.address</name>
    <value>rhel01:8030</value>
    </property>
    <property>
    <name>yarn.resourcemanager.resource-tracker.address</name>
    <value>rhel01:8031</value>
    </property>
    <property>
    <name>yarn.resourcemanager.admin.address</name>
    <value>rhel01:8033</value>
    </property>
    <property>
    <name>yarn.resourcemanager.webapp.address</name>
    <value>rhel01:8088</value>
    </property>
    </configuration>

    12、格式化namenode
    hdfs namenode -format

    13、启动dfs
    start-dfs.sh

    14、启动yarn
    start-yarn.sh

  • 相关阅读:
    Asp.net core 项目的发布和运行环境部署完整介绍(Window+IIS7)
    让IIS支持PUT和Delete请求
    js实现图片打点
    Ogre空间变换,setPosition(),setDerivedPosition()和translate()的正确使用。
    obs 工作室模式
    obs局域网串流插件之“OBSNDI”
    obs 媒体源获取之“VLC视频源”
    谈谈php中的mvc
    PHP 开发之MVC架构
    一个简单的php MVC实例
  • 原文地址:https://www.cnblogs.com/timlong/p/9802045.html
Copyright © 2020-2023  润新知