• yarn以及mapreduce部署


    修改hadoop的配置文件yarn-site.xml:

    <configuration>
    
    <!-- Site specific YARN configuration properties -->
    <property>
    <name>yarn.resourcemanager.hostname</name>
    <value>ubuntu-00</value>
    </property>
    
    <property>
    <name>yarn.nodemanager.aux-service</name>
    <value>mapreduce_shuffle</value>
    </property>
    
    
    </configuration>

    复制该配置文件到其他服务器

    scp yarn-site.xml ubuntu-01:$PWD

    yarn启动命令:

    start-yarn.sh

    MapReduce部署:

    修改mapred-site.xml

    <configuration>
    <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
    </property>
    
    <property>
     <name>mapreduce.jobhistory.address</name>
     <value>fireslate.cis.umac.mo:10020</value>
    </property>
    <property>
     <name>mapreduce.jobhistory.webapp.address</name>
     <value>fireslate.cis.umac.mo:19888</value>
    </property>
    
    <property>
    <name>mapreduce.application.classpath</name>
            <value>
                    $HADOOP_CONF_DIR,
                    $HADOOP_COMMON_HOME/share/hadoop/common/*,
                    $HADOOP_COMMON_HOME/share/hadoop/common/lib/*,
                    $HADOOP_HDFS_HOME/share/hadoop/hdfs/*,
                    $HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*,
                    $HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*,
                    $HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*,
                    $HADOOP_YARN_HOME/share/hadoop/yarn/*,
                    $HADOOP_YARN_HOME/share/hadoop/yarn/lib/*
            </value>
    </property>
    </configuration>

    修改mapred-env.sh,配置java_home

    export JAVA_HOME=/home/ldy//home/ldy/apps/jdk1.8.0_221
  • 相关阅读:
    如何在linux下使用sudo命令不用输入密码
    QT读写ini配置文件
    dubbo的灰度发布
    dubbo的重试原则
    dubbo的启动时检查
    dubbo的超时处理和配置覆盖
    dubbo配置文件加载顺序
    java基础-泛型的优点
    idea提交多个项目到github的同一个repository
    【掌握】dubbo环境搭建linux
  • 原文地址:https://www.cnblogs.com/ldy233/p/11456038.html
Copyright © 2020-2023  润新知