• Hadoop运行任务时一直卡在: INFO mapreduce.Job: Running job


    可能原因:

    1,配置文件写错了,服务没有完全起来

    master:

    [hadoop@master hadoop]$ jps
    33792 SecondaryNameNode
    34223 Jps
    33601 NameNode
    33943 ResourceManager

    slave:

    [hadoop@slave1 hadoop]$ jps
    47298 NodeManager
    47480 Jps
    47185 DataNode

    我的不是这个entire,然后查看日志:

    [hadoop@slave1 logs]$ vim yarn-hadoop-nodemanager-slave1.log

    发现

    2017-07-13 23:17:39,954 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:40,955 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:41,957 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:42,958 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:43,959 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:44,961 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:45,962 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:46,963 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:47,964 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:47,967 ERROR org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Unexpected error starting NodeStatusUpdater
    java.net.ConnectException: Call From slave1/10.3.236.241 to 0.0.0.0:8031 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused

    百度后了解到:

    需要在yarn-site.xml中添加如下代码

    <property>
    <name>yarn.resourcemanager.address</name>
    <value>master:8032</value>
    </property>
    <property>
    <name>yarn.resourcemanager.scheduler.address</name>
    <value>master:8030</value>
    </property>
    <property>
    <name>yarn.resourcemanager.resource-tracker.address</name>
    <value>master:8031</value>
    </property>

    重启服务后

    再运行mapreduce发现可疑运行了

    问题解决

  • 相关阅读:
    【动态规划】数的划分
    【动态规划】开心的小明
    【动态规划】回文字符串
    【动态规划】skiing_深度搜索_动态规划
    【动态规划】最大子串和
    JDBC中 execute 与 executeUpdate的区别
    poj 2449 Remmarguts' Date 求第k短路 Astar算法
    ACM-ICPC 2018 徐州赛区网络预赛 J. Maze Designer 最大生成树 lca
    hdu 2586 How far away ? 倍增求LCA
    acm模板生成
  • 原文地址:https://www.cnblogs.com/cityinthesky/p/7168971.html
Copyright © 2020-2023  润新知