• Hadoop运行任务时一直卡在: INFO mapreduce.Job: Running job


    可能原因:

    1,配置文件写错了,服务没有完全起来

    master:

    [hadoop@master hadoop]$ jps
    33792 SecondaryNameNode
    34223 Jps
    33601 NameNode
    33943 ResourceManager

    slave:

    [hadoop@slave1 hadoop]$ jps
    47298 NodeManager
    47480 Jps
    47185 DataNode

    我的不是这个entire,然后查看日志:

    [hadoop@slave1 logs]$ vim yarn-hadoop-nodemanager-slave1.log

    发现

    2017-07-13 23:17:39,954 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:40,955 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:41,957 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:42,958 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:43,959 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:44,961 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:45,962 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:46,963 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:47,964 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
    2017-07-13 23:17:47,967 ERROR org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Unexpected error starting NodeStatusUpdater
    java.net.ConnectException: Call From slave1/10.3.236.241 to 0.0.0.0:8031 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused

    百度后了解到:

    需要在yarn-site.xml中添加如下代码

    <property>
    <name>yarn.resourcemanager.address</name>
    <value>master:8032</value>
    </property>
    <property>
    <name>yarn.resourcemanager.scheduler.address</name>
    <value>master:8030</value>
    </property>
    <property>
    <name>yarn.resourcemanager.resource-tracker.address</name>
    <value>master:8031</value>
    </property>

    重启服务后

    再运行mapreduce发现可疑运行了

    问题解决

  • 相关阅读:
    Python---Flask--08--Flask-Ckeditor
    Python---Flask--07--SQLAlchemy基本关系
    Python---Flask--06--分页的实现
    Python---Flask--05--g对象和钩子函数
    maven项目管理构建
    POI 设置
    http状态码
    hibernate框架之-查询结果集返回类型
    Struts2框架之-注解开发
    Struts2框架之-Struts2的标签
  • 原文地址:https://www.cnblogs.com/cityinthesky/p/7168971.html
Copyright © 2020-2023  润新知