• hadoop job -kill 与 yarn application -kii(作业卡了或作业重复提交或MapReduce任务运行到running job卡住)


       

      问题详情

     解决办法

    [hadoop@master ~]$ hadoop job -kill job_1493782088693_0001
    DEPRECATED: Use of this script to execute mapred command is deprecated.
    Instead use the mapred command for it.
    
    17/05/03 11:48:25 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    17/05/03 11:48:25 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
    17/05/03 11:48:29 INFO impl.YarnClientImpl: Killed application application_1493782088693_0001
    Killed job job_1493782088693_0001
    
    
    
    
    
    
    [hadoop@master ~]$ hadoop job -kill job_1493782088693_0002
    DEPRECATED: Use of this script to execute mapred command is deprecated.
    Instead use the mapred command for it.
    
    17/05/03 11:48:25 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    17/05/03 11:48:25 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
    17/05/03 11:48:29 INFO impl.YarnClientImpl: Killed application application_1493782088693_0001
    Killed job job_1493782088693_0002
    
    
    
    
    
    
    [hadoop@master ~]$ hadoop job -kill job_1493782088693_0003
    DEPRECATED: Use of this script to execute mapred command is deprecated.
    Instead use the mapred command for it.
    
    17/05/03 11:48:25 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    17/05/03 11:48:25 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
    17/05/03 11:48:29 INFO impl.YarnClientImpl: Killed application application_1493782088693_0001
    Killed job job_1493782088693_0003

      有时候上述这样kill做下来,并不管用,得再来

    [hadoop@master ~]$ yarn application -kill application_1493782088693_0001
    17/05/03 14:45:23 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
    17/05/03 14:45:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    Application application_1493782088693_0001 has already finished 
    [hadoop@master ~]$ yarn application -kill application_1493782088693_0002
    17/05/03 14:45:59 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
    17/05/03 14:45:59 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    Application application_1493782088693_0002 has already finished 
    [hadoop@master ~]$ yarn application -kill application_1493782088693_0003
    17/05/03 14:46:47 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
    17/05/03 14:46:49 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    Application application_1493782088693_0003 has already finished 
    [hadoop@master ~]$

    [hadoop@master ~]$ hadoop job -list
    DEPRECATED: Use of this script to execute mapred command is deprecated.
    Instead use the mapred command for it.
    
    17/05/03 14:39:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
    17/05/03 14:39:24 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
    Total jobs:0
                      JobId         State         StartTime        UserName           Queue      Priority     UsedContainers     RsvdContainers     UsedMem     RsvdMem     NeededMem       AM info
    [hadoop@master ~]$ 

      同时,大家要注意,是否是进程的消失?

      也会是你的slave1 还是 slave2的进程自动消失了。注意 ,这是个很隐蔽的问题。

       重新停止集群,再重新启动集群。

      如果还是出现这个问题的话,则

    MapReduce任务运行到running job卡住

      关于这里参数配置的解释,见

    Hadoop YARN配置参数剖析(2)—权限与日志聚集相关参数

     注意,对master、slave1和slave2都要操作,然后,再[hadoop@master hadoop-2.6.0]$ sbin/stop-all.sh

      再[hadoop@master hadoop-2.6.0]$ sbin/start-all.sh即可。

       成功!

  • 相关阅读:
    linux上搭建tingproxy服务
    windows上搭建linux系统
    PHP将图片流存为图片文件
    openLayer矩形框选要素,展示要素属性
    点击选中的要素,更换标注图片,并添加文本标注
    openLayer点击要素获取对应的属性信息
    openLayer实现放大缩小
    openLayer的切换中心点、定位功能
    使用openLayer加载arcgis中的地图
    openLayer实现两个地图联动
  • 原文地址:https://www.cnblogs.com/zlslch/p/6801898.html
Copyright © 2020-2023  润新知