• spark利用yarn提交任务报:YARN application has exited unexpectedly with state UNDEFINED


    spark用yarn提交任务会报
    ERROR cluster.YarnClientSchedulerBackend: YARN application has exited unexpectedly with state UNDEFINED! Check the YARN application logs for more details.
    ERROR cluster.YarnClientSchedulerBackend: Diagnostics message: Shutdown hook called before final status was reported.

    只需要将hadoop集群的yarn-site.xml配置中加入如下两行即可解决:

    <property>
        <name>yarn.nodemanager.pmem-check-enabled</name>
        <value>false</value>
    </property>
    <property>
        <name>yarn.nodemanager.vmem-check-enabled</name>
        <value>false</value>
    </property>

    如果你用的java8,则多会报此错误。

  • 相关阅读:
    bzoj1505 [NOI2004]小H的小屋
    最大值
    数学
    OI中的快速傅里叶变换(FFT)
    旅游规划
    加分二叉树
    浅谈 字符串hash
    二分的弟弟“三分”
    Trie树(c++)
    克鲁斯卡尔
  • 原文地址:https://www.cnblogs.com/osyun/p/11906693.html
Copyright © 2020-2023  润新知