• 【异常】Spark提交到yarn报错:java.util.concurrent.TimeoutException: Futures timed out after [100000 milliseconds]


    1 详细异常

    java.util.concurrent.TimeoutException: Futures timed out after [100000 milliseconds]
            at scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:223)
            at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:227)
            at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:220)
            at org.apache.spark.deploy.yarn.ApplicationMaster.runDriver(ApplicationMaster.scala:468)
            at org.apache.spark.deploy.yarn.ApplicationMaster.org$apache$spark$deploy$yarn$ApplicationMaster$runImpl(ApplicationMaster.scala:305)
            at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$run$1.apply$mcV$sp(ApplicationMaster.scala:245)
            at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$run$1.apply(ApplicationMaster.scala:245)
            at org.apache.spark.deploy.yarn.ApplicationMaster$$anonfun$run$1.apply(ApplicationMaster.scala:245)
            at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$3.run(ApplicationMaster.scala:773)
            at java.security.AccessController.doPrivileged(Native Method)
            at javax.security.auth.Subject.doAs(Subject.java:422)
            at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1924)
            at org.apache.spark.deploy.yarn.ApplicationMaster.doAsUser(ApplicationMaster.scala:772)
            at org.apache.spark.deploy.yarn.ApplicationMaster.run(ApplicationMaster.scala:244)
            at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:797)
            at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)
    

      

    2 错误原因,配置文件错误

    提交到yarn 但是配置文件  应该改成yarn 
    spark.master=local
    修改为 spark.master=yarn 解决问题
     
  • 相关阅读:
    p3159 [CQOI2012]交换棋子
    三分法
    p2805 [NOI2009]植物大战僵尸
    p2604 [ZJOI2010]网络扩容
    p1129 [ZJOI2007]矩阵游戏
    有趣与愉快-------罗辑思维整理
    张小龙的书单
    会议
    使用CCProxy代理遇到的问题
    关于看书
  • 原文地址:https://www.cnblogs.com/QuestionsZhang/p/13023119.html
Copyright © 2020-2023  润新知