• spark编译报错信息简介


    spark编译需要环境
    git
    java1.7+
    maven
    R
    
    
    报错信息1:
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD FAILURE
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 52.661s (Wall Clock)
    [INFO] Finished at: Thu Aug 10 11:04:00 CST 2017
    [INFO] Final Memory: 25M/200M
    [INFO] ------------------------------------------------------------------------
    [ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce (enforce-versions) on project spark-parent_2.11: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1]
    [ERROR]
    [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
    [ERROR] Re-run Maven using the -X switch to enable full debug logging.
    [ERROR]
    [ERROR] For more information about the errors and possible solutions, please read the following articles:
    [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
    
    报错信息1的解决办法:maven 版本需要是3.3.9
    
    报错信息2:
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD FAILURE
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 11:25 min (Wall Clock)
    [INFO] Finished at: 2017-08-10T11:25:18+08:00
    [INFO] Final Memory: 50M/633M
    [INFO] ------------------------------------------------------------------------
    [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (sparkr-pkg) on project spark-core_2.11: Command execution failed. Process exited with an error: 1 (Exit value: 1) -> [Help 1]
    [ERROR]
    [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
    [ERROR] Re-run Maven using the -X switch to enable full debug logging.
    [ERROR]
    [ERROR] For more information about the errors and possible solutions, please read the following articles:
    [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
    [ERROR]
    [ERROR] After correcting the problems, you can resume the build with the command
    [ERROR]   mvn <goals> -rf :spark-core_2.11
    
    报错信息2的解决办法
    yum install -y R
    
    报错信息3
    [INFO] Spark Project Java 8 Tests ......................... SKIPPED
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD FAILURE
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 38:34 min (Wall Clock)
    [INFO] Finished at: 2017-08-10T17:21:52+08:00
    [INFO] Final Memory: 69M/664M
    [INFO] ------------------------------------------------------------------------
    [ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile (scala-test-compile-first) on project spark-hive_2.11: Execution scala-test-compile-first of goal net.alchim31.maven:scala-maven-plugin:3.2.2:testCompile failed. CompileFailed -> [Help 1]
    [ERROR]
    [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
    [ERROR] Re-run Maven using the -X switch to enable full debug logging.
    [ERROR]
    [ERROR] For more information about the errors and possible solutions, please read the following articles:
    [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
    [ERROR]
    [ERROR] After correcting the problems, you can resume the build with the command
    [ERROR]   mvn <goals> -rf :spark-hive_2.11
    Destroying 1 processes
    Destroying process..
    Destroyed 1 processes
    
    报错信息3的解决办法
    编译过程中加上参数 -DskipTests

    参考:Spark源码的编译过程详细解读(各版本)(博主推荐)

     
     
  • 相关阅读:
    【机器学习笔记五】聚类
    【机器学习笔记四】分类算法
    【机器学习笔记三】回归分析
    【机器学习笔记二】回归分析
    【机器学习笔记一】协同过滤算法
    一个简单的前端事件框架
    javascript面向对象理解的补充
    kafka基础知识点
    linux 常用监控命令备注
    最优化算法-梯度下降
  • 原文地址:https://www.cnblogs.com/zhzhang/p/7347523.html
Copyright © 2020-2023  润新知