报错信息:
java.lang.NoClassDefFoundError: org/apache/spark/api/java/function/Function at java.lang.Class.getDeclaredMethods0(Native Method) at java.lang.Class.privateGetDeclaredMethods(Class.java:2701) at java.lang.Class.privateGetMethodRecursive(Class.java:3048) at java.lang.Class.getMethod0(Class.java:3018) at java.lang.Class.getMethod(Class.java:1784) at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526) Caused by: java.lang.ClassNotFoundException: org.apache.spark.api.java.function.Function at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 7 more Error: A JNI error has occurred, please check your installation and try again Exception in thread "main"
上面的类是在spark-core中,查看pom.xml
<!-- Spark Core 依赖-->
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_2.11</artifactId>
<version>${spark.version}</version>
<scope>provided</scope>
</dependency>
解决方法,下图中打钩:
原因是:IDEA默认下是不加载pom下的provided依赖的,而Eclipse是支持的。
可以参考:https://blog.csdn.net/lh11077/article/details/80143996