Error: java.lang.NullPointerException
at outputformat.MysqlOutputFormat.getRecordWriter(MysqlOutputFormat.java:27)
at org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.<init>(ReduceTask.java:540)
at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:614)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
去yarn服务器查看本次mapreduce 任务的日志http://linux02:8088/proxy/application_1536674553441_0007/
方案: 查找代码 一定是密码出问题了
有时候没有加载到mysql驱动也报同样的错误 这样就是在执行的时候 加上 -libjars 例如:
~/modules/hadoop-2.7.2/bin/yarn jar ct_analysis-1.0-SNAPSHOT.jar mr.CountDurationRunner -libjars ./mysqlLib/mysql-connector-java-5.1.46-bin.jar
心得:1:不同的代码错误可能会导致同样的报错
2,排查的方式就是让任务执行完毕(就算是错误的)这样去看日志才能有全部的报错信息