一、问题：
在使用Hive0.11进行select查询的时候报：
hive> create table zhifu2 as select distinct uid,substr(time,1,12),site from zhifu; Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks not specified. Estimated from input data size: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=<number> In order to limit the maximum number of reducers: set hive.exec.reducers.max=<number> In order to set a constant number of reducers: set mapred.reduce.tasks=<number> Starting Job = job_201403211432_0001, Tracking URL = http://node1:60030/jobdetails.jsp?jobid=job_201403211432_0001 Kill Command = /app/hadoop/bin/../bin/hadoop job -kill job_201403211432_0001 Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1 2014-03-21 15:51:13,341 Stage-1 map = 0%, reduce = 0% 2014-03-21 15:52:14,121 Stage-1 map = 0%, reduce = 0% 2014-03-21 15:53:15,019 Stage-1 map = 0%, reduce = 0% 2014-03-21 15:54:21,120 Stage-1 map = 0%, reduce = 0% 2014-03-21 15:55:55,130 Stage-1 map = 0%, reduce = 0% 2014-03-21 15:57:40,915 Stage-1 map = 0%, reduce = 0% 2014-03-21 15:59:28,636 Stage-1 map = 0%, reduce = 0% Ended Job = job_201403211432_0001 with errors Error during job, obtaining debugging information... Job Tracking URL: http://node1:60030/jobdetails.jsp?jobid=job_201403211432_0001 Examining task ID: task_201403211432_0001_m_000002 (and more) from job job_201403211432_0001 null Exception in thread "Thread-46" java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.JobDebugger.showJobFailDebugInfo(JobDebugger.java:313) at org.apache.hadoop.hive.ql.exec.JobDebugger.run(JobDebugger.java:118) at java.lang.Thread.run(Thread.java:722) FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask MapReduce Jobs Launched: Job 0: Map: 1 Reduce: 1 HDFS Read: 0 HDFS Write: 0 FAIL Total MapReduce CPU Time Spent: 0 msec
查看jobtracker页面中该job的信息发现有：
java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) at org.apache.hadoop.mapred.Child.main(Child.java:170) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ... 5 more Caused by: java.lang.RuntimeException: Error in configuring object at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) ... 10 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) ... 13 more Caused by: java.lang.RuntimeException: Map operator initialization failed at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121) ... 18 more Caused by: java.lang.NoClassDefFoundError: org/codehaus/jackson/JsonFactory at org.apache.hadoop.hive.ql.udf.generic.GenericUDTFJSONTuple.(GenericUDTFJSONTuple.java:56) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:525) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.registerGenericUDTF(FunctionRegistry.java:526) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.registerGenericUDTF(FunctionRegistry.java:520) at org.apache.hadoop.hive.ql.exec.FunctionRegistry.(FunctionRegistry.java:423) at org.apache.hadoop.hive.ql.exec.DefaultUDFMethodResolver.getEvalMethod(DefaultUDFMethodResolver.java:59) at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:154) at org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:111) at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:141) at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:227) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407) at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451) at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407) at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:186) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:543) at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375) at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:100) ... 18 more Caused by: java.lang.ClassNotFoundException: org.codehaus.jackson.JsonFactory at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:423) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) ... 44 more
其中Caused by: java.lang.NoClassDefFoundError: org/codehaus/jackson/JsonFactory是引起问题的原因。
二、原因：
Hadoop版本是0.20.2的，在$HADOOP_HOME/lib中不包含：
[hadoop@node2 lib]$ ll jackson* -rw-rw-r-- 1 hadoop hadoop 227500 2012-01-21 09:24 jackson-core-asl-1.8.8.jar -rw-rw-r-- 1 hadoop hadoop 17884 2012-01-21 09:26 jackson-jaxrs-1.8.8.jar -rw-rw-r-- 1 hadoop hadoop 668564 2012-01-21 09:24 jackson-mapper-asl-1.8.8.jar -rw-rw-r-- 1 hadoop hadoop 32353 2012-01-21 09:26 jackson-xc-1.8.8.jar
这四个包，因此会报错。
三、解决方案：
方法有两种：
1、在集群各节点上的hadoop的lib包增加相关的4个jar包，并重启集群。
2、hive prompt里输入：
hive> add jars /app/hive/lib/jackson-core-asl-1.8.8.jar jackson-jaxrs-1.8.8.jar jackson-mapper-asl-1.8.8.jar jackson-xc-1.8.8.jar > ; Added /app/hive/lib/jackson-core-asl-1.8.8.jar to class path Added resource: /app/hive/lib/jackson-core-asl-1.8.8.jar Added jackson-jaxrs-1.8.8.jar to class path Added resource: jackson-jaxrs-1.8.8.jar Added jackson-mapper-asl-1.8.8.jar to class path Added resource: jackson-mapper-asl-1.8.8.jar Added jackson-xc-1.8.8.jar to class path Added resource: jackson-xc-1.8.8.jar
然后再执行select操作