• hive遇到的问题以及解决办法


    hive java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.data.JsonSerDe not found

    hadoop    | Error: java.lang.RuntimeException: Error in configuring object
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:113)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:79)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
    hadoop    | 	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:455)
    hadoop    | 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
    hadoop    | 	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177)
    hadoop    | 	at java.security.AccessController.doPrivileged(Native Method)
    hadoop    | 	at javax.security.auth.Subject.doAs(Subject.java:422)
    hadoop    | 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1886)
    hadoop    | 	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:171)
    hadoop    | Caused by: java.lang.reflect.InvocationTargetException
    hadoop    | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    hadoop    | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    hadoop    | 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    hadoop    | 	at java.lang.reflect.Method.invoke(Method.java:498)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:110)
    hadoop    | 	... 9 more
    hadoop    | Caused by: java.lang.RuntimeException: Error in configuring object
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:113)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:79)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:137)
    hadoop    | 	at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
    hadoop    | 	... 14 more
    hadoop    | Caused by: java.lang.reflect.InvocationTargetException
    hadoop    | 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    hadoop    | 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    hadoop    | 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    hadoop    | 	at java.lang.reflect.Method.invoke(Method.java:498)
    hadoop    | 	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:110)
    hadoop    | 	... 17 more
    hadoop    | Caused by: java.lang.RuntimeException: Map operator initialization failed
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:137)
    hadoop    | 	... 22 more
    hadoop    | Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.data.JsonSerDe not found
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:328)
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.MapOperator.setChildren(MapOperator.java:420)
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.configure(ExecMapper.java:106)
    hadoop    | 	... 22 more
    hadoop    | Caused by: java.lang.ClassNotFoundException: Class org.apache.hive.hcatalog.data.JsonSerDe not found
    hadoop    | 	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2299)
    hadoop    | 	at org.apache.hadoop.hive.ql.plan.PartitionDesc.getDeserializer(PartitionDesc.java:177)
    hadoop    | 	at org.apache.hadoop.hive.ql.exec.MapOperator.getConvertedOI(MapOperator.java:295)
    hadoop    | 	... 24 more
    hadoop    | 
    hadoop    | 
    hadoop    | FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
    hadoop    | MapReduce Jobs Launched: 
    hadoop    | Stage-Stage-1: Map: 1  Reduce: 1   HDFS Read: 0 HDFS Write: 0 FAIL
    hadoop    | Total MapReduce CPU Time Spent: 0 msec
    

    解决办法:

    方法一:添加add hive-hcatalog-core包

    ADD JAR /usr/local/hive/lib/hive-hcatalog-core-2.3.2.jar;

    方法二:Jar放入${HIVE_HOME}/auxlib目录(这个方法解决了我的问题)

    在${HIVE_HOME}建立一个auxlib目录,把hive-hcatalog-core.xxx.jar放到auxlib目录,重启hiverserver2,成功显示OK

  • 相关阅读:
    tensorflow搭建神经网络基本流程
    为什么Linux 实例执行 df 和 du 查看磁盘时结果不一致
    MySQL高可用架构之MHA
    Postgresql 用户管理
    PXE+Kickstart无人值守安装CentOS 7
    linux shell实用常用命令
    常用压缩命令
    mongodb 备份还原
    mongodb 备份脚本
    mongodb 日志清理
  • 原文地址:https://www.cnblogs.com/lfm601508022/p/9188819.html
Copyright © 2020-2023  润新知