• 使用Hive-JDBC遇到的一些问题解决


    使用Hive-JDBC遇到的一些问题解决(转)

    标签(空格分隔): Hive


    1,java.lang.NoClassDefFoundError: org/apache/hive/service/cli/thrift/TCLIService$Iface

    java.lang.NoClassDefFoundError: org/apache/hive/service/cli/thrift/TCLIService$Iface
    	at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
    	at java.sql.DriverManager.getConnection(DriverManager.java:664)
    	at java.sql.DriverManager.getConnection(DriverManager.java:208)
    	at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriverManager(DriverManagerDataSource.java:116)
    	at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:107)
    	at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:191)
    	at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:161)
    	at com.bonc.vbap.data.query.jdbc.ConnectionManager.getConnection(ConnectionManager.java:63)
    	at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext$SchemaHelper.getConnection(JdbcSchemaContext.java:308)
    	at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext$SchemaHelper.<init>(JdbcSchemaContext.java:112)
    	at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext.getSchemaHelper(JdbcSchemaContext.java:68)
    	at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext.getSchemaNames(JdbcSchemaContext.java:43)
    	at com.bonc.vbap.data.service.impl.SchemaServiceImpl.getSchemaNames(SchemaServiceImpl.java:34)
    	at com.bonc.vbap.data.schema.jdbc.TestJdbcColumn.testHiveDataQuery(TestJdbcColumn.java:252)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:52)
    	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    	at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75)
    	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    	at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86)
    	at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84)
    	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:252)
    	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:94)
    	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    	at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
    	at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
    	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191)
    	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
    	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
    	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
    Caused by: java.lang.ClassNotFoundException: org.apache.hive.service.cli.thrift.TCLIService$Iface
    	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    	... 44 more
    

    解决:
    所以去看下引入的依赖包,发现有hive-service和hive-service-rpc两个包中有相同的源码,所以在这里会抛异常,我就先把这其中一个依赖从hive-jdbc中去除掉,但是发现还是抛异常,我这里用的hive-jdbc1.1版本,发现hive-service是2.1版本,所以立马把两个都去除掉,重新引入一个hive-service进来就好拉

    <dependency>
      <groupId>org.apache.hive</groupId>
      <artifactId>hive-jdbc</artifactId>
      <exclusions>
        <exclusion>
          <groupId>org.apache.hive</groupId>
          <artifactId>hive-service-rpc</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.apache.hive</groupId>
          <artifactId>hive-service</artifactId>
        </exclusion>
      </exclusions>
      <version>1.1.0</version>
    </dependency>
    <dependency>
      <groupId>org.apache.hive</groupId>
      <artifactId>hive-service</artifactId>
      <version>1.1.0</version>
    </dependency>
    
    

    2,上述的问题解决之后,又出现了一个新的问题:java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration

    异常详情:

    
    java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
    	at org.apache.hive.jdbc.HiveConnection.createBinaryTransport(HiveConnection.java:393)
    	at org.apache.hive.jdbc.HiveConnection.openTransport(HiveConnection.java:187)
    	at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:163)
    	at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:105)
    	at java.sql.DriverManager.getConnection(DriverManager.java:664)
    	at java.sql.DriverManager.getConnection(DriverManager.java:208)
    	at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriverManager(DriverManagerDataSource.java:116)
    	at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:107)
    	at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnectionFromDriver(DriverManagerDataSource.java:191)
    	at com.bonc.vbap.data.query.jdbc.spring.datasource.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:161)
    	at com.bonc.vbap.data.query.jdbc.ConnectionManager.getConnection(ConnectionManager.java:63)
    	at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext$SchemaHelper.getConnection(JdbcSchemaContext.java:308)
    	at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext$SchemaHelper.<init>(JdbcSchemaContext.java:112)
    	at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext.getSchemaHelper(JdbcSchemaContext.java:68)
    	at com.bonc.vbap.data.schema.jdbc.JdbcSchemaContext.getSchemaNames(JdbcSchemaContext.java:43)
    	at com.bonc.vbap.data.service.impl.SchemaServiceImpl.getSchemaNames(SchemaServiceImpl.java:34)
    	at com.bonc.vbap.data.schema.jdbc.TestJdbcColumn.testHiveDataQuery(TestJdbcColumn.java:252)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:52)
    	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    	at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:75)
    	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
    	at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86)
    	at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84)
    	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:252)
    	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:94)
    	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    	at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
    	at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
    	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    	at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191)
    	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86)
    	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678)
    	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
    Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration
    	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    
    

    引入Hadoop-commom 包

    
    <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-common</artifactId>
      <version>2.6.5</version>
    </dependency>
    

    3. 在使用DatabaseMetaData获取元数据信息时,抛出异常:

    "Method not supported"

    这是因为HiveDatabaseMetaData在实现DatabaseMetaData接口时,没有做相应方法的实现,这里我使用到的是一个getUserName接口:

      ...
      public String getUserName() throws SQLException {
        throw new SQLException("Method not supported");
      }
    
    
    

    可以看到在HiveDatabaseMetaData中没有对getUserName支持,所以会抛异常,getUserName方法是获取到当前HiveConnection连接的用户名(Schema名称),但是这里不支持,那就得另外想办法,这里可以调用HiveConnection中的一个getConnParams方法得到连接参数,再调用getSchema方法获取到Schema名称。代码:

    
    HiveDatabaseMetaData hiveMetaData = (HiveDatabaseMetaData) metadata;
    HiveConnection hiveConn = (HiveConnection) hiveMetaData.getConnection();
    String hiveSchema = hiveConn.getSchema();
    
    
    

    原文:https://blog.csdn.net/u010429286/article/details/78780145

  • 相关阅读:
    C# 全局变量
    [C#]续:利用键代码自动转换生成字母键或其它键信息
    [WPF](小结2)DataGrid嵌套DataGrid(也叫主从表)
    [C#]winform窗口托盘
    C# arrayList动态添加对象元素,并取出对象元素的方法
    [WPF](小结3)DataGridInTreeView树嵌表
    [WPF](小结4)TreeView的数据分层模板
    [WPF](小结1)ListBox嵌套ListBox
    [C#]利用键代码自动转换生成字母键或其它键信息
    [C#]使用API 获取设置系统热键和快捷键
  • 原文地址:https://www.cnblogs.com/hit-zb/p/10972413.html
Copyright © 2020-2023  润新知