• 「hive」hive2.3.0配置derby


    【问题现象】

    hive默认自带derby数据库,但安装hive2.3.0后启动hive无法使用derby操作,遇到各种问题,折腾好一阵子,经测试已ok

    【先决条件】

    ubuntu已安装hadoop2.8.1

    【问题解决】

    1、官网下载hive版本2.3.0,在ubuntu中并安装;

    2、增加HIVE_HOME到/etc/profile,并$source /etc/profile 使得变量立刻生效;

    3、生成默认配置文件和自定义配置文件(重复项时自定义配置会覆盖默认配置),定位到hive根目录,

    $cp conf/hive-default.xml.template conf/hive-default.xml

    $cp conf/hive-default.xml conf/hive-site.xml

    4、hive-site.xml中去掉hive.metastore.local属性

    <!-- 
    <property>
      <name>hive.metastore.local</name>
      <value>true</value>
    </property>
    -->
    用于消除 HiveConf of name hive.metastore.local does not exist 错误

    5、设置hive.metastore.schema.verification为false,

    解决Version information not found in metastore错误。

    6、设置hive.druid.metadata.db.type为derby

    参考如下mysql配置

     1 <?xml version="1.0" encoding="UTF-8" standalone="no"?>
     2 <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
     3 <!--
     4    Licensed to the Apache Software Foundation (ASF) under one or more
     5    contributor license agreements.  See the NOTICE file distributed with
     6    this work for additional information regarding copyright ownership.
     7    The ASF licenses this file to You under the Apache License, Version 2.0
     8    (the "License"); you may not use this file except in compliance with
     9    the License.  You may obtain a copy of the License at
    10 
    11        http://www.apache.org/licenses/LICENSE-2.0
    12 
    13    Unless required by applicable law or agreed to in writing, software
    14    distributed under the License is distributed on an "AS IS" BASIS,
    15    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    16    See the License for the specific language governing permissions and
    17    limitations under the License.
    18 -->
    19 <configuration>
    20   <!-- WARNING!!! This file is auto generated for documentation purposes ONLY! -->
    21   <!-- WARNING!!! Any changes you make to this file will be ignored by Hive.   -->
    22   <!-- WARNING!!! You must make your changes in hive-site.xml instead.         -->
    23   <!-- Hive Execution Parameters -->
    24         <property>
    25             <name>javax.jdo.option.ConnectionURL</name>
    26             <value>jdbc:mysql://10.0.10.246:3306/hive?createDatabaseIfNotExist=true&amp;useUnicode=true&amp;characterEncoding=UTF-8&amp;useSSL=false</value>
    27              <description>数据库链接地址</description>
    28         </property>
    29         <property>
    30             <name>javax.jdo.option.ConnectionDriverName</name>
    31             <value>com.mysql.jdbc.Driver</value>
    32             <description>数据库驱动类</description>
    33         </property>
    34         <property>
    35             <name>javax.jdo.option.ConnectionUserName</name>
    36             <value>chendajian</value>
    37             <description>数据库用户名</description>
    38         </property>
    39         <property>
    40             <name>javax.jdo.option.ConnectionPassword</name>
    41             <value>123456</value>
    42             <description>数据库密码</description>
    43         </property>
    44         <property>
    45             <name>hive.metastore.warehouse.dir</name>
    46             <value>/usr/apps/hive/warehouse</value>
    47            <description>数据相对路径</description>
    48         </property>
    49         <property>
    50             <name>hive.exec.scratdir</name>
    51             <value>/usr/apps/hive/warehouse</value>
    52         <description>数据临时路径</description>
    53         </property>
    54 </configuration>
    View Code

    7、数据库相关初始化

    定位到hive根目录,$schematool -dbType derby -initSchema

    解决Required table missing : "DBS" in Catalog "" Schema "". DataNucleus requires this table to perform its persistence operations. Either your MetaData is incorrect, or you need to enable "datanucleus.schema.autoCreateTables错误

    8、$hive 即可进入hive shell操作啦。

    【其它问题】

    问题:Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStor

    配置有问题

    问题:Duplicate key name 'PCS_STATS_IDX'

    这个错误是因为,执行schematool -initSchema -dbType mysql的时候,hivedb数据库表里已经存在表了,所以在库里吧所有的表都删除了,再执行命令。

    问题:警告:You need either to explicitly disable SSL by setting useSSL=false, or set us

    连接时关闭ssl,

    <property>
                <name>javax.jdo.option.ConnectionURL</name>
                <value>jdbc:mysql://10.0.10.246:3306/hive?createDatabaseIfNotExist=true&amp;useUnicode=true&amp;characterEncoding=UTF-8&amp;useSSL=false</value>
                 <description>数据库链接地址</description>
            </property>

  • 相关阅读:
    HTML5开发 桌面提醒功能
    自然框架总体设计
    Inversion of Logging
    Wind.js在移动跨平台框架PhoneGap中的异步体验
    MSBuild基本概念
    PortalBasic Web 应用开发框架:应用篇
    检测常见ASP.NET配置安全漏洞
    线程与进程的比较
    多层架构下的一个进销存项目
    Model Driven Architecture
  • 原文地址:https://www.cnblogs.com/ftrako/p/7642572.html
Copyright © 2020-2023  润新知