• hive实现update和delete功能


    如果一个表要实现update和delete功能,该表就必须支持ACID,而支持ACID,就必须满足以下条件:
    1、表的存储格式必须是ORC(STORED AS ORC);
    2、表必须进行分桶(CLUSTERED BY (col_name, col_name, ...)  INTO num_buckets BUCKETS);
    3、Table property中参数transactional必须设定为True(tblproperties('transactional'='true'));
    4、以下配置项必须被设定:
    Hive->配置->类别->高级
    Client端:hive-site.xml 的 Hive 客户端高级配置代码段(安全阀)
    hive.support.concurrency – true
    hive.enforce.bucketing – true
    hive.exec.dynamic.partition.mode – nonstrict  
    hive.txn.manager – org.apache.hadoop.hive.ql.lockmgr.DbTxnManager  
    服务端:hive-site.xml 的 Hive 服务高级配置代码段(安全阀)
    hive.compactor.initiator.on – true
    hive.compactor.worker.threads – 1
    hive.txn.manager – org.apache.hadoop.hive.ql.lockmgr.DbTxnManager

    hive>set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;

    创建表
    create table t1(id int, name string)
    clustered by (id) into 8 buckets
    stored as orc TBLPROPERTIES ('transactional'='true');

    create table test_trancaction
    (user_id Int,name String)
    clustered by (user_id) into 3 buckets
    stored as orc TBLPROPERTIES ('transactional'='true');

    hive> create table test_insert_test(id int,name string) row format delimited fields TERMINATED BY ','; ---临时表
    hive> LOAD DATA LOCAL INPATH '/data/test.txt' OVERWRITE INTO TABLE test_insert_test;
    hive> select * from test_insert_test;
    OK
    1,jerrick
    2,tom
    3,jerry
    4,lily
    5,hanmei
    6,limlei
    7,lucky

    hive>insert into test_trancaction select * from test_insert_test;
    FAILED: SemanticException [Error 10265]: This command is not allowed on an ACID table merge_data.transactions with a non-ACID transaction manager. Failed

    hive>set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
    hive>update test_trancaction set name='ccx' where user_id=2;

  • 相关阅读:
    html5+css3中的background: -moz-linear-gradient 用法 (转载)
    CentOS 安装Apache服务
    Linux 笔记
    CURL 笔记
    Spring Application Context文件没有提示功能解决方法
    LeetCode 389. Find the Difference
    LeetCode 104. Maximum Depth of Binary Tree
    LeetCode 520. Detect Capital
    LeetCode 448. Find All Numbers Disappeared in an Array
    LeetCode 136. Single Number
  • 原文地址:https://www.cnblogs.com/zsfishman/p/12423042.html
Copyright © 2020-2023  润新知