• Hbase0.98.0完全分布式搭建---【使用外部zookeeper】


    Hbase是一个分布式的实时数据库,他可以基于hadoop的hdfs,S3等分布式存储系统。而且使用zookeeper来通信(查询元数据和获取数据所在位置等功能)
    本文的Hbase使用的是hadoop的hdfs和外部的zookeeper。在这里假设你已经搭建好hadoop和zookeeper。
     
    Hbase搭建过程:
    1.修改hbase-env.xml文件
    export JAVA_HOME=/home/liangjf/app/jdk1.8.0_144     #java安装的根目录
    export HBASE_MANAGES_ZK=false    #false-使用非hbase自带的zookeeper。true-使用hbase自带的zookeeper
     
    2.修改hbase-size.xml文件
    <configuration>
        <property>
            <name>hbase.rootdir</name>
            <value>hdfs://master:9000/hbase</value>
        </property>
        <property>
            <name>hbase.zookeeper.property.clientPort</name>
            <value>2181</value>
        </property>
        <property>
            <name>hbase.cluster.distributed</name>
            <value>true</value>    #指明使用完全分布式   false-单机或伪分布式
        </property>
        <property>     
            <name>zookeeper.znode.parent</name>           
            <value>/hbase</value>    #注意点1          
            </property>
        <property>
            <name>hbase.master</name>
            <value>hdfs://master:60000</value>
        </property>
        <property>
            <name>hbase.zookeeper.quorum</name>
            <value>master,slave1</value>
        </property>
        <property>
            <name>dfs.datanode.max.xcievers</name>
            <value>4096</value>
        </property>
    </configuration>
    注意点1:hbase是使用zookeeper来完成其功能的。所以肯定是在zookeeper上创建一个节点来存放其必须的信息。因此在启动hbase客户端的之前,必须在zookeeper上创建一个目录来保存数据。这里是/hbase(实际上这里的名字是随意的,不过为了方便使用了/hbase)。
    步骤:
    进入zookeeper/bin,执行命令 
    [liangjf@master bin]$./zkCli.sh
    [zk: localhost:2181(CONNECTED) 0] ls /
    [zookeeper]
    [zk: localhost:2181(CONNECTED) 0] create /hbase "123"
    [zk: localhost:2181(CONNECTED) 0] ls /
    [zookeeper, hbase]
    如果缺少在zookeeper上创建 /hbase这一步( create /hbase "123" ),在使用的时候会出现以下错误。在这里吐槽一下,网上出现这个错误的解决办法真少。
    hbase(main):001:0> list
    TABLE                                                                                                                                                     
    SLF4J: Class path contains multiple SLF4J bindings.
    SLF4J: Found binding in [jar:file:/home/liangjf/app/hbase-0.98.0-hadoop1/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: Found binding in [jar:file:/home/liangjf/app/hadoop-1.2.1/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
    SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
    2017-11-18 10:07:38,240 ERROR [main] client.HConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
    2017-11-18 10:07:38,344 ERROR [main] client.HConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
    2017-11-18 10:07:38,553 ERROR [main] client.HConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
    2017-11-18 10:07:38,863 ERROR [main] client.HConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
    2017-11-18 10:07:39,369 ERROR [main] client.HConnectionManager$HConnectionImplementation: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master
     
    3.分发整个hbase目录到各节点
    scp -r /home/liangjf/app/hbase-0.98.0-hadoop1 liangjf@slave1:/home/liangjf/app
    

     

    4.启动hbase
    进入 /hbase-0.98.0-hadoop1/bin/ 下, ./start-hbase.sh 
     
    5.启动hbase客户端
    进入 /hbase-0.98.0-hadoop1/bin/ 下, 执行命令 hbase shell ,进入shell客户端。
     
    6.测试
    hbase(main):006:0> create 'member','address','info'
    0 row(s) in 0.7770 seconds
    => Hbase::Table - member
     
    hbase(main):009:0> list
    TABLE                                                                                                                                                     
    member                                                                                                                                                    
    1 row(s) in 0.0410 seconds
    => ["member"]
     
    hbase(main):015:0> describe 'member'
    DESCRIPTION                                                                                         ENABLED                                               
    'member', {NAME => 'address', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DE true                                                  
    LETED_CELLS => 'false', DATA_BLOCK_ENCODING => 'NONE', TTL => '2147483647', COMPRESSION => 'NONE',                                                       
      MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME                                                       
      => 'info', BLOOMFILTER => 'ROW', VERSIONS => '1', IN_MEMORY => 'false', KEEP_DELETED_CELLS => 'fa                                                       
    lse', DATA_BLOCK_ENCODING => 'NONE', TTL => '2147483647', COMPRESSION => 'NONE', MIN_VERSIONS => '                                                       
    0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}                                                                                
    1 row(s) in 0.0560 seconds
     
    hbase(main):022:0> put'member','zhangsan','info:age','24'
    0 row(s) in 0.1560 seconds
    hbase(main):023:0> put'member','lisi','info:birthday','1994-01-01'
    0 row(s) in 0.0120 seconds
     
    hbase(main):038:0> scan 'member'
    ROW                                     COLUMN+CELL                                                                                                                                                 
    zhangsan                                column=info:age, timestamp=1511029691288, value=24                                                               
    lisi                                    column=info:birthday, timestamp=1511029538058, value=1994-01-01   
     
  • 相关阅读:
    LeetCode Best Time to Buy and Sell Stock
    LeetCode Scramble String
    LeetCode Search in Rotated Sorted Array II
    LeetCode Gas Station
    LeetCode Insertion Sort List
    LeetCode Maximal Rectangle
    Oracle procedure
    浏览器下载代码
    Shell check IP
    KVM- 存储池配置
  • 原文地址:https://www.cnblogs.com/liangjf/p/7857707.html
Copyright © 2020-2023  润新知