• Hadoop体系结构管理


    一.查看Zookeeper信息

    [hadoop@weekend01 ~]$zkServer.sh status

    JMX enabled by default

    Using config: /hadoop/zookeeper-3.4.6/bin/../conf/zoo.cfg

    Mode: leader

    [hadoop@weekend02 ~]$zkServer.sh status

    JMX enabled by default

    Using config: /hadoop/zookeeper-3.4.6/bin/../conf/zoo.cfg

    Mode: follower

    [hadoop@weekend03 ~]$ zkServer.sh status

    JMX enabled by default

    Using config: /hadoop/zookeeper-3.4.6/bin/../conf/zoo.cfg

    Mode: follower

    二.命名空间的操作

    1.创建myns1命名空间

    hbase(main):002:0> create_namespace 'myns1'

    0 row(s) in 0.5840 seconds

    2.在myns1命名空间上创建表mytable,并查看该命名空间中的表信息

    hbase(main):003:0> create 'myns1','mytable'

    0 row(s) in 0.3140 seconds

    => Hbase::Table - myns1

    3.删除myns1命名空间

    hbase(main):004:0> drop_namespace 'myns1'

    0 row(s) in 0.1320 seconds

    三.查看meta系统表信息

    创建nyist表,并观察meta系统表前后的信息变化

    创建前

    hbase(main):005:0> scan 'hbase:meta'

    ROW COLUMN+CELL

    hbase:namespace,,146 column=info:regioninfo, timestamp=1463478332970, value={EN

    3478330523.c515e6022 CODED => c515e602219215debe5c20797ef49598, NAME => 'hbase:

    19215debe5c20797ef49 namespace,,1463478330523.c515e602219215debe5c20797ef49598.

    598. ', STARTKEY => '', ENDKEY => ''} 。。。。。。。内容太多以下省略

    创建表nyist

    hbase(main):006:0> create 'nyist','info'

    0 row(s) in 0.3900 seconds

    => Hbase::Table – nyist

    创建后

    hbase(main):007:0> scan 'hbase:meta'

    。。。。。

    nyist,,1464873740887 column=info:regioninfo, timestamp=1464873741163, value={EN

    .743fd741807411c415e CODED => 743fd741807411c415e8186523994416, NAME => 'nyist,

    8186523994416. ,1464873740887.743fd741807411c415e8186523994416.', STARTKE

    Y => '', ENDKEY => ''}

    nyist,,1464873740887 column=info:seqnumDuringOpen, timestamp=1464873741181, val

    .743fd741807411c415e ue=x00x00x00x00x00x00x00x02

    8186523994416.

    nyist,,1464873740887 column=info:server, timestamp=1464873741181, value=slave2:

    .743fd741807411c415e 16020

    8186523994416.

    nyist,,1464873740887 column=info:serverstartcode, timestamp=1464873741181, valu

    .743fd741807411c415e e=1464869626765

    8186523994416. 。。。。。

    这是创建后多出来的信息。

    四.查看HBase HDFS上目录,并给出HBase有哪些目录

    第一种方法是用命令查看

    [root@master ~]# hadoop fs -ls hdfs://192.168.186.134:9000/hbase

    16/06/02 21:26:27 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

    Found 8 items

    drwxr-xr-x - root supergroup 0 2016-06-02 20:18 hdfs://192.168.134.131:9000/hbase/.tmp

    drwxr-xr-x - root supergroup 0 2016-06-02 20:14 hdfs://192.168.134.131:9000/hbase/WALs

    drwxr-xr-x - root supergroup 0 2016-06-02 21:19 hdfs://192.168.134.131:9000/hbase/archive

    drwxr-xr-x - root supergroup 0 2016-05-30 20:29 hdfs://192.168.134.131:9000/hbase/corrupt

    drwxr-xr-x - root supergroup 0 2016-06-02 21:17 hdfs://192.168.134.131:9000/hbase/data

    -rw-r--r-- 2 root supergroup 42 2016-05-30 20:29 hdfs://192.168.134.131:9000/hbase/hbase.id

    -rw-r--r-- 2 root supergroup 7 2016-05-30 20:29 hdfs://192.168.134.131:9000/hbase/hbase.version

    drwxr-xr-x - root supergroup 0 2016-06-02 21:18 hdfs://192.168.134.131:9000/hbase/oldWALs

    第二种使用了浏览器

    http://192.168.186.134:50070

    五、flush、compact、split操作

    创建学生表'student',列族为'info'

    hbase(main):013:0> create 'student','info'

    0 row(s) in 0.3920 seconds

    => Hbase::Table - student

    插入数据'info:name','lisi1'

    hbase(main):019:0> put 'student','rk0001','info:name','lisi1'

    0 row(s) in 0.1780 seconds

    使用flush写入磁盘

    hbase(main):020:0> flush 'student'

    0 row(s) in 0.4990 seconds

    插入数据'info:name','lisi2'

    hbase(main):021:0> put 'student','rk0002','info:name','lisi2'

    0 row(s) in 0.0070 seconds

    使用flush写入磁盘

    hbase(main):022:0> flush 'student'

    0 row(s) in 0.2870 seconds

    使用compact合并磁盘文件

    hbase(main):023:0> compact 'student'

    0 row(s) in 0.1340 seconds

    使用split分离region

    hbase(main):024:0> split 'region'

    0 row(s) in 0.0670 seconds

    关闭表'student'的region

    hbase(main):036:0>close_region 'student,,1463730257094.ed1902898fb90cae03772240a505cf75.'

    0 row(s) in 0.2370 seconds

    查看表'student'数据

    hbase(main):037:0> scan 'student'

    ROW COLUMN+CELL

    rk0001 column=info:name, timestamp=1464874655412, value=lisi1

    rk0002 column=info:name, timestamp=1464874756191, value=lisi2

    2 row(s) in 0.3200 seconds

    启动region

    hbase(main):038:0>assign 'student,,1463730257094.ed1902898fb90cae03772240a505cf75.'

    0 row(s) in 0.1980 seconds

    再查看表'student'数据

    hbase(main):039:0> scan 'student'

    ROW COLUMN+CELL

    rk0001 column=info:name, timestamp=1464874655412, value=lisi1

    rk0002 column=info:name, timestamp=1464874756191, value=lisi2

    2 row(s) in 0.3200 seconds

    删除表'student',观察hbase目录中是否有表'student'的文件信息

    hbase(main):002:0> disable 'student'

    0 row(s) in 1.3360 seconds

    hbase(main):003:0> drop 'student'

    0 row(s) in 0.2170 seconds

    hbase(main):004:0> list

    TABLE

    hbase_student

    my_data

    myns1

    new_scores

    nyist

    scores

    user

    7 row(s) in 0.0150 seconds

    => ["hbase_student", "my_data", "myns1", "new_scores", "nyist", "scores", "user"]

    本次实验全部是命令的练习,和对HBase体系结构的理解,只要勤加练习,这部分不是问题,加上之前的数据库的基础知识,接受这部分内容还是没有问题的,另外还有就是在练习过程中敲错命令是常事,感觉还是有点不习惯,对这种数据存储结构和运作流程还需要课下多阅读官方文档,自己找一些书籍,多练习下才能对这部分掌握熟练。

    help

    list_namespace --显示命名空间

    scan 'hbase:namespace' --显示命名空间

    create_namespace 'myns1' --创建命名空间

    list_namespace

    scan 'hbase:namespace'

    list_namespace_tables 'myns1' --显示命名空间中的表

    create 'myns1:mytable'

    list_namespace_tables 'myns1'

    drop_namespace 'myns1' --删除命名空间

    list_namespace

    scan 'hbase:namespace'

    四、HBase HDFS上目录

    1.查看HBase的方法

    两种:

    a.浏览器 IP或主机名:50070

    b.hadoop 命令

    cd /hadoop/hadoop-2.6.0/bin/

    ./hadoop fs -ls hdfs://192.168.1.2:9000/hbase

    2.HBase HDFS上目录分析

    a、.tmp目录:代表的hbase的临时目录,当我们在对表进行删除和创建的时候,hbase会将表先move到.tmp目录,然后,进行操作。

    b、WALs目录:预写日志文件,在0.94.x版本时叫.logs目录,0.98.x中就改版了,是RegionServer在处理数据插入和删除的过程中用来记录操作内容的一种日志

    比如:向HBas

    3.

    hbase(main):002:0> list_namespace

    NAMESPACE

    default

    hbase

    2 row(s) in 0.5070 seconds

    hbase(main):003:0> scan 'hbase:namespace'

    ROW COLUMN+CELL

    default column=info:d, timestamp=1462885868353, value=x0Ax07defa

    ult

    hbase column=info:d, timestamp=1462885868394, value=x0Ax05hbas

    e

    2 row(s) in 0.3690 seconds

    hbase(main):004:0> create_namespace 'myns1'

    0 row(s) in 0.3700 seconds

    hbase(main):005:0> list_namespace

    NAMESPACE

    default

    hbase

    myns1

    3 row(s) in 0.0510 seconds

    hbase(main):006:0> list_namespace_tables 'myns1'

    TABLE

    0 row(s) in 0.0780 seconds

    hbase(main):007:0> drop_namespace 'myns1'

    0 row(s) in 0.3120 seconds

    hbase(main):008:0> list_namespace

    NAMESPACE

    default

    hbase

    2 row(s) in 0.0320 seconds

    hbase(main):009:0>

    4.

    hbase(main):010:0> create 'nuist','info'

    0 row(s) in 2.7440 seconds

    => Hbase::Table - nuist

    hbase(main):012:0> sacn 'nuist'

    NoMethodError: undefined method `sacn' for #<Object:0x1d5ff9c>

    hbase(main):013:0> disable 'nuist'

    0 row(s) in 1.3630 seconds

    hbase(main):014:0> drop 'nuist'

    0 row(s) in 0.4870 seconds

    hbase(main):001:0> create 'student','info'

    0 row(s) in 4.6830 seconds

    => Hbase::Table - student

    hbase(main):002:0> put 'student','rk0001','info:name','lisi'

    0 row(s) in 0.2790 seconds

    hbase(main):003:0> scan 'student'

    ROW COLUMN+CELL

    rk0001 column=info:name, timestamp=1463729729329, value=lisi

    1 row(s) in 0.0580 seconds

    hbase(main):004:0> flush 'student'

    0 row(s) in 0.8910 seconds

    hbase(main):005:0> put 'student','rk0002','info:name','lisi2'

    0 row(s) in 0.0460 seconds

    hbase(main):006:0> flush 'student'

    0 row(s) in 0.3100 seconds

    hbase(main):007:0> put 'student','rk0003','info:name','lisi3'

    0 row(s) in 0.0240 seconds

    hbase(main):008:0> flush 'student'

    0 row(s) in 0.3850 seconds

    hbase(main):009:0> compact 'student'

    0 row(s) in 0.1420 seconds

    hbase(main):010:0> split 'student'

    0 row(s) in 0.0570 seconds

    hbase(main):011:0> split 'student','rook0002'

    0 row(s) in 0.0930 seconds

    hbase(main):012:0> split 'student','rk0002'

    0 row(s) in 0.1230 seconds

    hbase(main):001:0> close_region 'student,,1463730257094.ed1902898fb90cae03772240a505cf75.'

    0 row(s) in 0.2370 seconds

    hbase(main):003:0> scan 'student'

    ROW COLUMN+CELL

    ERROR: org.apache.hadoop.hbase.NotServingRegionException: Region student,,1463730257094.ed1902898fb90cae03772240a505cf75. is not online on slave2,16020,1463728132224

    hbase(main):007:0> assion 'student,,1463730257094.ed1902898fb90cae03772240a505cf75.'

    NoMethodError: undefined method `assion' for #<Object:0x161412>

    hbase(main):008:0> assign 'student,,1463730257094.ed1902898fb90cae03772240a505cf75.'

    0 row(s) in 0.1980 seconds

    hbase(main):009:0> scan 'student'

    ROW COLUMN+CELL

    rk0001 column=info:name, timestamp=1463729729329, value=lisi

    rk0002 column=info:name, timestamp=1463729915233, value=lisi2

    rk0003 column=info:name, timestamp=1463729938724, value=lisi3

    3 row(s) in 0.4290 seconds

  • 相关阅读:
    MySQL数据库初识
    Python中面向对象初识到进阶
    python 函数进阶
    Python 函数的初识
    Python的发展与应用
    什么是产品经理 以及职责
    I/O----复制文本文件
    获取次日日期(主要两种方法)
    vector以及array和数组
    编辑软件注释快捷键
  • 原文地址:https://www.cnblogs.com/zd520pyx1314/p/7246608.html
Copyright © 2020-2023  润新知