• 集群节点基于Hadoop集群的HBase集群的配置【2】


    首先声明,我是一个菜鸟。一下文章中现出技术误导情况盖不负责

        

    基于Hadoop集群的HBase集群的配置

        

    1.准备工作

        

    a.经已实现安装的Hadoop集群环境

        

    b.所须要的软件包

        zookeeper-3.4.4.tar.gz下载地址:

        http://downloads.gotsoft.com/pub/apache/zookeeper/zookeeper-3.4.4/

        hbase-0.94.6.1tar.gz下载地址:

        http://labs.mop.com/apache-mirror/hbase/stable/hbase-0.94.6.1.tar.gz

        

    2.独单安装的ZooKeeper集群,不基于Hbase集群管理

        

    a.在Master节点上解压zookeeper

        l  ##提议用使hduser权限,免避后之现出SSH错误

        hduser@master:/usr/local$sudo tar -zxvfzookeeper-3.4.4.tar.gz

        l  ##server前面的数字是zooKeeper识别时后的Id,须要与前面的置设保持一致

        hduser@master:/usr/local$tvi /zookeeper/conf/zoo.cfg

        tickTime=2000

          initLimit=5

          syncLimit=2

         dataDir=/data/zookeeper

          clientPort=2181

         server.1=master:2888:3888

         server.2=slave1:2888:3888

          server.3=slave2:2888:3888

        root@master:/usr/local# chown -Rhduser:hadoop zookeeper

        

    3.将zookeeper复制到其他呆板

        hduser@master:/usr/local$scp -r zookeeper-3.4.5hduser@slave1:/home/hduser

        hduser@master:/usr/local$ scp -rzookeeper-3.4.5 hduser@slave2:/home/hduser

        hduser@slave1:~#sudomv zookeeper-3.4.5/ /usr/local/zookeeper-3.4.5

        hduser@slave2:~$sudomv zookeeper-3.4.5/ /usr/local/zookeeper-3.4.5

        

    4.在每台呆板配置的dataDir目录上面,创立myid文件标记前当机主

        Master:     $touch /data/zookeeper/myid

                    $echo "1" >/data/zookeeper/myid

        Slave1:      $mkdir -p /data/zookeeper

                    $touch /data/zookeeper/myid

                    $echo "3" > /data/zookeeper/myid

        Slave2:       $mkdir -p/data/zookeeper

                    $touch /data/zookeeper/myid

                    $echo "5" > /data/zookeeper/myid

        

    5. 动启zookeeper集群

        hduser@master:/usr/local/zookeeper/bin$./zkServer.sh start
    JMX enabled by default
    Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
    题问1Starting zookeeper ... ./zkServer.sh: line 113:/data/zookeeper/zookeeper_server.pid: Permission denied
    FAILED TO WRITE PID

        因原是刚才/data/zookeeper是root权限,而不是hduser的,改修权限之~

        题问1处理root@master:/# chown -Rhduser:hadoop /data

        其他两个slave节点也一样~

        然后题问处理:

        hduser@master:/usr/local/zookeeper$ cd bin
    hduser@master:/usr/local/zookeeper/bin$ ./zkServer.sh start
    JMX enabled by default
    Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
    Starting zookeeper ... STARTED

        其他节点也样同动启zookeeper~
    hduser@master:/usr/local/zookeeper/bin$ 

        hduser@slave1:/usr/local/zookeeper/bin$./zkServer.sh status
    JMX enabled by default
    Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
    题问2Error contacting service. It is probably notrunning.

        


        1)查看zoopkeeper.out

        r@232] - Unexpected exception,tries=0, connecting to slave1/192.168.123.12:2888
    java.net.ConnectException: Connection refused
            atjava.net.PlainSocketImpl.socketConnect(Native Method)
            atjava.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
            atjava.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
            atjava.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
            atjava.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
            at java.net.Socket.connect(Socket.java:529)

        


        root@master:/usr/local/zookeeper/bin#cat zookeeper.out 

        2013-04-27 00:22:37,595 [myid:] - INFO [main:QuorumPeerConfig@101] - Reading configuration from:/usr/local/zookeeper/bin/../conf/zoo.cfg

        2013-04-27 00:22:37,615 [myid:] - ERROR[main:QuorumPeerMain@85] - Invalid config, exiting abnormally

        org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException:Error processing /usr/local/zookeeper/bin/../conf/zoo.cfg

            at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:121)

            atorg.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:101)

            atorg.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)

        Caused by:java.lang.IllegalArgumentException: dataLogDir /home/hduser/zookeeper/log ismissing.

            atorg.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:247)

            atorg.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:117)

            ... 2 more

        Invalid config, exiting abnormally

        


        【错误2处理】没有将zookeeper复制到其他的节点!!!

                       没有在个每节点动启zookeeper

        

    6.安装配置Hbase集群

        

    a>动启HDFS集群例实,并创立目录

        ##hdfs://master01:54310/hbase【注意与hadoop,core-site.xml中的fs.default.name中的端口号保持一致,一统置设为54310;hbase-site.xml中的hbase:rootdir中的端口号】

        hduser@master:/usr/local/hadoop$bin/hadoop fs -mkdir /hbase

        Warning: $HADOOP_HOME is deprecated.

        13/04/27 02:08:46 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 0 time(s).

        13/04/27 02:08:47 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 1 time(s).

        每日一道理
    有些冷,有些凉,心中有些无奈,我一个人走在黑夜中,有些颤抖,身体瑟缩着,新也在抖动着,我看不清前方的路,何去何从,感觉迷茫,胸口有些闷,我环视了一下周围,无人的街头显得冷清,感到整个世界都要将我放弃。脚步彷徨之间,泪早已滴下……

        13/04/27 02:08:48 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 2 time(s).

        13/04/27 02:08:49 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 3 time(s).

        13/04/27 02:08:50 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 4 time(s).

        13/04/27 02:08:51 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 5 time(s).

        13/04/27 02:08:52 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 6 time(s).

        13/04/27 02:08:53 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 7 time(s).

        13/04/27 02:08:54 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 8 time(s).

        13/04/27 02:08:55 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 9 time(s).

        Bad connection to FS. command aborted.exception: Call to master/127.0.1.1:54310 failed on connection exception:java.net.ConnectException: Connection refused

        hduser@master:/usr/local/hadoop$ 

        Call tomaster/127.0.1.1:54310 failed【改修localhost地址为127.0.0.1

         

         

        HBASE SHELL 错误NativeException: org.apache.hadoop.hbase.MasterNotRunningException:null

        【处理步调:】

        1、动启hadoop后,须要等一段时间,再开启hbase

        2、去失落hadoop的安全模式:

        hadoop dfsadmin -safemode leave

        3、把/etc/hosts里的master01的IP为改服务器前当的IP

        4、确认hbase的hbase-site.xml中

        <name>hbase.rootdir</name>

        <value>hdfs://localhost:8020/hbase</value>

        与hadoop的core-site.xml中

        <name>fs.default.name</name>

        <value>hdfs://localhost:8020</value>

        红字分部保持一致

        5、确认hadoop版本是0.20.2,hbase版本是0.20.6

        6、从新执行./start-hbase.sh之前,先kill失落前当的hbase和zookeeper程进

        后之再执行:

        $bin/hadoop fs -mkdir /hbase

        

     b> 解压hbase-0.94.5.tar.gz,改修配置文件,正式安装HBase:

        l  #vi hbase-0.94.5/conf/hbase-env.sh 

          exportJAVA_HOME=/usr/java/jdk1.6.0_43

          exportHBASE_CLASSPATH=$HBASE_HOME/

          #exportHBASE_HEAPSIZE=128

          #exportHBASE_OPTS="-XX:+UseConcMarkSweepGC"

          exportHBASE_MANAGES_ZK=false

        #vi hbase-0.94.5/conf/regionservers

        slave1

        slave2

        l  将hbase-webapps目录改名为webapps

        hduser@master:/usr/local/hbase-0.94.6$ mvhbase-webapps/ /webapps

        l  检查HBase配置,复制到其他的slave节点:

        hduser@master:/usr/local$ scp -rhbase-0.94.6 hduser@slave1:/home/hduser/

        hduser@master:/usr/local$ scp -rhbase-0.94.6 hduser@slave2:/home/hduser/

        l  动启HBase:

        hduser@master:/usr/local$ ./hbase-0.94.6/bin/start-hbase.sh
    starting master, logging to/usr/local/hbase-0.94.6/bin/../logs/hbase-hduser-master-master.out
    slave2: starting regionserver, logging to/usr/local/hbase-0.94.6/bin/../logs/hbase-hduser-regionserver-slave2.out
    slave1: starting regionserver, logging to/usr/local/hbase-0.94.6/bin/../logs/hbase-hduser-regionserver-slave1.out

        l  查看关相程进:

        hduser@master:/usr/local$ jps

        3339 QuorumPeerMain

        8210 JobTracker

        7894 DataNode

        9187 HMaster

        8126 SecondaryNameNode

        8459 TaskTracker

        9328 Jps

        7659 NameNode

        hduser@master:/usr/local$

        hduser@slave1:~$ jps

        5416 TaskTracker

        5879 Jps

        5210 DataNode

        3323 QuorumPeerMain

        hduser@slave1:~$

     hduser@slave2:/usr/local/hbase-0.94.6$ jps

        5083 TaskTracker

        5544 Jps

        4877 DataNode

        3042 QuorumPeerMain

        【错误】13/04/27 05:47:21ERROR client.HConnectionManager$HConnectionImplementation: Check the valueconfigured in 'zookeeper.znode.parent'. There could be a mismatch with the oneconfigured in the master.

        


        l  去失落hadoop的安全模式

        hduser@master:/usr/local/hadoop$ bin/hadoopdfsadmin -safemode leave
    Warning: $HADOOP_HOME is deprecated.

    Safe mode is OFF

        【处理方法】:http://blog.sina.com.cn/s/blog_718335510100zchp.html

        l  检修HBase安装实现:

        hbase(main):003:0> create'member','member_id','address','info'
    0 row(s) in 1.6400 seconds

        
    hbase(main):004:0> list
    TABLE                                                                          
    member                                                                         
    1 row(s) in 0.0730 seconds

        hbase(main):006:0* describe 'member'
    DESCRIPTION                                         ENABLED                   
    {NAME => 'member', FAMILIES => [{NAME => 'address',true                      
      DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>'NON                           
    E', REPLICATION_SCOPE => '0', VERSIONS => '3', COMP                           
    RESSION => 'NONE', MIN_VERSIONS => '0', TTL =>'214                           
    7483647', KEEP_DELETED_CELLS => 'false',BLOCKSIZE                            
    => '65536', IN_MEMORY => 'false', ENCODE_ON_DISK=>                           
      'true', BLOCKCACHE => 'true'}, {NAME => 'info',DA                           
    TA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>'NONE',                           
      REPLICATION_SCOPE => '0', VERSIONS => '3',COMPRES                           
    SION => 'NONE', MIN_VERSIONS => '0', TTL =>'214748                           
    3647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE=>                            
    '65536', IN_MEMORY => 'false', ENCODE_ON_DISK =>'t                           
    rue', BLOCKCACHE => 'true'}, {NAME =>'member_id',                            
    DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>'NONE                           
    ', REPLICATION_SCOPE => '0', VERSIONS => '3',COMPR                           
    ESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147                           
    483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE=                           
    > '65536', IN_MEMORY => 'false', ENCODE_ON_DISK=>                            
    'true', BLOCKCACHE =>'true'}]}                                               
    1 row(s) in 0.2170 seconds

    文章结束给大家分享下程序员的一些笑话语录: 3G普不普及现在已经不是看终端了,而是看应用,有好的,便宜实用的应用,花1000多买个能用的智能手机应该不是什么难事。反过来说,你200元拿一个智能手机,没有好的应用,看个电影要几十元,也是没人用3G。

  • 相关阅读:
    python的复制,深拷贝和浅拷贝的区别(转)
    linux下ffmpeg安装(转)
    Linux下的tar压缩解压缩命令详解(转)
    centos7安装python-pip(转)
    爬山算法和模拟退火算法简介
    协方差、协方差矩阵定义与计算
    七种常见阈值分割代码(Otsu、最大熵、迭代法、自适应阀值、手动、迭代法、基本全局阈值法)
    Canny边缘检测算法原理及其VC实现详解(二)
    Canny边缘检测算法原理及其VC实现详解(一)
    perforce变量配置与使用
  • 原文地址:https://www.cnblogs.com/jiangu66/p/3049978.html
Copyright © 2020-2023  润新知