• (一) storm的集群安装与配置


    机器:

    192.168.180.101

    192.168.187.16

    需要准备的软件有:

    zookeeper(zookeeper-3.4.4.tar.gz),storm(storm-0.8.1.zip) ,jdk

    1、配置zookeeper

    解压zookeeper,将conf目录下的zoo_sample.cfg 重命名为:zoo.cfg

    修改后内容为:

    # The number of milliseconds of each tick
    tickTime=2000
    # The number of ticks that the initial 
    # synchronization phase can take
    initLimit=10
    # The number of ticks that can pass between 
    # sending a request and getting an acknowledgement
    syncLimit=5
    # the directory where the snapshot is stored.
    # do not use /tmp for storage, /tmp here is just 
    # example sakes.
    dataDir=/data/zookeeper/data
    dataLogDir=/data/zookeeper/log
    # the port at which the clients will connect
    clientPort=2181
    #
    # Be sure to read the maintenance section of the 
    # administrator guide before turning on autopurge.
    #
    # http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
    #
    # The number of snapshots to retain in dataDir
    #autopurge.snapRetainCount=3
    # Purge task interval in hours
    # Set to "0" to disable auto purge feature
    #autopurge.purgeInterval=1
    server.1=192.168.187.16:2888:3888
    server.2=192.168.180.101:2888:3888

    具体配置可以参看:
     http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_configuration

    注意,最后两行的配置:

    格式为:server.id=host:port:port

    id只能为数字 1-255,同时需要在 dataDir目录下面新建一个文件名为myid的文件,里面的内容只有一行:"id"

    Every machine that is part of the ZooKeeper ensemble should know about every other machine in the ensemble. You accomplish this with the series of lines of the form server.id=host:port:port. The parameters host and port are straightforward. You attribute the server id to each machine by creating a file named myid, one for each server, which resides in that server's data directory, as specified by the configuration file parameter dataDir.

    接着还需要添加环境变量:

    export ZOOKEEPER_HOME=/home/zhxia/apps/db/zookeeper

    两台机器上环境配置相同,但是myid文件内的id指不一样

     2、配置storm

    解压storm

    进入conf目录,编辑storm.yaml文件

    ########## These MUST be filled in for a storm configuration
     storm.zookeeper.servers:
         - "192.168.187.16"
         - "192.168.180.101"
    
     nimbus.host: "192.168.187.16"
    
     storm.local.dir: "/data/storm/data"
     ##### These may optionally be filled in:
    
    # List of custom serializations
    # topology.kryo.register:
    #     - org.mycompany.MyType
    #     - org.mycompany.MyType2: org.mycompany.MyType2Serializer
    #
    ## List of custom kryo decorators
    # topology.kryo.decorators:
    #     - org.mycompany.MyDecorator
    
    # Locations of the drpc servers
    # drpc.servers:
        # - "127.0.0.1"
         #- "server2"
    ## to nimbus 
    #nimbus.childopts: "-Xmx1024m" 
    #
    ## to supervisor 
    #supervisor.childopts: "-Xmx1024m" 
    #
    ## to worker 
    #worker.childopts: "-Xmx768m" 

    配置完成之后,开始启动zookeeper和storm

    启动zookeeper

    bin/zkServer.sh start

    启动storm

    bin/storm nimbus

    bin/storm supervisor

    bin/storm ui

    浏览器打开: http://localhost:8080 查看集群的运行状态

  • 相关阅读:
    P3368 【模板】树状数组 2
    P3374 【模板】树状数组 1
    P1631 序列合并
    P1387 最大正方形
    P1197 [JSOI2008]星球大战
    P2866 [USACO06NOV]糟糕的一天Bad Hair Day
    P1196 [NOI2002]银河英雄传说
    SP1805 HISTOGRA
    P1334 瑞瑞的木板
    2019信息学夏令营游记
  • 原文地址:https://www.cnblogs.com/xiazh/p/2827715.html
Copyright © 2020-2023  润新知