• 打包运行storm代码


    两个节点的storm集群:

    host01

    host02

    zk

     

    nimbus

    supervisor

    启动集群:

    在host01上:
    1. /usr/local/storm/bin/storm nimbus
    在host02上:
    1. /usr/local/storm/bin/storm supervisor
     
    报错:
    1. 2015-09-21T21:08:17.294+0800 o.a.s.z.ClientCnxn[WARN]Session0x0for server null, unexpected error, closing socket connection and attempting reconnect
    2. java.net.NoRouteToHostException:No route to host
    3. at sun.nio.ch.SocketChannelImpl.checkConnect(NativeMethod)~[na:1.7.0_51]
    4. at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)~[na:1.7.0_51]
    5. at org.apache.storm.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)~[storm-core-0.9.5.jar:0.9.5]
    6. at org.apache.storm.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)~[storm-core-0.9.5.jar:0.9.5]
    7. 2015-09-21T21:08:18.412+0800 o.a.s.z.ClientCnxn[INFO]Opening socket connection to server host01.zh/192.168.1.104:2181.Will not attempt to authenticate using SASL (unknown error)
    发现
    关掉zookeeper节点的防火墙解决:
    1. [root@host01 logs]# service iptables stop iptables: Setting chains to policy ACCEPT: filter [ OK ] iptables: Flushing firewall rules: [ OK ] iptables: Unloading modules: [ OK
    UI进程启动,在浏览器打不开storm UI: http://192.168.1.104:8080/index.html 
    1. [root@host01 logs]#/usr/local/storm/bin/storm list
    2. Running:/usr/local/jdk/bin/java -client -Dstorm.options=-Dstorm.home=/usr/local/storm -Dstorm.log.dir=/usr/local/storm/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file=-cp /usr/local/storm/lib/reflectasm-1.07-shaded.jar:/usr/local/storm/lib/storm-core-0.9.5.jar:/usr/local/storm/lib/jline-2.11.jar:/usr/local/storm/lib/log4j-over-slf4j-1.6.6.jar:/usr/local/storm/lib/snakeyaml-1.11.jar:/usr/local/storm/lib/logback-classic-1.0.13.jar:/usr/local/storm/lib/clj-stacktrace-0.2.2.jar:/usr/local/storm/lib/clojure-1.5.1.jar:/usr/local/storm/lib/logback-core-1.0.13.jar:/usr/local/storm/lib/tools.macro-0.1.0.jar:/usr/local/storm/lib/jetty-6.1.26.jar:/usr/local/storm/lib/commons-logging-1.1.3.jar:/usr/local/storm/lib/jgrapht-core-0.9.0.jar:/usr/local/storm/lib/joda-time-2.0.jar:/usr/local/storm/lib/minlog-1.2.jar:/usr/local/storm/lib/ring-jetty-adapter-0.3.11.jar:/usr/local/storm/lib/clj-time-0.4.1.jar:/usr/local/storm/lib/ring-core-1.1.5.jar:/usr/local/storm/lib/tools.logging-0.2.3.jar:/usr/local/storm/lib/slf4j-api-1.7.5.jar:/usr/local/storm/lib/chill-java-0.3.5.jar:/usr/local/storm/lib/commons-codec-1.6.jar:/usr/local/storm/lib/ring-devel-0.3.11.jar:/usr/local/storm/lib/tools.cli-0.2.4.jar:/usr/local/storm/lib/ring-servlet-0.3.11.jar:/usr/local/storm/lib/math.numeric-tower-0.0.1.jar:/usr/local/storm/lib/kryo-2.21.jar:/usr/local/storm/lib/jetty-util-6.1.26.jar:/usr/local/storm/lib/hiccup-0.3.6.jar:/usr/local/storm/lib/commons-io-2.4.jar:/usr/local/storm/lib/servlet-api-2.5.jar:/usr/local/storm/lib/carbonite-1.4.0.jar:/usr/local/storm/lib/clout-1.0.1.jar:/usr/local/storm/lib/json-simple-1.1.jar:/usr/local/storm/lib/asm-4.0.jar:/usr/local/storm/lib/objenesis-1.2.jar:/usr/local/storm/lib/compojure-1.1.3.jar:/usr/local/storm/lib/commons-lang-2.5.jar:/usr/local/storm/lib/commons-exec-1.1.jar:/usr/local/storm/lib/disruptor-2.10.1.jar:/usr/local/storm/lib/core.incubator-0.1.0.jar:/usr/local/storm/lib/commons-fileupload-1.2.1.jar:/usr/local/storm/conf:/usr/local/storm/bin backtype.storm.command.list
    3. 2535[main] INFO backtype.storm.thrift -Connecting to Nimbus at host01.zh:6627
    4. Topology_nameStatusNum_tasksNum_workersUptime_secs
    5. -------------------------------------------------------------------
    6. log-topology ACTIVE 00656
     
     

    打包提交代码

    eclipse项目目录下:

    命令行中执行maven打包命令:

     

    1. cd E:git_local_repoigdataStormXB
    2. mvn package
     

    上传到storm集群中:

    运行:

    1. /usr/local/storm/bin/storm jar StormXB-0.0.1-SNAPSHOT.jar ali.storm.action.WebLogStatToplgy
     

     

     提交任务后,在supervisor上会多出:work进程

    启动UI

     

    1. [root@host01 logs]#/usr/local/storm/bin/storm ui >/dev/null 2>&1&
    2. [1]2729
    3. [root@host01 logs]# jps
    4. 2729 core
    5. 2769Jps
    6. 1376QuorumPeerMain
    7. 2376 nimbus
     

    Storm UI

    http://192.168.1.104:8080/index.html

     

     

    对任务进行操作:杀死 激活等

     





  • 相关阅读:
    虚拟机备份的五大常见错误及规避方法
    交换机的link-dependency链路依赖功能
    IntelliJ Idea 常用快捷键列表
    Spark-1.X编译构建及配置安装
    maven快速下载jar镜像
    hive权威指南<一>
    vsftpd安装和使用 Linux系统和window系统
    git安装和使用 linux系统和window系统
    cdh版本的hue安装配置部署以及集成hadoop hbase hive mysql等权威指南
    cdh版本的sqoop安装以及配置
  • 原文地址:https://www.cnblogs.com/xfly/p/4827600.html
Copyright © 2020-2023  润新知