• storm入门demo


    一.storm入门demo的介绍

           storm的入门helloworld有2种方式,一种是本地的,另一种是远程。

      本地实现:

        本地写好demo之后,不用搭建storm集群,下载storm的相关jar包即可实现storm的相关操作

      远程实现:

        本地写好demo之后,需要将其打成jar包,然后通过nimbus将jar包运行即可

        本地打包注意事项:

          由于打好的jar包会将其放到storm的集群上,因此storm上已经包含了运行的相关环境,但是在通过maven打包时需要storm-core设置成provided范围,不需要将storm-core的相关类打进jar包,以避免引起冲突

    二.本地demo的实现

      下载storm 所需jar包storm-core

    1.spout数据源的实现


    public
    class RandomStringSpout extends BaseRichSpout{ private final static Map<Integer,String> map = new HashMap<Integer, String>(); private SpoutOutputCollector collector; public RandomStringSpout(){ map.put(0, "kafka"); map.put(1, "nifi"); map.put(2, "flink"); map.put(3, "storm"); map.put(4, "spark"); } //在Spout组件初始化时被调用 public void open(Map arg0, TopologyContext topologyContextrg1, SpoutOutputCollector spoutOutputCollector) { System.err.println(" ============== open"); this.collector = spoutOutputCollector; } //nextTuple()方法是Spout实现的核心。 //也就是主要执行方法,用于输出信息,通过collector.emit方法发射 public void nextTuple() { //发送数据 collector.emit(new Values(map.get(ThreadLocalRandom.current().nextInt(4)))); try { TimeUnit.SECONDS.sleep(5); } catch (InterruptedException e) { e.printStackTrace(); } } //用于声明数据格式,即输出的一个Tuple中,包含几个字段 public void declareOutputFields(OutputFieldsDeclarer declarer) { declarer.declare(new Fields("stream")); } }

    2.Bolt数据过滤

    public class WrapStarBolt extends BaseBasicBolt{
    
        public void execute(Tuple tuple, BasicOutputCollector Collector) {
            String value = tuple.getStringByField("stream");
            System.err.println("******"+value);
        }
    
        public void declareOutputFields(OutputFieldsDeclarer declarer) {
             //nothing to do 
        }
    }
    public class WrapWellBolt extends BaseBasicBolt{
    
        public void execute(Tuple tuple, BasicOutputCollector collector) {
            String value = tuple.getStringByField("stream");
            System.err.println("#######"+value);
        }
    
        public void declareOutputFields(OutputFieldsDeclarer arg0) {
            //nothing to do 
        }
    }

    3.创建topology

    //所有的spout bolt 会组成一个topology
    public
    class RadomStringTopologyLocal { public static void main(String[] args) throws InterruptedException { TopologyBuilder builder = new TopologyBuilder(); builder.setSpout("RandomStringSpout", new RandomStringSpout()); builder.setBolt("wrapStarBolt", new WrapStarBolt()).shuffleGrouping("RandomStringSpout"); builder.setBolt("wrapWellBolt", new WrapWellBolt()).shuffleGrouping("RandomStringSpout"); Config config = new Config(); config.setDebug(true); LocalCluster cluster = new LocalCluster(); cluster.submitTopology("RadomStringTopologyLocal", config, builder.createTopology()); System.err.println("the first topology is start running at local"); TimeUnit.SECONDS.sleep(30); cluster.killTopology("RadomStringTopologyLocal"); cluster.shutdown(); } }
    //从运行的结果中可以看出写的demo已经运行

    三.远程demo的实现

      1.使用上一个demo中的RandomStringSpout  WrapStarBolt  WrapWellBolt  这三个类,然后编写新的RandomStringTopologyRemote类

    public class RandomStringTopologyRemote {
        
        public static void main(String[] args) {
            final TopologyBuilder builder = new TopologyBuilder();
            builder.setSpout("RandomStringSpout", new RandomStringSpout());
            builder.setBolt("WrapStarBolt", new WrapStarBolt(),4).shuffleGrouping("RandomStringSpout");
            builder.setBolt("WrapWellBolt", new WrapWellBolt(),4).shuffleGrouping("RandomStringSpout");
            
            final Config config = new Config();
            config.setNumWorkers(3);
            try {
                StormSubmitter.submitTopology("RandomStringTopologyRemote", config, builder.createTopology());
            } catch (Exception e) {
                e.printStackTrace();
            } 
        }
    }

    2.把上面4个类通过maven工具打成jar包,并上传至nimbus所在的服务器上,上传完毕后通过以下命令远程启动storm

      storm jar storm_test-0.0.1-SNAPSHOT.jar com.zpb.RandomStringTopologyRemote

      远程启动storm的命令是:

        storm jar + *.jar  main函数的全路径名

    3.通过UI工具查看

      

     4.关闭提交的topology

      storm kill 提交的topology名  

           

          

  • 相关阅读:
    Leetcode:Convert Sorted List to Binary Search Tree
    Leetcode:Unique Binary Search Trees & Unique Binary Search Trees II
    Leetcode::Longest Common Prefix && Search for a Range
    Leetcode::Flatten Binary Tree to Linked List
    Leetcode::JumpGame
    leetcode power(x,n)
    Leetcode Letter Combinations of a Phone Number
    leetcode Reverse Nodes in k-Group
    leetcode Merge k Sorted Lists
    word ladder
  • 原文地址:https://www.cnblogs.com/MrRightZhao/p/11005174.html
Copyright © 2020-2023  润新知