• storm入门demo


    一.storm入门demo的介绍

           storm的入门helloworld有2种方式,一种是本地的,另一种是远程。

      本地实现:

        本地写好demo之后,不用搭建storm集群,下载storm的相关jar包即可实现storm的相关操作

      远程实现:

        本地写好demo之后,需要将其打成jar包,然后通过nimbus将jar包运行即可

        本地打包注意事项:

          由于打好的jar包会将其放到storm的集群上,因此storm上已经包含了运行的相关环境,但是在通过maven打包时需要storm-core设置成provided范围,不需要将storm-core的相关类打进jar包,以避免引起冲突

    二.本地demo的实现

      下载storm 所需jar包storm-core

    1.spout数据源的实现


    public
    class RandomStringSpout extends BaseRichSpout{ private final static Map<Integer,String> map = new HashMap<Integer, String>(); private SpoutOutputCollector collector; public RandomStringSpout(){ map.put(0, "kafka"); map.put(1, "nifi"); map.put(2, "flink"); map.put(3, "storm"); map.put(4, "spark"); } //在Spout组件初始化时被调用 public void open(Map arg0, TopologyContext topologyContextrg1, SpoutOutputCollector spoutOutputCollector) { System.err.println(" ============== open"); this.collector = spoutOutputCollector; } //nextTuple()方法是Spout实现的核心。 //也就是主要执行方法,用于输出信息,通过collector.emit方法发射 public void nextTuple() { //发送数据 collector.emit(new Values(map.get(ThreadLocalRandom.current().nextInt(4)))); try { TimeUnit.SECONDS.sleep(5); } catch (InterruptedException e) { e.printStackTrace(); } } //用于声明数据格式,即输出的一个Tuple中,包含几个字段 public void declareOutputFields(OutputFieldsDeclarer declarer) { declarer.declare(new Fields("stream")); } }

    2.Bolt数据过滤

    public class WrapStarBolt extends BaseBasicBolt{
    
        public void execute(Tuple tuple, BasicOutputCollector Collector) {
            String value = tuple.getStringByField("stream");
            System.err.println("******"+value);
        }
    
        public void declareOutputFields(OutputFieldsDeclarer declarer) {
             //nothing to do 
        }
    }
    public class WrapWellBolt extends BaseBasicBolt{
    
        public void execute(Tuple tuple, BasicOutputCollector collector) {
            String value = tuple.getStringByField("stream");
            System.err.println("#######"+value);
        }
    
        public void declareOutputFields(OutputFieldsDeclarer arg0) {
            //nothing to do 
        }
    }

    3.创建topology

    //所有的spout bolt 会组成一个topology
    public
    class RadomStringTopologyLocal { public static void main(String[] args) throws InterruptedException { TopologyBuilder builder = new TopologyBuilder(); builder.setSpout("RandomStringSpout", new RandomStringSpout()); builder.setBolt("wrapStarBolt", new WrapStarBolt()).shuffleGrouping("RandomStringSpout"); builder.setBolt("wrapWellBolt", new WrapWellBolt()).shuffleGrouping("RandomStringSpout"); Config config = new Config(); config.setDebug(true); LocalCluster cluster = new LocalCluster(); cluster.submitTopology("RadomStringTopologyLocal", config, builder.createTopology()); System.err.println("the first topology is start running at local"); TimeUnit.SECONDS.sleep(30); cluster.killTopology("RadomStringTopologyLocal"); cluster.shutdown(); } }
    //从运行的结果中可以看出写的demo已经运行

    三.远程demo的实现

      1.使用上一个demo中的RandomStringSpout  WrapStarBolt  WrapWellBolt  这三个类,然后编写新的RandomStringTopologyRemote类

    public class RandomStringTopologyRemote {
        
        public static void main(String[] args) {
            final TopologyBuilder builder = new TopologyBuilder();
            builder.setSpout("RandomStringSpout", new RandomStringSpout());
            builder.setBolt("WrapStarBolt", new WrapStarBolt(),4).shuffleGrouping("RandomStringSpout");
            builder.setBolt("WrapWellBolt", new WrapWellBolt(),4).shuffleGrouping("RandomStringSpout");
            
            final Config config = new Config();
            config.setNumWorkers(3);
            try {
                StormSubmitter.submitTopology("RandomStringTopologyRemote", config, builder.createTopology());
            } catch (Exception e) {
                e.printStackTrace();
            } 
        }
    }

    2.把上面4个类通过maven工具打成jar包,并上传至nimbus所在的服务器上,上传完毕后通过以下命令远程启动storm

      storm jar storm_test-0.0.1-SNAPSHOT.jar com.zpb.RandomStringTopologyRemote

      远程启动storm的命令是:

        storm jar + *.jar  main函数的全路径名

    3.通过UI工具查看

      

     4.关闭提交的topology

      storm kill 提交的topology名  

           

          

  • 相关阅读:
    How to change hostname on SLE
    How to install starDIct on suse OS?
    python logging usage
    How to reset password for unknow root
    How to use wget ?
    How to only capute sub-matched character by grep
    How to inspect who is caller of func and who is the class of instance
    How to use groovy script on jenkins
    Vim ide for shell development
    linux高性能服务器编程 (二) --IP协议详解
  • 原文地址:https://www.cnblogs.com/MrRightZhao/p/11005174.html
Copyright © 2020-2023  润新知