• Flink 读取 Kafka数据


    POM

        <dependencies>
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-scala_2.11</artifactId>
                <version>1.10.2</version>
            </dependency>
            <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-streaming-scala -->
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-streaming-scala_2.11</artifactId>
                <version>1.10.2</version>
            </dependency>
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-connector-kafka-0.11_2.11</artifactId>
                <version>1.10.1</version>
            </dependency>
        </dependencies>

    源码: 

    package com.kpwong.aiptest
    
    import java.util.Properties
    
    import org.apache.flink.api.common.serialization.SimpleStringSchema
    import org.apache.flink.streaming.api.scala.{DataStream, StreamExecutionEnvironment}
    import org.apache.flink.api.scala._
    import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer011
    
    object KafkaTest {
    
      def main(args: Array[String]): Unit = {
        val env: StreamExecutionEnvironment = StreamExecutionEnvironment.getExecutionEnvironment
    
        //从kafka读取数据
        val prob: Properties = new Properties()
        prob.setProperty("bootstrap.servers","hadoop202:9092")
        //bin/kafka-console-producer.sh --broker-list hadoop202:9092 --topic two 发送数据命令
        val kafkaDS: DataStream[String] = env.addSource( new FlinkKafkaConsumer011[String]("two",new SimpleStringSchema(),prob))
    
        kafkaDS.print()
        env.execute()
    
      }
    
    }

    Kafka发送数据:

    //bin/kafka-console-producer.sh --broker-list hadoop202:9092 --topic two 

    运行结果:

  • 相关阅读:
    Levmar 配置
    函数前加static与不加static的区别
    关于形如--error LNK2005: xxx 已经在 msvcrtd.lib ( MSVCR90D.dll ) 中定义--的问题分析解决
    Latex 算法Algorithm
    matlab 曲线拟合
    生成eps图形
    libSVM在matlab下的使用安装
    spring 事务管理笔记
    集合框架
    小程序 滑动弹窗阻止主页面滑动
  • 原文地址:https://www.cnblogs.com/kpwong/p/14089801.html
Copyright © 2020-2023  润新知