• Flink 读取 Kafka数据


    POM

        <dependencies>
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-scala_2.11</artifactId>
                <version>1.10.2</version>
            </dependency>
            <!-- https://mvnrepository.com/artifact/org.apache.flink/flink-streaming-scala -->
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-streaming-scala_2.11</artifactId>
                <version>1.10.2</version>
            </dependency>
            <dependency>
                <groupId>org.apache.flink</groupId>
                <artifactId>flink-connector-kafka-0.11_2.11</artifactId>
                <version>1.10.1</version>
            </dependency>
        </dependencies>

    源码: 

    package com.kpwong.aiptest
    
    import java.util.Properties
    
    import org.apache.flink.api.common.serialization.SimpleStringSchema
    import org.apache.flink.streaming.api.scala.{DataStream, StreamExecutionEnvironment}
    import org.apache.flink.api.scala._
    import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer011
    
    object KafkaTest {
    
      def main(args: Array[String]): Unit = {
        val env: StreamExecutionEnvironment = StreamExecutionEnvironment.getExecutionEnvironment
    
        //从kafka读取数据
        val prob: Properties = new Properties()
        prob.setProperty("bootstrap.servers","hadoop202:9092")
        //bin/kafka-console-producer.sh --broker-list hadoop202:9092 --topic two 发送数据命令
        val kafkaDS: DataStream[String] = env.addSource( new FlinkKafkaConsumer011[String]("two",new SimpleStringSchema(),prob))
    
        kafkaDS.print()
        env.execute()
    
      }
    
    }

    Kafka发送数据:

    //bin/kafka-console-producer.sh --broker-list hadoop202:9092 --topic two 

    运行结果:

  • 相关阅读:
    SAP中的文档维护
    SAP日期处理函数汇总(转)
    PR PO通过fm创建时,如何传输增强字段
    html+jQuery简单的利息计算器
    上传项目到github
    Nodejs-毕业设计5-小技巧
    Nodejs-毕业设计4-登录页面
    Nodejs-毕业设计3-路径
    Nodejs-毕业设计2-下载依赖准备工作
    Nodejs-毕业设计1-部署环境
  • 原文地址:https://www.cnblogs.com/kpwong/p/14089801.html
Copyright © 2020-2023  润新知