• flume sink kafka


    agent.sources = s1   
    agent.channels = c1 
    agent.sinks = sk1 
     
    agent.sources.s1.type = netcat 
    agent.sources.s1.bind = localhost 
    agent.sources.s1.port = 5678 
    agent.sources.s1.channels = c1 
    
    agent.sinks.sk1.type = org.apache.flume.sink.kafka.KafkaSink
    agent.sinks.sk1.topic = hellotest
    agent.sinks.sk1.brokerList = localhost:9092
    agent.sinks.sk1.requiredAcks = 1
    agent.sinks.sk1.batchSize = 20
    agent.sinks.sk1.channel = c1
     
    agent.channels.c1.type = memory
    agent.channels.c1.capacity = 1000    
    agent.channels.c1.transactionCapacity = 100
    

     开启zookeeper和kafka,创建一个hellotest的topic

    Last login: Tue Mar 21 10:57:36 on ttys004
    lihudeMacBook-Pro:~ SunAndLi$  telnet  localhost 5678
    Trying ::1...
    telnet: connect to address ::1: Connection refused
    Trying 127.0.0.1...
    Connected to localhost.
    Escape character is '^]'.
    lihuhu 
    OK
    lihuhu is my husband
    OK
    

     查看

    Last login: Tue Mar 21 10:56:50 on ttys003
    lihudeMacBook-Pro:~ SunAndLi$ cd kafka_2.11-0.10.1.0/
    lihudeMacBook-Pro:kafka_2.11-0.10.1.0 SunAndLi$ bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic hellotest --from-beginning
    Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
    lihuhu
    lihuhu is my husband
    
  • 相关阅读:
    【docker】更换挂载目录
    【设计】交互走查表
    MySQL常用字符串函数
    VIM_manual
    MySQL操作符
    基础SELECT实例
    MySQL字符集及校对规则的理解
    Linux命令之tar-rsync
    Linux-PATH_环境变量
    MySQL常用数据类型
  • 原文地址:https://www.cnblogs.com/sunyaxue/p/6593992.html
Copyright © 2020-2023  润新知