• flume操作步骤


    1.启动服务:

     bin/flume-ng agent -c conf -f ./conf/udp-2kafka.conf  -n myagent

    2.发送数据

    nc -vu 183.213.26.120 9056

    2.两个Kafka集群去看数据

     

     

     

    myagent.sources = r1

    myagent.sinks = k1 k2

    myagent.channels = c1 c2

    myagent.sources.r1.selector.type = replicating

     

    myagent.sources.r1.type = syslogudp

    myagent.sources.r1.bind=172.16.0.112

    myagent.sources.r1.port = 9056

    myagent.sources.r1.handler = org.apache.flume.source.http.JSONHandler

     

    myagent.sources.r1.channels = c1 c2

     

    myagent.sinks.k1.channel = c1

    myagent.channels.c1.type = memory

    myagent.channels.c1.capacity = 100000

    myagent.channels.c1.transactionCapacity = 1000

     

    myagent.sinks.k2.channel = c2

    myagent.channels.c2.type = memory

    myagent.channels.c2.capacity = 100000

    myagent.channels.c2.transactionCapacity = 1000

     

    myagent.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSink

    myagent.sinks.k1.kafka.topic = test

    myagent.sinks.k1.kafka.bootstrap.servers = 172.16.0.112:9092

    myagent.sinks.k1.kafka.flumeBatchSize = 20

    myagent.sinks.k1.kafka.producer.acks = 1

    myagent.sinks.k1.kafka.producer.linger.ms = 1

    myagent.sinks.k1.kafka.producer.compression.type = snappy

     

    myagent.sinks.k2.type = org.apache.flume.sink.kafka.KafkaSink

    myagent.sinks.k2.kafka.topic = test

    #myagent.sinks.k2.kafka.bootstrap.servers = 172.16.0.188:9092,172.16.0.49:9092,172.16.0.63:9092

    myagent.sinks.k2.kafka.bootstrap.servers = 183.213.26.130:9092

    myagent.sinks.k2.kafka.flumeBatchSize = 20

    myagent.sinks.k2.kafka.producer.acks = 1

    myagent.sinks.k2.kafka.producer.linger.ms = 1

    myagent.sinks.k2.kafka.producer.compression.type = snappy

  • 相关阅读:
    Digital Square(hdu4394)搜索
    Substrings(hdu1238)字符串匹配
    Graveyard(poj3154)
    数据库的范式理论
    模和除
    You can Solve a Geometry Problem too (hdu1086)几何,判断两线段相交
    对vector等STL标准容器进行排序操作(转!)
    0 or 1(hdu2608)数学题
    LintCode 4.丑数
    LintCode 9.Fizz Buzz问题
  • 原文地址:https://www.cnblogs.com/yunger/p/14592104.html
Copyright © 2020-2023  润新知