• flume发送与接受日志


    apache-flume-1.5.0.1-bin.tar.gz
    
    日志接收端
    
    1.修改配置文件
    cp flume-conf.properties.template flume-conf.properties
    vim flume-conf.properties
    collector1.sources = AvroIn
    collector1.sources.AvroIn.type = avro
    collector1.sources.AvroIn.bind = 0.0.0.0
    collector1.sources.AvroIn.port = 4545
    collector1.sources.AvroIn.channels = mc1
    
    collector1.channels = mc1
    collector1.channels.mc1.type = memory
    collector1.channels.mc1.capacity = 100
    
    collector1.sinks = LocalOut
    
    collector1.sinks.LocalOut.type = file_roll
    collector1.sinks.LocalOut.sink.directory = /var/log/flume/collector1
    collector1.sinks.LocalOut.sink.rollInterval = 0
    collector1.sinks.LocalOut.channel = mc1
    
    
    cp flume-env.sh.template  flume-env.sh
    vim flume-env.sh
    修改JAVA_HOME
    
    
    2.运行
    bin/flume-ng agent -c conf -f conf/flume-conf.properties -n collector1
    
    日志发送端
    1.修改配置文件
    cp flume-conf.properties.template flume-conf.properties
    vim flume-conf.properties
    agent1.sources = ngrinder
    agent1.sources.ngrinder.type = exec 
    agent1.sources.ngrinder.command = tail -F /var/log/apache2/access.log
    agent1.sources.ngrinder.channels = mc1
    
    agent1.channels = mc1
    agent1.channels.mc1.type = memory
    agent1.channels.mc1.capacity = 100
    
    agent1.sinks = avro-sink
    
    agent1.sinks.avro-sink.type = avro
    agent1.sinks.avro-sink.channel = mc1
    agent1.sinks.avro-sink.hostname = 192.168.0.111
    agent1.sinks.avro-sink.port = 4545
    #缓存日志断点续传
    agent1.channels.memory4log.type = memory 
    agent1.channels.memory4log.capacity = 1000 
    agent1.channels.memory4log.transactionCapacity = 100
    
    
    cp flume-env.sh.template  flume-env.sh
    vim flume-env.sh
    修改JAVA_HOME
    
    2.执行
    bin/flume-ng agent -c conf -f conf/flume-conf.properties -n agent1
    
    3.注意
    /var/log/flume/collector1/ 要写入权限
    /var/log/apache2/access.log  要读取权限
  • 相关阅读:
    信号量的实现
    锁的实现
    Do PDB Files Affect Performance?
    分布式系统一致性算法(Raft)
    idea开发工具破解地址
    IOS设备上传图片,使用ImageIO.write 图片翻转纠正(JAVA)
    使用WebSocket进行消息推送(附代码)
    SpringCloud构建微服务 | 服务注册与发现(一)提供Demo
    @slf4j注解找不到log变量-已解决
    springBoot事务无法回滚 MyISAM与InnoDB区别
  • 原文地址:https://www.cnblogs.com/DDSys/p/4027369.html
Copyright © 2020-2023  润新知