• flume发送与接受日志


    apache-flume-1.5.0.1-bin.tar.gz
    
    日志接收端
    
    1.修改配置文件
    cp flume-conf.properties.template flume-conf.properties
    vim flume-conf.properties
    collector1.sources = AvroIn
    collector1.sources.AvroIn.type = avro
    collector1.sources.AvroIn.bind = 0.0.0.0
    collector1.sources.AvroIn.port = 4545
    collector1.sources.AvroIn.channels = mc1
    
    collector1.channels = mc1
    collector1.channels.mc1.type = memory
    collector1.channels.mc1.capacity = 100
    
    collector1.sinks = LocalOut
    
    collector1.sinks.LocalOut.type = file_roll
    collector1.sinks.LocalOut.sink.directory = /var/log/flume/collector1
    collector1.sinks.LocalOut.sink.rollInterval = 0
    collector1.sinks.LocalOut.channel = mc1
    
    
    cp flume-env.sh.template  flume-env.sh
    vim flume-env.sh
    修改JAVA_HOME
    
    
    2.运行
    bin/flume-ng agent -c conf -f conf/flume-conf.properties -n collector1
    
    日志发送端
    1.修改配置文件
    cp flume-conf.properties.template flume-conf.properties
    vim flume-conf.properties
    agent1.sources = ngrinder
    agent1.sources.ngrinder.type = exec 
    agent1.sources.ngrinder.command = tail -F /var/log/apache2/access.log
    agent1.sources.ngrinder.channels = mc1
    
    agent1.channels = mc1
    agent1.channels.mc1.type = memory
    agent1.channels.mc1.capacity = 100
    
    agent1.sinks = avro-sink
    
    agent1.sinks.avro-sink.type = avro
    agent1.sinks.avro-sink.channel = mc1
    agent1.sinks.avro-sink.hostname = 192.168.0.111
    agent1.sinks.avro-sink.port = 4545
    #缓存日志断点续传
    agent1.channels.memory4log.type = memory 
    agent1.channels.memory4log.capacity = 1000 
    agent1.channels.memory4log.transactionCapacity = 100
    
    
    cp flume-env.sh.template  flume-env.sh
    vim flume-env.sh
    修改JAVA_HOME
    
    2.执行
    bin/flume-ng agent -c conf -f conf/flume-conf.properties -n agent1
    
    3.注意
    /var/log/flume/collector1/ 要写入权限
    /var/log/apache2/access.log  要读取权限
  • 相关阅读:
    Vue+ElementUI 安装与应用
    python 之serial、pyusb 使用开发
    ASP.NET Swagger 创建与汉化生成 API说明文档
    DataGridView绑定数据源后动态删除行
    MySql动态拼接SQL并动态赋值
    MySql存储过程
    DEV控件之TreeList使用
    DataGridView单元格格式化
    ajax通过PUT方式调用WEBAPI
    解决跨域session 同步问题
  • 原文地址:https://www.cnblogs.com/DDSys/p/4027369.html
Copyright © 2020-2023  润新知