• flume发送与接受日志


    apache-flume-1.5.0.1-bin.tar.gz
    
    日志接收端
    
    1.修改配置文件
    cp flume-conf.properties.template flume-conf.properties
    vim flume-conf.properties
    collector1.sources = AvroIn
    collector1.sources.AvroIn.type = avro
    collector1.sources.AvroIn.bind = 0.0.0.0
    collector1.sources.AvroIn.port = 4545
    collector1.sources.AvroIn.channels = mc1
    
    collector1.channels = mc1
    collector1.channels.mc1.type = memory
    collector1.channels.mc1.capacity = 100
    
    collector1.sinks = LocalOut
    
    collector1.sinks.LocalOut.type = file_roll
    collector1.sinks.LocalOut.sink.directory = /var/log/flume/collector1
    collector1.sinks.LocalOut.sink.rollInterval = 0
    collector1.sinks.LocalOut.channel = mc1
    
    
    cp flume-env.sh.template  flume-env.sh
    vim flume-env.sh
    修改JAVA_HOME
    
    
    2.运行
    bin/flume-ng agent -c conf -f conf/flume-conf.properties -n collector1
    
    日志发送端
    1.修改配置文件
    cp flume-conf.properties.template flume-conf.properties
    vim flume-conf.properties
    agent1.sources = ngrinder
    agent1.sources.ngrinder.type = exec 
    agent1.sources.ngrinder.command = tail -F /var/log/apache2/access.log
    agent1.sources.ngrinder.channels = mc1
    
    agent1.channels = mc1
    agent1.channels.mc1.type = memory
    agent1.channels.mc1.capacity = 100
    
    agent1.sinks = avro-sink
    
    agent1.sinks.avro-sink.type = avro
    agent1.sinks.avro-sink.channel = mc1
    agent1.sinks.avro-sink.hostname = 192.168.0.111
    agent1.sinks.avro-sink.port = 4545
    #缓存日志断点续传
    agent1.channels.memory4log.type = memory 
    agent1.channels.memory4log.capacity = 1000 
    agent1.channels.memory4log.transactionCapacity = 100
    
    
    cp flume-env.sh.template  flume-env.sh
    vim flume-env.sh
    修改JAVA_HOME
    
    2.执行
    bin/flume-ng agent -c conf -f conf/flume-conf.properties -n agent1
    
    3.注意
    /var/log/flume/collector1/ 要写入权限
    /var/log/apache2/access.log  要读取权限
  • 相关阅读:
    Halcon一日一练:图像分割之基本概念
    Halcon一日一练:创建AOI
    Halcon一日一练:获取图像属性
    Halcon一日一练:创建三通道图像
    Halcon一日一练:Halcon异常判断方法
    Java基础_类的加载机制和反射
    Java基础_死锁、线程组、定时器Timer
    Java基础_通过模拟售票情景解决线程不安全问题
    Java基础_线程的使用及创建线程的三种方法
    HTML/HTML5 知识点思维导图
  • 原文地址:https://www.cnblogs.com/DDSys/p/4027369.html
Copyright © 2020-2023  润新知