想尝试多个agent来进行传输数据其中的一个配置文件如下:
al.sources = r1
al.sinks = k1
al.channels = c1
al.sources.r1.type = avro
al.sources.r1.channels = c1
al.sources.r1.bind = 0.0.0.0
al.sources.r1.port = 4141
al.sinks.k1.type = logger
al.channels.c1.type = memory
al.channels.c1.capacity = 1000
al.channels.c1.transactionCapacity = 100
al.sources.r1.channels = c1
al.sinks.k1.channel = c1
然后用命令执行:
bin/flume-ng agent -c conf -f conf/spool-logger2.conf -n al -Dflume.root.logger=INFO,console
却发现一直报错,部分错如下 :
2018-09-09T19:11:03,230 INFO [lifecycleSupervisor-1-0] org.apache.flume.node.PollingPropertiesFileConfigurationProvider - Configuration provider starting
2018-09-09T19:11:03,265 INFO [conf-file-poller-0] org.apache.flume.node.PollingPropertiesFileConfigurationProvider - Reloading configuration file:conf/spool-logger.conf
2018-09-09T19:11:03,292 INFO [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Added sinks: k1 Agent: a1
2018-09-09T19:11:03,294 INFO [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Added sinks: k1 Agent: al
2018-09-09T19:11:03,294 INFO [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Processing:k1
2018-09-09T19:11:03,295 INFO [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Processing:k1
2018-09-09T19:11:03,295 INFO [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Processing:k1
2018-09-09T19:11:03,296 INFO [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Processing:k1
2018-09-09T19:11:03,296 INFO [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Processing:k1
2018-09-09T19:11:03,348 WARN [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Agent configuration for 'al' does not contain any channels. Marking it as invalid.
2018-09-09T19:11:03,354 WARN [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Agent configuration invalid for agent 'al'. It will be removed.
2018-09-09T19:11:03,355 INFO [conf-file-poller-0] org.apache.flume.conf.FlumeConfiguration - Post-validation flume configuration contains configuration for agents: [a1]
18-09-09T19:11:03,356 WARN [conf-file-poller-0] org.apache.flume.node.AbstractConfiguratio▽Provider - No configuration found for this host:al
2018-09-09T19:11:03,386 INFO [conf-file-poller-0] org.apache.flume.node.Application - Starting new configuration:{ sourceRunners:{} sinkRunners:{} channels:{} }
重点在于No configuration found for this host:al
这个意思是配置文件没有al这个名字,但是我反复看了我的配置文件,我发现我的al没有错呀,所以一直没有改
一直到后来,我尝试把整个文件复制出来,把全部的al全部在改成al然后在尝试执行命令,果然成功了。
这个结果真是令人抓狂因为英文的al和英文a+数字1很像,所以一直以为是对的
所以最后就是把al拿出来统一改一遍应该就不会错了。