efk简介

此次采用docker 安装es和kinaba。filebeat 本地安装。
建议这三个组件要保持版本一致

1. docker 安装es和kinaba

1.1 安装es

# 下载镜像
docker pull elasticsearch:7.8.0

# 创建自定义的网络(用于连接到连接到同一网络的其他服务(例如Kibana))
docker network create somenetwork 

# 运行 elasticsearch
docker run -d --name elasticsearch --net somenetwork -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:7.8.0

# 查看容器状态
docker ps

此时es已经安装完成且运行

1.2 安装kibana

# 下载镜像
docker pull kibana:7.8.0

# 运行 Kibana
docker run -d --name kibana --net somenetwork -p 5601:5601 kibana:7.8.0

访问 http://127.0.0.1:5601 （启动可能会较慢，如失败等几秒再尝试刷新一下）

好了，这就安装完成和运行起来了。是不是感觉docker安装很简单

2. 安装filebeat 7.8.0

这里和上面保持一致 安装7.8.0版本。这个下载下来直接解压就好了

2.1 filebeat.yml配置

#=========================== Filebeat inputs =============================

filebeat.inputs:

# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.

- type: log
  # Change to true to enable this input configuration.
  enabled: true
  # Paths that should be crawled and fetched. Glob based paths.
  paths:
  # 日志实际路径地址
   - /Users/zhenghengbin/code/bailing-service-drug/logs/bailing-service-drug/bailing-service-drug-info*.log
  fields:
  # 日志标签，区别不同日志，下面建立索引会用到
    type: "bailing-service-drug"
  fields_under_root: true
  # 指定被监控的文件的编码类型，使用plain和utf-8都是可以处理中文日志的
  encoding: utf-8
  # 多行日志开始的那一行匹配的pattern
  multiline.pattern: ^s*dddd-dd-dd
  # 是否需要对pattern条件转置使用，不翻转设为true，反转设置为false。  【建议设置为true】
  multiline.negate: true
  # 匹配pattern后，与前面（before）还是后面（after）的内容合并为一条日志
  multiline.match: after
  #============================== Kibana =====================================

# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:              
#kibanaIP地址
host: "127.0.0.1:5601"
  # Kibana Host
  # Scheme and port can be left out and will be set to the default (http and 5601)
  # In case you specify and additional path, the scheme is required: http://localhost:5601/path
  # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
  #host: "localhost:5601"

  # Kibana Space ID
  # ID of the Kibana Space into which the dashboards should be loaded. By default,
  # the Default Space will be used.
  #space.id:

#============================= Elastic Cloud ==================================
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
  enabled: true
  # Array of hosts to connect to
  hosts: ["127.0.0.1:9200"]
#  index: "logs-%{[beat.version]}-%{+yyyy.MM.dd}"
  indices:
       #索引名称，一般为  ‘服务名称+ip+ --%{+yyyy.MM.dd}’。
    - index: "bailing-service-drug-%{+yyyy.MM.dd}"  
      when.contains:
      #标签，对应日志和索引，和上面对应
        type: "bailing-service-drug"

  # Optional protocol and basic auth credentials.
  #protocol: "https"
  # username: "elastic"
  # password: "bljk@123"

#----------------------------- Logstash output --------------------------------

#================================ Processors =====================================

# Configure processors to enhance or manipulate events generated by the beat.

processors:
- drop_fields:
# 去除多余字段
     fields: ["agent.type", "agent.version","log.file.path","log.offset","input.type","ecs.version","host.name","agent.ephemeral_id","agent.hostname","agent.id","_id","_index","_score","_suricata.eve.timestamp","agent.hostname","cloud. availability_zone","host.containerized","host.os.kernel","host.os.name","host.os.version"]

2.2 启动filebeat程序,进入filebeat文件夹

 nohup  ./filebeat  -c  filebeat.yml  -e  >/dec/null  2>&1  &

3.配置kibana

到现在基本配置完成。如果安装正常，应可在kibana后台设置看到配置的索引名称，如查看不到，检查filebeat程序是否正常启动，或者filebeat.yml配置是否正确。

参考：https://www.cnblogs.com/fbtop/p/11005469.html#4736536