• docker-compose 部署 EFK


    信息:

    • Docker版本($ docker --version):Docker版本18.06.1-ce,版本e68fc7a
    • 系统信息($ cat /etc/centos-release):CentOS Linux release 7.5.1804 (Core)

    第一步:配置镜像加速

    sudo mkdir -p /etc/docker
    sudo tee /etc/docker/daemon.json <<-'EOF'
    {
      "registry-mirrors": ["https://registry.docker-cn.com/"]
    }
    EOF
    sudo systemctl daemon-reload
    sudo systemctl restart docker
    阿里镜像加速地址

    第二步:系统调优(必须)

    1.修改/etc/sysctl.conf,追加内容
    $ vm.max_map_count=655360
    $ fs.file-max=655350
    $ sysctl -p

    第三步: 部署EFK

    $ mkdir my_efk
    $ cd my_efk
    $ mkdir conf
    $ cd
    conf
    vim fluent.conf
    # my_efk/conf/fluent.conf
    <source>
      @type forward
      port 24224
      bind 0.0.0.0
    </source>
    <match *.**>
      @type copy
      <store>
        @type elasticsearch
        host elasticsearch
        port 9200
        logstash_format true
        logstash_prefix fluentd
        logstash_dateformat %Y%m%d
        include_tag_key true
        type_name access_log
        tag_key @log_name
        flush_interval 1s
      </store>
      <store>
        @type stdout
      </store>
    </match>
    vim Dockerfile
     # my_efk/Dockerfile
     FROM fluent/fluentd
     RUN ["gem", "install", "fluent-plugin-elasticsearch", "--no-rdoc", "--no-ri", "--version", "1.9.7"]
    vim docker-compose.yml 
    # my_efk/docker-compose.yaml
    version: '3.1'
    
    services:
    
      fluentd:
        container_name: 'fluentd'
        build: .
        ports: 
            - 24224:24224
            - 24224:24224/udp
        networks:
          - net_db
        volumes:
          - ./conf:/fluentd/etc
          - ./log:/fluentd/log
    
      elasticsearch:
        container_name: 'elasticsearch'
        image: docker.elastic.co/elasticsearch/elasticsearch:6.2.4
        environment:
          - discovery.type=single-node
        volumes:
          - ./esdata:/usr/share/elasticsearch/data
        ports: 
            - 9200:9200
            - 9300:9300
        networks:
          - net_db
    
      kibana:
        container_name: 'kibana'
        image: docker.elastic.co/kibana/kibana:6.2.4
        ports:
          - 5601:5601
        networks:
          - net_db
    
    networks:
      net_db:
        external: true
    docker network create net_db
    docker-compose up -d
  • 相关阅读:
    网络爬虫基础练习
    综合练习-词频统计
    组合数据类型
    【Hadoop】Hadoop综合大作业
    理解MapReduce操作
    Hadoop-熟悉常用的HDFS操作
    python-爬取中药信息
    Pytthon-数据结构化与保存
    python-爬取校园新闻首页的新闻
    python-爬虫基础
  • 原文地址:https://www.cnblogs.com/guoyiwen/p/10417016.html
Copyright © 2020-2023  润新知