• ELK安装和配置及常用插件安装


    环境

    CentOS 7.3
    root 用户
    JDK 版本:1.8(最低要求),主推:JDK 1.8.0_121 以上
    关闭 firewall
    systemctl stop firewalld.service #停止firewall
    systemctl disable firewalld.service #禁止firewall开机启动
    
    关闭selinux
    

     

    安装 Elasticsearch

    elasticsearch运行需要使用普通用户

    修改 /etc/security/limits.conf

    *    soft              nofile     600000
    *    hard              nofile     600000
    *    soft              nproc      60000
    *    hard              nproc      60000
    jt_app soft memlock unlimited
    jt_app hard memlock unlimited
    

    修改/etc/sysctl.conf

    net.ipv4.ip_forward = 1
    net.ipv4.conf.default.rp_filter = 1
    net.ipv4.conf.default.accept_source_route = 0
    kernel.sysrq = 0
    vm.swappiness = 0
    kernel.core_uses_pid = 1
    net.ipv4.tcp_syncookies = 1
    kernel.msgmnb = 65536
    kernel.msgmax = 65536
    kernel.shmmax = 68719476736
    kernel.shmall = 4294967296
    net.core.somaxconn = 16384
    vm.max_map_count = 262144
    

    修改配置文件:

      生产环境主要配置:

    #grep -v '^#' elasticsearch.yml|grep -v '^$'
    cluster.name: prod_es_cluster
    node.name: elk-log-srv01
    node.master: true
    node.data: true
    path.data: /opt/es_data/data
    path.logs: /opt/elasticsearch/logs
    bootstrap.memory_lock: false
    network.host: elk-log-srv01
    http.port: 9200
    transport.tcp.port: 9300
    discovery.zen.ping_timeout: 3s
    discovery.zen.fd.ping_timeout: 60s
    discovery.zen.fd.ping_interval: 10s
    discovery.zen.ping.unicast.hosts: ["elk-log-srv01", "elk-log-srv02","elk-log-srv03"]
    discovery.zen.minimum_master_nodes: 2
    gateway.recover_after_nodes: 3
    gateway.expected_nodes: 3
    gateway.recover_after_time: 5m
    indices.query.bool.max_clause_count: 10240
    http.cors.enabled: true
    http.cors.allow-origin: "*"
    http.cors.allow-credentials: true
    search.max_buckets: 1000000
    

      

     启动:

    ./bin/elasticsearch -d
    

     启动脚本:

    [root@elk-log-srv01 elasticsearch]# cat /usr/lib/systemd/system/elasticsearch.service
    [Unit]
    Description=Elasticsearch
    Documentation=http://www.elastic.co
    Wants=network-online.target
    After=network-online.target
    
    [Service]
    RuntimeDirectory=elasticsearch
    Environment=ES_HOME=/opt/elasticsearch
    Environment=ES_PATH_CONF=/opt/elasticsearch/config
    Environment=PID_DIR=/opt/elasticsearch
    #EnvironmentFile=-/etc/sysconfig/elasticsearch
    #Environment=JAVA_HOME=/opt/jdk
    
    WorkingDirectory=/opt/elasticsearch
    
    User=jt_app
    Group=jt_app
    
    ExecStart=/opt/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet
    
    # StandardOutput is configured to redirect to journalctl since
    # some error messages may be logged in standard output before
    # elasticsearch logging system is initialized. Elasticsearch
    # stores its logs in /var/log/elasticsearch and does not use
    # journalctl by default. If you also want to enable journalctl
    # logging, you can simply remove the "quiet" option from ExecStart.
    StandardOutput=journal
    StandardError=inherit
    
    # Specifies the maximum file descriptor number that can be opened by this process
    LimitNOFILE=65536
    
    # Specifies the maximum number of processes
    LimitNPROC=4096
    
    # Specifies the maximum size of virtual memory
    LimitAS=infinity
    
    # Specifies the maximum file size
    LimitFSIZE=infinity
    
    #
    LimitMEMLOCK=infinity
    
    # Disable timeout logic and wait until process is stopped
    TimeoutStopSec=0
    
    # SIGTERM signal is used to stop the Java process
    KillSignal=SIGTERM
    
    # Send the signal only to the JVM rather than its control group
    KillMode=process
    
    # Java process is never killed
    SendSIGKILL=no
    
    # When a JVM receives a SIGTERM signal it exits with code 143
    SuccessExitStatus=143
    
    [Install]
    WantedBy=multi-user.target
    
    # Built for packages-6.3.2 (packages)
    [root@elk-log-srv01 elasticsearch]# 
    

     

    安装 Kibana

    选择一台节点安装即可
    进入安装目录修改配置文件:
        config/kibana.yml
    
    server.port: 5601                                  #端口
    server.host: "elk-log-srv01"                       #访问ip地址
    elasticsearch.url: "http://elk-log-srv01:9200"      #连接elastic               
    kibana.index: ".kibana"                            #在elastic中添加.kibana索引
    pid.file: /opt/kibana/kibana.pid
    logging.dest: /opt/kibana/kibana.log
    

     启动:

    nohup ./bin/kibana &
    

    logstash安装

    elasticsearch 常用插件安装

    只是版本不一样,方法是一样的,替换成自己的版本即可

    采用离线安装插件的方法

    1、sql插件

    ### 项目地址
    https://github.com/NLPchina/elasticsearch-sql
    历史版本:
    https://github.com/NLPchina/elasticsearch-sql/releases
    ### 下载sql插件
    下载
    wget https://github.com/NLPchina/elasticsearch-sql/releases/download/5.5.1.0/elasticsearch-sql-5.5.1.0.zip
    安装
    ./bin/elasticsearch-plugin install file:///opt/elasticsearch-sql-5.5.1.0.zip
    
    安装web访问
    wget https://github.com/NLPchina/elasticsearch-sql/releases/download/5.4.1.0/es-sql-site-standalone.zip
    unzip ./es-sql-site-standalone.zip
    cd site-server
    npm install express --save
    node node-server.js & #后台启动
    默认端口:8080
    cd _site
    vim controllers.js 
    修改链接es地址
    url = "http://localhost:9200"
    

     2 分词器

    项目地址:
    https://github.com/medcl/elasticsearch-analysis-ik/
    下载地址:
    wget https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v5.5.1/elasticsearch-analysis-ik-5.5.1.zip
    安装插件
    ./bin/elasticsearch-plugin install file:///opt/elasticsearch-analysis-ik-5.5.1.zip
    

      

    elasticsearch5.x安装head插件

    5.0以上版本中不支持直接安装head插件,需要启动一个服务。
    由于head插件本质上还是一个nodejs的工程,因此需要安装node,使用npm来安装依赖的包。(npm可以理解为maven)
    #安装git
    yum -y install git
    #下载源码
    git clone git://github.com/mobz/elasticsearch-head.git
    安装 nodejs,修改环境变量
    node -v
    2、安装npm
    
    3、使用npm安装grunt
    由于 npm 是国外的源,下载速度比较慢,推荐使用国内淘宝镜像
    npm install -g cnpm --registry=https://registry.npm.taobao.org
    
    下面开始修改 head 插件的配置
    
    地址:
    https://github.com/mobz/elasticsearch-head
    

      

    cerebro插件安装

    以单独进程启动
    下载
    wget https://github.com/lmenezes/cerebro/releases/download/v0.6.6/cerebro-0.6.6.zip
    解压
    unzip cerebro-0.6.6.zip
    启动:
    bin/cerebro -Dhttp.port=1234 -Dhttp.address=0.0.0.0 &
    ------------------------------------------------
    其他配置
    -Dconfig.file=/some/other/dir/alternate.conf
    
    ##项目地址
    https://github.com/lmenezes/cerebro
    

     

    kibana安装x-pack插件

    先下载x-pack-5.5.1.zip
    https://artifacts.elastic.co/downloads/packs
    在线安装
    bin/kibana-plugin install x-pack
    离线安装
    ./bin/kibana-plugin install file:///opt/x-pack-5.5.1.zip
    
    elasticsearch安装此插件一样
    

      

  • 相关阅读:
    洛谷 P4001 [ICPC-Beijing 2006]狼抓兔子
    杂题20201201
    杂题20210103
    杂题20201010
    杂题20200928
    ACL1 Contest 1 简要题解
    杂题20200906
    杂题20200623
    USACO2020DEC Gold/Platinum 解题报告
    CSP-S2020游记
  • 原文地址:https://www.cnblogs.com/xuliang666/p/11202928.html
Copyright © 2020-2023  润新知