1. 安装环境
系统版本:centos 6.9 java版本:1.8.0_181 程序版本:6.6 (整个stack需保持相同的版本)
2. 安装顺序
1 Elasticsearch 2 Kibana 3 Logstash 4 Beats 5 Elasticsearch Hadoop
3. 安装 Elasticsearch
tar -xzf elasticsearch-6.6.2.tar.gz useradd elasticsearch chown -R elasticsearch:elasticsearch /opt/elasticsearch-6.6.2 su - elasticsearch /opt/elasticsearch-6.6.2/bin/elasticsearch -d
启动报错: Error: max number of threads [1024] for user [elasticsearch] is too low, increase to at least [4096] vi /etc/security/limits.d/90-nproc.conf add one line elasticsearch soft nproc 4096 Error: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] vi /etc/sysctl.conf add one line vm.max_map_count=655360 sysctl -p Error: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536] vi /etc/security/limits.conf elasticsearch hard nofile 102400 elasticsearch soft nofile 102400 Error: system call filters failed to install; check the logs and fix your configuration or disable system call filters at your own risk vi elasticsearch.yml network.host: 0.0.0.0 bootstrap.system_call_filter: false
curl http://localhost:9200/
4. 安装kibana
tar -xzf kibana-6.6.2-darwin-x86_64.tar.gz
vi /opt/kibana-6.6.2-linux-x86_64/config/kibana.yml server.host: "192.168.1.197" elasticsearch.hosts: ["http://localhost:9200"]
5. 安装logstash
tar xaf logstash-6.6.2.tar.gz
vi /opt/logstash-6.6.2/config/pipelines.yml - pipeline.id: test pipeline.workers: 1 pipeline.batch.size: 1 config.string: input { beats { port=>"5045" } } output { elasticsearch { index=>"logstat-test1" hosts=>["localhost:9200"] } }
vi /opt/logstash-6.6.2/config/pipelines.yml - pipeline.id: test pipeline.workers: 1 pipeline.batch.size: 1 config.string: input { beats { port=>"5045" } } filter {grok {match => { "message" => '%{IPV4:clientip} - - [%{HTTPDATE:time}] %{IPV4:serverip} "%{WORD:verb} %{NOTSPACE:request} HTTP/%{NUMBER:httpversion}|%{DATA:rawrequest}" %{NUMBER:response} %{NUMBER:bytes}|-' }}} output { elasticsearch { index=>"httpd_log_188_%{+YYYY.MM.dd}" hosts=>["elasticsearch.oohome.net:9200"] } }
6. 安装filebeat
tar xf filebeat-6.6.2-linux-x86_64.tar.gz
vi /opt/filebeat-6.6.2-linux-x86_64/filebeat.yml filebeat.inputs: - type: log enabled: true paths: - /var/log/secure output.logstash: hosts: ["localhost:5045"]