ELK搭建

ELK安装

参考资料：https://blog.csdn.net/tzs_1041218129/article/details/78907350

ES安装

环境

1、VMware

2、Centos 7

3、Elasticsearch 5.5.2

4、JDK 1.8

主机

es1 192.168.100.101

es2 192.168.100.102

es3 192.168.100.103

软件安装

JDK安装

下载

官网下载jdk1.8

解压

mkdir /usr/local/java/

tar -zvxf jdk-8u191-linux-x64.tar.gz -C /usr/local/java/

配置环境变量

vim /etc/profile

在末尾新建行添加：

#jdk

export JAVA_HOME=/usr/local/java/jdk1.8.0_191

export PATH=$PATH:$JAVA_HOME/bin

配置生效

source /etc/profile

ES安装

下载

链接：https://artifacts.elastic.co

解压

tar -zvxf elasticsearch-5.5.2.tar.gz -C /usr/local/

解压后的目录结构：

[root@es1 elasticsearch-5.5.2]# cd /usr/local/elasticsearch-5.5.2

[root@es1 elasticsearch-5.5.2]# ll

drwxr-xr-x  2 es es   4096 Dec 27 00:30 bin

drwxr-xr-x  3 es es     90 Dec 27 00:51 config

drwxr-xr-x  3 es es     19 Dec 27 00:51 data

drwxr-xr-x  2 es es   4096 Aug 14  2017 lib

-rw-r--r--  1 es es  11358 Aug 14  2017 LICENSE.txt

drwxr-xr-x  2 es es    162 Dec 27 00:49 logs

drwxr-xr-x 13 es es    236 Aug 14  2017 modules

-rw-r--r--  1 es es 194187 Aug 14  2017 NOTICE.txt

drwxr-xr-x  3 es es     16 Dec 27 00:37 plugins

-rw-r--r--  1 es es   9549 Aug 14  2017 README.textile

修改配置文件

[root@es1 elasticsearch-5.5.2]# vim /usr/local/elasticsearch-5.5.2/config/elasticsearch.yml

#这两个是安装 head 插件要用的（ELK安装中不需要）

http.cors.enabled: true

http.cors.allow-origin: "*"

 

# 集群名称（自定义，集群中所有主机必须一致）

cluster.name: aubin-cluster

 

# 节点名称，仅仅是描述名称，用于在日志中区分

node.name: es1

 

#该节点有机会成为master节点

node.master: true

 

#该节点可以存储数据

node.data: true

 

# 数据的默认存放路径

path.data: /opt/install/elasticsearch-5.5.2/data

 

# 日志的默认存放路径

path.logs: /opt/install/elasticsearch-5.5.2/logs

 

# 当前节点的IP地址

network.host: 192.168.100.103

 

# 对外提供服务的端口，9300为集群服务的端口

http.port: 9200

 

#设置节点之间交互的端口号

transport.tcp.port: 9300

 

# 集群个节点IP地址，也可以使用els、els.shuaiguoxia.com等名称，需要各节点能够解析

discovery.zen.ping.unicast.hosts: ["192.168.100.101:9300", "192.168.100.102:9300","192.168.100.103:9300"]

 

# 为了避免脑裂，集群节点数最少为 半数+1

discovery.zen.minimum_master_nodes: 2

新增用户/组

(ES启动是不允许使用root用户)

groupadd elsearch

useradd elsearch -g elsearch -p elasticsearch

启动

切换到elsearch用户

su elsearch

cd /usr/local/elasticsearch-5.5.2/

./bin/elasticsearch

安装ik中文分词器

1. 下载 （注意，下载对应的版本，本文下载的是5.5.2）

https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v5.5.2/elasticsearch-analysis-ik-5.5.2.zip

1. 解压

首先在ES安装目录下的plugins目录下新建目录ik

cd /usr/local/elasticsearch-5.5.2/plugins

mkdir ik

再将分词器解压到ik目录下（先解压再移动/复制也可以）

unzip 命令（如果出现command not found，安装一下yum -y install unzip）

解压后的目录结构：

[root@es1 ik]# cd /opt/install/elasticsearch-5.5.2/plugins/ik/

[root@es1 ik]# ll

total 1420

-rw-rw-r-- 1 elsearch elsearch 263965 Aug 30  2017 commons-codec-1.9.jar

-rw-rw-r-- 1 elsearch elsearch  61829 Aug 30  2017 commons-logging-1.2.jar

drwxrwxr-x 2 elsearch elsearch    299 Aug 30  2017 config

-rw-rw-r-- 1 elsearch elsearch  51395 Aug 30  2017 elasticsearch-analysis-ik-5.5.2.jar

-rw-rw-r-- 1 elsearch elsearch 736658 Aug 30  2017 httpclient-4.5.2.jar

-rw-rw-r-- 1 elsearch elsearch 326724 Aug 30  2017 httpcore-4.4.4.jar

-rw-rw-r-- 1 elsearch elsearch   2666 Aug 30  2017 plugin-descriptor.properties

3.重启ES即可

logstash安装

下载

链接：https://artifacts.elastic.co

解压

tar -zvxf logstash-5.5.2.tar.gz -C /usr/local/

解压后目录结构

[root@es1 logstash-5.5.2]# ll /usr/local/logstash-5.5.2

total 180

drwxr-xr-x 2 root root    304 Dec 27 17:35 bin

-rw-r--r-- 1 root root 111573 Aug 14  2017 CHANGELOG.md

drwxr-xr-x 2 root root     93 Dec 27 17:32 config

-rw-r--r-- 1 root root   2249 Aug 14  2017 CONTRIBUTORS

drwxr-xr-x 5 root root     84 Dec 27 01:24 data

-rw-r--r-- 1 root root   3994 Aug 14  2017 Gemfile

-rw-r--r-- 1 root root  21548 Aug 14  2017 Gemfile.jruby-1.9.lock

drwxr-xr-x 5 root root     65 Dec 27 00:54 lib

-rw-r--r-- 1 root root    589 Aug 14  2017 LICENSE

drwxr-xr-x 2 root root     66 Dec 27 01:24 logs

drwxr-xr-x 4 root root     84 Dec 27 00:54 logstash-core

drwxr-xr-x 3 root root     57 Dec 27 00:54 logstash-core-plugin-api

drwxr-xr-x 3 root root     40 Dec 27 00:54 modules

-rw-r--r-- 1 root root  29345 Aug 14  2017 NOTICE.TXT

drwxr-xr-x 3 root root     30 Dec 27 00:54 tools

drwxr-xr-x 4 root root     33 Dec 27 00:54 vendor

修改配置文件

[root@es1 logstash-5.5.2]# vim /usr/local/logstash-5.5.2/bin/logstash.conf

input{

  file{

      path=>["/var/log/*.log"]

      type=>"logstash_log"

      start_position =>"beginning"

  }

}

output{

    elasticsearch{

        hosts=>["192.168.100.101:9200","192.168.100.102:9200","192.168.100.103:9200"]

        index=>"log-%{+YYYY.MM.dd.HH.mm.ss}"

        }

}

启动

cd /usr/local/logstash-5.5.2/bin

./logstash -f logstash.conf

 

kibana安装

下载

链接：https://artifacts.elastic.co

解压

tar -zvxf kibana-5.5.2-linux-x86_64.tar.gz -C /usr/local/

解压后的目录结构

[root@es1 kibana-5.5.2]# ll

total 832

drwxr-xr-x   2 es es     41 Aug 14  2017 bin

drwxrwxr-x   2 es es     24 Dec 27 19:23 config

drwxrwxr-x   2 es es     18 Dec 27 01:48 data

-rw-rw-r--   1 es es    562 Aug 14  2017 LICENSE.txt

drwxrwxr-x   6 es es    108 Aug 14  2017 node

drwxrwxr-x 619 es es  20480 Aug 14  2017 node_modules

-rw-rw-r--   1 es es 798064 Aug 14  2017 NOTICE.txt

drwxrwxr-x   3 es es     45 Aug 14  2017 optimize

-rw-rw-r--   1 es es    721 Aug 14  2017 package.json

drwxrwxr-x   2 es es      6 Aug 14  2017 plugins

-rw-rw-r--   1 es es   4909 Aug 14  2017 README.txt

drwxr-xr-x  12 es es    153 Aug 14  2017 src

drwxrwxr-x   5 es es     52 Aug 14  2017 ui_framework

drwxr-xr-x   2 es es    309 Aug 14  2017 webpackShims

修改配置文件

vim config/kibana.yml

server.port: 5601  # 配置kibana的端口

server.host: 192.168.100.101  # 配置监听ip

elasticsearch.url: "http://192.168.100.101:9200"  # 配置es服务器的ip，如果是集群则配置该集群中主节点的ip

logging.dest: /var/log/kibana.log  # 配置kibana的日志文件路径，不然默认是messages里记录日志

启动

cd /usr/local/kibana-5.5.2/

./bin/kibana

浏览器访问

http://192.168.100.101:5601

@font-face{ font-family:"Times New Roman"; } @font-face{ font-family:"宋体"; } @font-face{ font-family:"Calibri"; } @font-face{ font-family:"DejaVu Sans"; } @font-face{ font-family:"方正黑体_GBK"; } @font-face{ font-family:"Arial"; } p.MsoNormal{ mso-style-name:正文; mso-style-parent:""; margin:0pt; margin-bottom:.0001pt; mso-pagination:none; text-align:justify; text-justify:inter-ideograph; font-family:Calibri; mso-fareast-font-family:宋体; mso-bidi-font-family:'Times New Roman'; font-size:10.5000pt; mso-font-kerning:1.0000pt; } h2{ mso-style-name:"标题 2"; mso-style-noshow:yes; mso-style-next:正文; margin-top:13.0000pt; margin-bottom:13.0000pt; mso-para-margin-top:0.0000gd; mso-para-margin-bottom:0.0000gd; page-break-after:avoid; mso-pagination:lines-together; text-align:justify; text-justify:inter-ideograph; mso-outline-level:2; line-height:172%; font-family:'DejaVu Sans'; mso-fareast-font-family:方正黑体_GBK; mso-bidi-font-family:'Times New Roman'; font-weight:bold; font-size:16.0000pt; mso-font-kerning:1.0000pt; } h3{ mso-style-name:"标题 3"; mso-style-noshow:yes; mso-style-next:正文; margin-top:13.0000pt; margin-bottom:13.0000pt; mso-para-margin-top:0.0000gd; mso-para-margin-bottom:0.0000gd; page-break-after:avoid; mso-pagination:lines-together; text-align:justify; text-justify:inter-ideograph; mso-outline-level:3; line-height:172%; font-family:Calibri; mso-fareast-font-family:宋体; mso-bidi-font-family:'Times New Roman'; font-weight:bold; font-size:16.0000pt; mso-font-kerning:1.0000pt; } span.10{ font-family:'Times New Roman'; } span.15{ font-family:'Times New Roman'; color:rgb(0,0,255); text-decoration:underline; text-underline:single; } span.msoIns{ mso-style-type:export-only; mso-style-name:""; text-decoration:underline; text-underline:single; color:blue; } span.msoDel{ mso-style-type:export-only; mso-style-name:""; text-decoration:line-through; color:red; } @page{mso-page-border-surround-header:no; mso-page-border-surround-footer:no;}@page Section0{ } div.Section0{page:Section0;}

补充

elasticsearch-head安装

下载：https://github.com/mobz/elasticsearch-head 

Git地址： git clone https://github.com/mobz/elasticsearch-head.git

进入 elasticsearch-head目录：cd命令  

执行命令：npm install

如果出现下面的错误：

@font-face{ font-family:"Times New Roman"; } @font-face{ font-family:"宋体"; } @font-face{ font-family:"Calibri"; } p.MsoNormal{ mso-style-name:正文; mso-style-parent:""; margin:0pt; margin-bottom:.0001pt; mso-pagination:none; text-align:justify; text-justify:inter-ideograph; font-family:Calibri; mso-fareast-font-family:宋体; mso-bidi-font-family:'Times New Roman'; font-size:10.5000pt; mso-font-kerning:1.0000pt; } span.msoIns{ mso-style-type:export-only; mso-style-name:""; text-decoration:underline; text-underline:single; color:blue; } span.msoDel{ mso-style-type:export-only; mso-style-name:""; text-decoration:line-through; color:red; } @page{mso-page-border-surround-header:no; mso-page-border-surround-footer:no;}@page Section0{ } div.Section0{page:Section0;}

执行下面的：

npm install phantomjs-prebuilt@2.1.16 --ignore-scripts

修改/etc/sysctl.conf 文件否则会报max virtual memory areas vm.max_map_count [65530]切换到root用户修改配置sysctl.conf

vi /etc/sysctl.conf

添加下面配置：

vm.max_map_count=655360

并执行命令：

sysctl -p

修改/etc/security/limits.conf文件，否则会报max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]错误

vim /etc/security/limits.conf

* soft nofile 65536

* hard nofile 65536

* soft nproc 2048

* hard nproc 4096

如果遭到java.net.BindException: Cannot assign requested address问题

检查elasticsearch.yml配置文件看看host是否写错了