1、下载安装包
es官网比较慢,可以去镜像下载, 镜像下载最好用官网sha进行完整性校验 yum install -y perl-Digest-SHA shasum -a 512 xxx 校验文件
2、解压缩
tar -xvfz es.tar.gz
3、创建新用户 es,并赋予es用户解压出来的文件权限(elasticsearch-7.3.2为解压缩文件)
useradd es chown -R es:es elasticsearch-7.3.2
4、到bin目录运行脚本 或 配置bin目录到export直接运行
su es ./elasticsearch -E node.name=node0 -E cluster.name=sinfcloud -E path.data=node0_data -d
其中,node.name是节点名称,cluster.name集群名称,path.data文件存储名称
5、可能遇到错误
5.1、不能以root用户启动
"stacktrace": ["org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root", "at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-7.3.2.jar:7.3.2]", "at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-7.3.2.jar:7.3.2]", "at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-7.3.2.jar:7.3.2]"
es自带最小权限限制,不能用root用户启动,需要单独建立用户
5.2、端口占用
netstat -anp |grep 9300 -n
显示网络IP 端口 -p 显示程序PID
5.3、JDK问题
ES 5,安装需要 JDK 8 以上,ES 7.X内置了JDK 12 手动安装JDK yum install -y java-1.8.0-openjdk
5.4、文件、内存、进程数限制
5.4.1、进程数
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536] [2]: max number of threads [2048] for user [elastic] is too low, increase to at least [4096]
先切换到root账户下面，使用 vi /etc/security/limits.conf ，增加如下内容
es soft nofile 65536 es hard nofile 65536 es soft nproc 4096 es hard nproc 4096
nproc : 是操作系统级别对每个用户创建的进程数的限制
nofile : 是每个进程可以打开的文件数的限制
soft xxx : 代表警告的设定，可以超过这个设定值，但是超过后会有警告
hard xxx : 代表严格的设定，不允许超过这个设定的值
ulimit -n是设置当前shell的当前用户所有进程能打开的最大文件数量
这个文件只有root身份可以调整!
5.4.2、虚拟内存限制
[3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
使用 vim /etc/sysctl.conf ，增加如下的内容
vm.max_map_count=262144
输入：sysctl -p 保存
max_map_count文件包含限制一个进程可以拥有的VMA(虚拟内存区域)的数量。虚拟内存区域是一个连续的虚拟地址空间区域。在进程的生命周期中，每当程序尝试在内存中映射文件，链接到共享内存段，或者分配堆空间的时候，这些区域将被创建。调优这个值将限制进程可拥有VMA的数量。限制一个进程拥有VMA的总数可能导致应用程序出错，因为当进程达到了VMA上线但又只能释放少量的内存给其他的内核进程使用时，操作系统会抛出内存不足的错误。如果你的操作系统在NORMAL区域仅占用少量的内存，那么调低这个值可以帮助释放内存给内核用
5.4.3、内存
{"type": "server", "timestamp": "2020-03-05T16:55:47,618+0800", "level": "WARN", "component": "o.e.b.JNANatives", "cluster.name": "sinfcloud", "node.name": "node1", "message": "Unable to lock JVM Memory: error=12, reason=无法分配内存" } {"type": "server", "timestamp": "2020-03-05T16:55:47,622+0800", "level": "WARN", "component": "o.e.b.JNANatives", "cluster.name": "sinfcloud", "node.name": "node1", "message": "This can result in part of the JVM being swapped out." } {"type": "server", "timestamp": "2020-03-05T16:55:47,623+0800", "level": "WARN", "component": "o.e.b.JNANatives", "cluster.name": "sinfcloud", "node.name": "node1", "message": "Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536" }
先切换到root账户下面，使用 vi /etc/security/limits.conf ，增加如下内容
es soft memlock unlimited
es hard memlock unlimited
5.5、IP不能访问
启动es之后只有本机127.0.0.1可以访问,修改配置文件elasticsearch.yml里的network.host 改成服务器IP即可
5.6、索引只读错误
Elasticsearch exception [type=cluster_block_exception, reason=index [service_instance_inventory] blocked by: [FORBIDDEN/12/index read-only / allow delete (api)]
原因是可能是磁盘空间不足,es自己变为只读模式禁止写入 也可能是索引本身配置问题
cluster.routing.allocation.disk.watermark.low: 默认 85% 当达到时，replica 不再写入 cluster.routing.allocation.disk.watermark.high: 默认 90% 当达到时，shards 会尝试写入其他节点 cluster.routing.allocation.disk.watermark.flood_stage: 默认 95% 当达到时，所有索引变为 readonly状态
5.6.1、修改配置文件 elasticsearch.yml
改为cluster.routing.allocation.disk.watermark.flood_stage: 99%
5.6.2、加大磁盘空间
部署es时候 最好指定空间较大磁盘,且注意监控磁盘容量!
5.6.3、解除只读限制
curl -XPUT -H "Content-Type: application/json" http://127.0.0.1:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'
5.7、lock错误
{"type": "server", "timestamp": "2020-03-10T14:03:01,374+0800", "level": "ERROR", "component": "o.e.b.Bootstrap", "cluster.name": "sinfcloud", "node.name": "node0", "message": "Exception" , "stacktrace": ["java.lang.IllegalStateException: failed to obtain node locks, tried [[/home/es/data/node0_data]] with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?"
磁盘满了迁移到其他盘重启报lock错误,原因是es启动用户没有对迁移之后的数据文件写入权限
chown -R es [node0_数据文件]
5.8、skywalking报es错误
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index [endpoint_inventory]","resource.type":"index_expression","resource.id":"endpoint_inventory","index_uuid":"_na_","index":"endpoint_inventory"}],"type":"index_not_found_exception","reason":"no such index [endpoint_inventory]","resource.type":"index_expression","resource.id":"endpoint_inventory","index_uuid":"_na_","index":"endpoint_inventory"},"status":404}
es可以中途重启,skywalking顶多连不到数据,但是如果es删掉数据了,skywalking必须重启来构建es索引!
5.9、es分片满了
{"error":{"root_cause":[{"type":"validation_exception","reason":"Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [1999]/[2000] maximum shards open;"}],"type":"validation_exception","reason":"Validation Failed: 1: this action would add [2] total shards, but this cluster currently has [1999]/[2000] maximum shards open;"},"status":400}
从Elasticsearch v7.0.0 开始，集群中的每个节点默认限制 1000 个shard，如果你的es集群有3个数据节点，那么最多 3000 shards。48服务器部署两个节点es,所以有2000分片上限
以下API可扩大分片:
curl -XPUT -H "Content-Type: application/json" -d '{"transient":{"cluster":{"max_shards_per_node":10000}}}' 'http://localhost:9200/_cluster/settings'
transient这种参数设置是临时的,重启会失效!
curl -XPUT -H "Content-Type: application/json" -d '{"persistents":{"cluster":{"max_shards_per_node":10000}}}' 'http://localhost:9200/_cluster/settings'
6、修改内存
配置文件config/jvm.options 修改内存!