• ElasticSearch 镜像 & 安装 & 简易集群


    ES镜像

    https://mirrors.huaweicloud.com/elasticsearch/6.8.5/
    https://mirrors.huaweicloud.com/kibana/6.8.5/
    
    • ES下载中心
    https://elasticsearch.cn/download/
    找到对应版本名字,拼域名下载
    wget https://elasticsearch.thans.cn/downloads/elasticsearch/elasticsearch-5.6.16.tar.gz
    

    JDK镜像

    清华大学开源镜像库

    https://mirrors.tuna.tsinghua.edu.cn/AdoptOpenJDK/
    

    华为

    https://mirrors.huaweicloud.com/java/jdk/
    

    安装

    安装环境:Vmware Centos7.4 jdk8 elasticsearch 5.1.1

    1. 安装JDK

    export JAVA_HOME=/opt/software/jdk1.8.0_201  #jdk安装目录
    export JRE_HOME=${JAVA_HOME}/jre
    export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib:$CLASSPATH
    export JAVA_PATH=${JAVA_HOME}/bin:${JRE_HOME}/bin
    export PATH=$PATH:${JAVA_PATH}
    

    2. 解压安装ES

    tar zxf  elasticsearch-5.1.1.tar.gz -C /opt/software/
    

    3. 配置

    3.1 修改配置文件

    vim /opt/software/elasticsearch-5.1.1/config/elasticsearch.yml

    # 方便管理,自定义了数据和日志目录
    path.data: /opt/software/elasticsearch-5.1.1/data
    path.data: /opt/software/elasticsearch-5.1.1/data
    
    # 0.0.0.0为所有人可访问,默认只能本机器访问
    network.host: 0.0.0.0
    

    3.2 修改jvm参数,调小java内存,安装的环境是虚拟机,提供不了内存

    vim /opt/software/elasticsearch-5.1.1/config/jvm.options
    
    -Xms256m
    -Xmx256m
    

    4. 新建用户

    4.1 新建ES用户(5.0以后,只能用非root用户启动)

    groupadd es
    useradd es -g es
    

    4.2 文件改用户

    [root@localhost ~]# chown -R es:es /opt/software/elasticsearch-5.1.1
    [root@localhost ~]# ll /opt/software/
    total 0
    drwxr-xr-x. 9 es es  155 May  1 06:06 elasticsearch-5.1.1
    drwxr-xr-x. 7 10 143 245 Dec 16  2018 jdk1.8.0_201
    
    

    5. 启动

    切换到es用户,启动

    [root@localhost ~]# cd /opt/software/elasticsearch-5.1.1/
    [root@localhost elasticsearch-5.1.1]# su es
    [es@localhost elasticsearch-5.1.1]$ bin/elasticsearch
    
    

    成功后可以查看

    curl localhost:9200
    
    {
    "name": "node1",
    "cluster_name": "baby-application",
    "cluster_uuid": "UH40FdtFSr-DDfRv4AEZeA",
    "version": {
    "number": "5.1.1",
    "build_hash": "5395e21",
    "build_date": "2016-12-06T12:36:15.409Z",
    "build_snapshot": false,
    "lucene_version": "6.3.0"
    },
    "tagline": "You Know, for Search"
    }
    

    不出意外会报错。。。下面是踩坑

    踩坑

    1. root启用报错

    [root@localhost elasticsearch-5.1.1]# bin/elasticsearch
    [2020-05-01T06:30:12,140][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [node1] uncaught exception in thread [main]
    org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root
    	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-5.1.1.jar:5.1.1]
    	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-5.1.1.jar:5.1.1]
    	at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54) ~[elasticsearch-5.1.1.jar:5.1.1]
    	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:96) ~[elasticsearch-5.1.1.jar:5.1.1]
    	at org.elasticsearch.cli.Command.main(Command.java:62) ~[elasticsearch-5.1.1.jar:5.1.1]
    	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:89) ~[elasticsearch-5.1.1.jar:5.1.1]
    	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:82) ~[elasticsearch-5.1.1.jar:5.1.1]
    Caused by: java.lang.RuntimeException: can not run elasticsearch as root
    	at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:100) ~[elasticsearch-5.1.1.jar:5.1.1]
    	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:176) ~[elasticsearch-5.1.1.jar:5.1.1]
    	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:306) ~[elasticsearch-5.1.1.jar:5.1.1]
    	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) ~[elasticsearch-5.1.1.jar:5.1.1]
    	... 6 more
    [root@localhost elasticsearch-5.1.1]# 
    
    

    提示很明显,要用非root用户启动

    2. max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]

    2.1 打开文件数有限制,报错如下

    [2020-05-01T06:34:04,496][INFO ][o.e.n.Node               ] [node1] starting ...
    [2020-05-01T06:34:04,608][INFO ][o.e.t.TransportService   ] [node1] publish_address {192.168.1.200:9300}, bound_addresses {[::]:9300}
    [2020-05-01T06:34:04,613][INFO ][o.e.b.BootstrapCheck     ] [node1] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
    ERROR: bootstrap checks failed
    max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]
    [2020-05-01T06:34:04,629][INFO ][o.e.n.Node               ] [node1] stopping ...
    [2020-05-01T06:34:04,670][INFO ][o.e.n.Node               ] [node1] stopped
    [2020-05-01T06:34:04,670][INFO ][o.e.n.Node               ] [node1] closing ...
    [2020-05-01T06:34:04,687][INFO ][o.e.n.Node               ] [node1] closed
    
    

    2.2 修改配置,在文件末尾,End of File之前,End of File之前, End of File 之前加入配置信息,给启动es用户,调大数量。

    [root@localhost elasticsearch-5.1.1]# vim /etc/security/limits.conf
    
    #@student        -       maxlogins       4
    
    es               soft    nofile           655350
    es               hard    nofile           655350
    
    # End of file
    

    3. max virtual memory areas vm.max_map_count [2048] is too low, increase to at least [262144]

    [es@localhost elasticsearch-5.1.1]$ bin/elasticsearch
    [2020-05-01T06:38:18,677][INFO ][o.e.n.Node               ] [node1] initializing ...
    [2020-05-01T06:38:18,734][INFO ][o.e.e.NodeEnvironment    ] [node1] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [24.5gb], net total_space [26.9gb], spins? [unknown], types [rootfs]
    [2020-05-01T06:38:18,734][INFO ][o.e.e.NodeEnvironment    ] [node1] heap size [247.5mb], compressed ordinary object pointers [true]
    [2020-05-01T06:38:18,735][INFO ][o.e.n.Node               ] [node1] node name [node1], node ID [mY1O__gESOiyAFBcJOh9eA]
    [2020-05-01T06:38:18,737][INFO ][o.e.n.Node               ] [node1] version[5.1.1], pid[2150], build[5395e21/2016-12-06T12:36:15.409Z], OS[Linux/3.10.0-693.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_201/25.201-b09]
    [2020-05-01T06:38:19,329][INFO ][o.e.p.PluginsService     ] [node1] loaded module [aggs-matrix-stats]
    [2020-05-01T06:38:19,329][INFO ][o.e.p.PluginsService     ] [node1] loaded module [ingest-common]
    [2020-05-01T06:38:19,330][INFO ][o.e.p.PluginsService     ] [node1] loaded module [lang-expression]
    [2020-05-01T06:38:19,330][INFO ][o.e.p.PluginsService     ] [node1] loaded module [lang-groovy]
    [2020-05-01T06:38:19,330][INFO ][o.e.p.PluginsService     ] [node1] loaded module [lang-mustache]
    [2020-05-01T06:38:19,330][INFO ][o.e.p.PluginsService     ] [node1] loaded module [lang-painless]
    [2020-05-01T06:38:19,331][INFO ][o.e.p.PluginsService     ] [node1] loaded module [percolator]
    [2020-05-01T06:38:19,331][INFO ][o.e.p.PluginsService     ] [node1] loaded module [reindex]
    [2020-05-01T06:38:19,332][INFO ][o.e.p.PluginsService     ] [node1] loaded module [transport-netty3]
    [2020-05-01T06:38:19,332][INFO ][o.e.p.PluginsService     ] [node1] loaded module [transport-netty4]
    [2020-05-01T06:38:19,333][INFO ][o.e.p.PluginsService     ] [node1] no plugins loaded
    [2020-05-01T06:38:20,684][INFO ][o.e.n.Node               ] [node1] initialized
    [2020-05-01T06:38:20,684][INFO ][o.e.n.Node               ] [node1] starting ...
    [2020-05-01T06:38:20,788][INFO ][o.e.t.TransportService   ] [node1] publish_address {192.168.1.200:9300}, bound_addresses {[::]:9300}
    [2020-05-01T06:38:20,792][INFO ][o.e.b.BootstrapCheck     ] [node1] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
    ERROR: bootstrap checks failed
    max virtual memory areas vm.max_map_count [2048] is too low, increase to at least [262144]
    [2020-05-01T06:38:20,808][INFO ][o.e.n.Node               ] [node1] stopping ...
    [2020-05-01T06:38:20,852][INFO ][o.e.n.Node               ] [node1] stopped
    [2020-05-01T06:38:20,852][INFO ][o.e.n.Node               ] [node1] closing ...
    [2020-05-01T06:38:20,858][INFO ][o.e.n.Node               ] [node1] closed
    
    

    调大vm.max_map_count

    3.1 root用户,编辑sysctl.conf

    [root@localhost elasticsearch-5.1.1]# vim /etc/sysctl.conf
    

    最后加入

    vm.max_map_count=262144
    

    3.2 刷新配置

    [root@localhost elasticsearch-5.1.1]# sysctl -p
    vm.max_map_count = 262144
    [root@localhost elasticsearch-5.1.1]# 
    
    

    集群搭建

    在单节点成功的情况下

    1. 修改yml

    # nodename要保证不重复,例如node1,node2
    node.name: node1
    # 选其中一个节点为主节点
    node.master: true
    
    # 自动发现主机配置,填其余节点的信息
    discovery.zen.ping.unicast.hosts: ["192.168.1.201"]
    

    2. 启动双节点

    成功后,可访问

    [root@localhost ~]# curl http://localhost:9200/_cluster/health?pretty
    {
      "cluster_name" : "baby-application",
      "status" : "green",
      "timed_out" : false,
      "number_of_nodes" : 2,
      "number_of_data_nodes" : 2,
      "active_primary_shards" : 0,
      "active_shards" : 0,
      "relocating_shards" : 0,
      "initializing_shards" : 0,
      "unassigned_shards" : 0,
      "delayed_unassigned_shards" : 0,
      "number_of_pending_tasks" : 0,
      "number_of_in_flight_fetch" : 0,
      "task_max_waiting_in_queue_millis" : 0,
      "active_shards_percent_as_number" : 100.0
    }
    
  • 相关阅读:
    使用hadoop平台进行小型网站日志分析
    flume分布式日志收集系统操作
    map-reduce任务的执行流程
    linux设置定制器自动执行任务
    pig笔记
    hive操作记录
    hbase集群安装和shell操作
    Spark Streaming揭秘 Day24 Transformation和action图解
    Spark Streaming揭秘 Day23 启动关闭源码图解
    Spark Streaming揭秘 Day22 架构源码图解
  • 原文地址:https://www.cnblogs.com/AganRun/p/12811651.html
Copyright © 2020-2023  润新知