• ES 冷热节点部署三

    目录

    ▪ 用途 ▪ 架构 ▪ 192.168.1.51 elasticsearch-data部署双实例 ▪ 192.168.1.52 elasticsearch-data部署双实例 ▪ 192.168.1.53 elasticsearch-data部署双实例 ▪ 测试

    用途

    前情提要：

    ▷ 在第一篇《EFK教程 - 快速入门指南》中，阐述了EFK的安装部署，其中ES的架构为三节点，即master、ingest、data角色同时部署在三台服务器上。 ▷ 在第二篇《EFK教程 - ElasticSearch高性能高可用架构》中，阐述了EFK的data/ingest/master角色的用途及分别部署三节点，在实现性能最大化的同时保障高可用

    前两篇文章，ES集群中只存在一个实例，而在本文中，将在一个集群中部署多个ES实例，来实现资源合理分配。例如data服务器存在SSD与SAS硬盘，可以将热数据存放到SSD，而冷数据存放到SAS，实现数据冷热分离。

    在本文中，将为data服务器创建2个实例，分别基于SSD和基于SAS硬盘，将nginx的9月份索引放在SAS盘上，其它的全放在SSD盘上

    架构

    架构图

    图片

    服务器配置

    192.168.1.51 elasticsearch-data部署双实例

    索引迁移（此步不能忽略）：将192.168.1.51上的索引放到其它2台data节点上

    1. curl -X PUT "192.168.1.31:9200/*/_settings?pretty" -H 'Content-Type: application/json' -d'

    2. {

    3. "index.routing.allocation.include._ip": "192.168.1.52,192.168.1.53"

    4. }'

    确认当前索引存储位置：确认所有索引不在192.168.1.51节点上

    1. curl "http://192.168.1.31:9200/_cat/shards?h=n"

    停掉192.168.1.51的进程，修改目录结构及配置：请自行按SSD和SAS硬盘挂载好数据盘

    图片

    1. # 安装包下载和部署请参考第一篇《EFK教程 - 快速入门指南》

    2. cd /opt/software/

    3. tar -zxvf elasticsearch-7.3.2-linux-x86_64.tar.gz

    4. mv /opt/elasticsearch /opt/elasticsearch-SAS

    5. mv elasticsearch-7.3.2 /opt/

    6. mv /opt/elasticsearch-7.3.2 /opt/elasticsearch-SSD

    7. chown elasticsearch.elasticsearch /opt/elasticsearch-* -R

    8. rm -rf /data/SAS/*

    9. chown elasticsearch.elasticsearch /data/* -R

    10. mkdir -p /opt/logs/elasticsearch-SAS

    11. mkdir -p /opt/logs/elasticsearch-SSD

    12. chown elasticsearch.elasticsearch /opt/logs/* -R

    SAS实例/opt/elasticsearch-SAS/config/elasticsearch.yml配置

    1. cluster.name: my-application

    2. node.name: 192.168.1.51-SAS

    3. path.data: /data/SAS

    4. path.logs: /opt/logs/elasticsearch-SAS

    5. network.host: 192.168.1.51

    7. http.port: 9200

    8. transport.port: 9300

    9. # discovery.seed_hosts和cluster.initial_master_nodes 一定要带上端口号，不然会走http.port和transport.port端口

    10. discovery.seed_hosts: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]

    11. cluster.initial_master_nodes: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]

    12. http.cors.enabled: true

    13. http.cors.allow-origin: "*"

    15. node.master: false

    16. node.ingest: false

    17. node.data: true

    19. # 本机只允行启2个实例

    20. node.max_local_storage_nodes: 2

    SSD实例/opt/elasticsearch-SSD/config/elasticsearch.yml配置

    1. cluster.name: my-application

    2. node.name: 192.168.1.51-SSD

    3. path.data: /data/SSD

    4. path.logs: /opt/logs/elasticsearch-SSD

    5. network.host: 192.168.1.51

    7. http.port: 9201

    8. transport.port: 9301

    9. # discovery.seed_hosts和cluster.initial_master_nodes 一定要带上端口号，不然会走http.port和transport.port端口

    10. discovery.seed_hosts: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]

    11. cluster.initial_master_nodes: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]

    12. http.cors.enabled: true

    13. http.cors.allow-origin: "*"

    15. node.master: false

    16. node.ingest: false

    17. node.data: true

    19. # 本机只允行启2个实例

    20. node.max_local_storage_nodes: 2

    SAS实例和SSD实例启动方式

    1. sudo -u elasticsearch /opt/elasticsearch-SAS/bin/elasticsearch

    2. sudo -u elasticsearch /opt/elasticsearch-SSD/bin/elasticsearch

    确认SAS和SSD已启2实例

    1. curl "http://192.168.1.31:9200/_cat/nodes?v"

    图片

    192.168.1.52 elasticsearch-data部署双实例

    索引迁移（此步不能忽略）：将192.168.1.52上的索引放到其它2台data节点上

    1. curl -X PUT "192.168.1.31:9200/*/_settings?pretty" -H 'Content-Type: application/json' -d'

    2. {

    3. "index.routing.allocation.include._ip": "192.168.1.51,192.168.1.53"

    4. }'

    确认当前索引存储位置：确认所有索引不在192.168.1.52节点上

    1. curl "http://192.168.1.31:9200/_cat/shards?h=n"

    停掉192.168.1.52的进程，修改目录结构及配置：请自行按SSD和SAS硬盘挂载好数据盘

    1. # 安装包下载和部署请参考第一篇《EFK教程 - 快速入门指南》

    2. cd /opt/software/

    3. tar -zxvf elasticsearch-7.3.2-linux-x86_64.tar.gz

    4. mv /opt/elasticsearch /opt/elasticsearch-SAS

    5. mv elasticsearch-7.3.2 /opt/

    6. mv /opt/elasticsearch-7.3.2 /opt/elasticsearch-SSD

    7. chown elasticsearch.elasticsearch /opt/elasticsearch-* -R

    8. rm -rf /data/SAS/*

    9. chown elasticsearch.elasticsearch /data/* -R

    10. mkdir -p /opt/logs/elasticsearch-SAS

    11. mkdir -p /opt/logs/elasticsearch-SSD

    12. chown elasticsearch.elasticsearch /opt/logs/* -R

    SAS实例/opt/elasticsearch-SAS/config/elasticsearch.yml配置

    1. cluster.name: my-application

    2. node.name: 192.168.1.52-SAS

    3. path.data: /data/SAS

    4. path.logs: /opt/logs/elasticsearch-SAS

    5. network.host: 192.168.1.52

    7. http.port: 9200

    8. transport.port: 9300

    9. # discovery.seed_hosts和cluster.initial_master_nodes 一定要带上端口号，不然会走http.port和transport.port端口

    10. discovery.seed_hosts: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]

    11. cluster.initial_master_nodes: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]

    12. http.cors.enabled: true

    13. http.cors.allow-origin: "*"

    15. node.master: false

    16. node.ingest: false

    17. node.data: true

    19. # 本机只允行启2个实例

    20. node.max_local_storage_nodes: 2

    SSD实例/opt/elasticsearch-SSD/config/elasticsearch.yml配置

    1. cluster.name: my-application

    2. node.name: 192.168.1.52-SSD

    3. path.data: /data/SSD

    4. path.logs: /opt/logs/elasticsearch-SSD

    5. network.host: 192.168.1.52

    7. http.port: 9201

    8. transport.port: 9301

    9. # discovery.seed_hosts和cluster.initial_master_nodes 一定要带上端口号，不然会走http.port和transport.port端口

    10. discovery.seed_hosts: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]

    11. cluster.initial_master_nodes: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]

    12. http.cors.enabled: true

    13. http.cors.allow-origin: "*"

    15. node.master: false

    16. node.ingest: false

    17. node.data: true

    19. # 本机只允行启2个实例

    20. node.max_local_storage_nodes: 2

    SAS实例和SSD实例启动方式

    1. sudo -u elasticsearch /opt/elasticsearch-SAS/bin/elasticsearch

    2. sudo -u elasticsearch /opt/elasticsearch-SSD/bin/elasticsearch

    确认SAS和SSD已启2实例

    1. curl "http://192.168.1.31:9200/_cat/nodes?v"

    图片

    192.168.1.53 elasticsearch-data部署双实例

    索引迁移（此步不能忽略）：一定要做这步，将192.168.1.53上的索引放到其它2台data节点上

    1. curl -X PUT "192.168.1.31:9200/*/_settings?pretty" -H 'Content-Type: application/json' -d'

    2. {

    3. "index.routing.allocation.include._ip": "192.168.1.51,192.168.1.52"

    4. }'

    确认当前索引存储位置：确认所有索引不在192.168.1.52节点上

    1. curl "http://192.168.1.31:9200/_cat/shards?h=n"

     

    停掉192.168.1.53的进程，修改目录结构及配置：请自行按SSD和SAS硬盘挂载好数据盘

    1. # 安装包下载和部署请参考第一篇《EFK教程 - 快速入门指南》

    2. cd /opt/software/

    3. tar -zxvf elasticsearch-7.3.2-linux-x86_64.tar.gz

    4. mv /opt/elasticsearch /opt/elasticsearch-SAS

    5. mv elasticsearch-7.3.2 /opt/

    6. mv /opt/elasticsearch-7.3.2 /opt/elasticsearch-SSD

    7. chown elasticsearch.elasticsearch /opt/elasticsearch-* -R

    8. rm -rf /data/SAS/*

    9. chown elasticsearch.elasticsearch /data/* -R

    10. mkdir -p /opt/logs/elasticsearch-SAS

    11. mkdir -p /opt/logs/elasticsearch-SSD

    12. chown elasticsearch.elasticsearch /opt/logs/* -R

    SAS实例/opt/elasticsearch-SAS/config/elasticsearch.yml配置

    1. cluster.name: my-application

    2. node.name: 192.168.1.53-SAS

    3. path.data: /data/SAS

    4. path.logs: /opt/logs/elasticsearch-SAS

    5. network.host: 192.168.1.53

    7. http.port: 9200

    8. transport.port: 9300

    9. # discovery.seed_hosts和cluster.initial_master_nodes 一定要带上端口号，不然会走http.port和transport.port端口

    10. discovery.seed_hosts: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]

    11. cluster.initial_master_nodes: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]

    12. http.cors.enabled: true

    13. http.cors.allow-origin: "*"

    15. node.master: false

    16. node.ingest: false

    17. node.data: true

    19. # 本机只允行启2个实例

    20. node.max_local_storage_nodes: 2

    SSD实例/opt/elasticsearch-SSD/config/elasticsearch.yml配置

    1. cluster.name: my-application

    2. node.name: 192.168.1.53-SSD

    3. path.data: /data/SSD

    4. path.logs: /opt/logs/elasticsearch-SSD

    5. network.host: 192.168.1.53

    7. http.port: 9201

    8. transport.port: 9301

    9. # discovery.seed_hosts和cluster.initial_master_nodes 一定要带上端口号，不然会走http.port和transport.port端口

    10. discovery.seed_hosts: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]

    11. cluster.initial_master_nodes: ["192.168.1.31:9300","192.168.1.32:9300","192.168.1.33:9300"]

    12. http.cors.enabled: true

    13. http.cors.allow-origin: "*"

    15. node.master: false

    16. node.ingest: false

    17. node.data: true

    19. # 本机只允行启2个实例

    20. node.max_local_storage_nodes: 2

    SAS实例和SSD实例启动方式

    1. sudo -u elasticsearch /opt/elasticsearch-SAS/bin/elasticsearch

    2. sudo -u elasticsearch /opt/elasticsearch-SSD/bin/elasticsearch

    确认SAS和SSD已启2实例

    1. curl "http://192.168.1.31:9200/_cat/nodes?v"

    图片

    测试

    将所有索引移到SSD硬盘上

    1. # 下面的参数会在后面的文章讲解，此处照抄即可

    2. curl -X PUT "192.168.1.31:9200/*/_settings?pretty" -H 'Content-Type: application/json' -d'

    3. {

    4. "index.routing.allocation.include._host_ip": "",

    5. "index.routing.allocation.include._host": "",

    6. "index.routing.allocation.include._name": "",

    7. "index.routing.allocation.include._ip": "",

    8. "index.routing.allocation.require._name": "*-SSD"

    9. }'

    确认所有索引全在SSD硬盘上

    1. curl "http://192.168.1.31:9200/_cat/shards?h=n"

    将nginx9月份的日志索引迁移到SAS硬盘上

    1. curl -X PUT "192.168.1.31:9200/nginx_*_2019.09/_settings?pretty" -H 'Content-Type: application/json' -d'

    2. {

    3. "index.routing.allocation.require._name": "*-SAS"

    4. }'

    确认nginx9月份的日志索引迁移到SAS硬盘上

    1. curl "http://192.168.1.31:9200/_cat/shards"

     转载自小漫哥公众号

    螃蟹在剥我的壳，笔记本在写我，漫天的我落在枫叶上雪花上，而你在想我。 --章怀柔
  • 相关阅读:
    腾讯微博
    城市左右选择添加按钮案例
    jQuery元素操作1
    动态创建表格
    五角星评论案例
    点击图片箭头回到顶部案例
    HDU1506： Largest Rectangle in a Histogram(最大子矩阵，好题动态优化左右边界）
    HDU1165： Eddy's research II（递推）
    HDU1158：Employment Planning（线性dp)
    HDU1081：To The Max（最大子矩阵，线性DP）
  • 原文地址：https://www.cnblogs.com/lovezhr/p/15034103.html
Copyright © 2020-2023  润新知