• ElasticSearch 命令行管理工具Curator


    一、背景


    elastic官网现在已经大面积升级到了5.x版本,然而针对elasticsearch的命令行管理工具curator现在仍然是4.0版本。

    刚开始找到此工具,深深的怕因为版本更迭无法使用,还好在试用之后,顾虑烟消云散。

    • 应用目的

    我主要用到了它删除数据的功能。因为直接通过curl命令删除数据需要输入密码,然而在利用<< EOF的方式也无法绕过密码校验。

    懒得自己写python脚本,既然已经有了对应工具,何不马上用起来。

     

    二、安装


    1、Signing Keyedit

    Download and install the public signing key:

    rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch

    2、Repository Configurationedit

    Add the following in your /etc/yum.repos.d/ directory in a file with a .repo suffix, for example curator.repo

    Warning
    The repositories are different for CentOS/RHEL 6 and 7 due to library and path differences. Be sure to use the correct version for your system!

    RHEL/CentOS 6
    [curator-4]
    name=CentOS/RHEL 6 repository for Elasticsearch Curator 4.x packages
    baseurl=http://packages.elastic.co/curator/4/centos/6
    gpgcheck=1
    gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
    enabled=1
    RHEL/CentOS 7
    [curator-4]
    name=CentOS/RHEL 7 repository for Elasticsearch Curator 4.x packages
    baseurl=http://packages.elastic.co/curator/4/centos/7
    gpgcheck=1
    gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
    enabled=1

    3、Binary Package Installationedit

    Starting with version 4.1.2, Curator comes with a binary version. What this really means is that the source is compiled, and all required libraries are bundled with the curator binary, so there are no conflicts.

    There are separate binary packages for RedHat variants. The binary packages resulting from the creation process have been tested on CentOS 6 & 7, with a different binary for each. They may work on similar variants and/or derivatives, but they have not been tested.

    yum install elasticsearch-curator

    This will install the necessary files into /opt/elasticsearch-curator and make a symlink at /usr/bin/curator that points to the curator binary in the aforementioned directory.

    三、配置


    运行报错
    curator_cli     
    Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment.  Either run this under Python 2 or consult http://click.pocoo.org/python3/ for mitigation steps.
    This system lists a couple of UTF-8 supporting locales that
    you can pick from.  The following suitable locales where
    discovered: aa_DJ.utf8, aa_ER.utf8, aa_ET.utf8, af_ZA.utf8, am_ET.utf8, an_ES.utf8, ar_AE.utf8, ar_BH.utf8, ar_DZ.utf8, ar_EG.utf8, ar_IN.utf8, ar_IQ.utf8, ar_JO.utf8, ar_KW.utf8, ar_LB.utf8, ar_LY.utf8, ar_MA.utf8, ar_OM.utf8, ar_QA.utf8, ar_SA.utf8, ar_SD.utf8, ar_SY.utf8, ar_TN.utf8, ar_YE.utf8, as_IN.utf8, ast_ES.utf8, ayc_PE.utf8, az_AZ.utf8, be_BY.utf8, bem_ZM.utf8, ber_DZ.utf8, ber_MA.utf8, bg_BG.utf8, bho_IN.utf8, bn_BD.utf8, bn_IN.utf8, bo_CN.utf8, bo_IN.utf8, br_FR.utf8, brx_IN.utf8, bs_BA.utf8, byn_ER.utf8, ca_AD.utf8, ca_ES.utf8, ca_FR.utf8, ca_IT.utf8, crh_UA.utf8, cs_CZ.utf8, csb_PL.utf8, cv_RU.utf8, cy_GB.utf8, da_DK.utf8, de_AT.utf8, de_BE.utf8, de_CH.utf8, de_DE.utf8, de_LU.utf8, doi_IN.utf8, dv_MV.utf8, dz_BT.utf8, el_CY.utf8, el_GR.utf8, en_AG.utf8, en_AU.utf8, en_BW.utf8, en_CA.utf8, en_DK.utf8, en_GB.utf8, en_HK.utf8, en_IE.utf8, en_IN.utf8, en_NG.utf8, en_NZ.utf8, en_PH.utf8, en_SG.utf8, en_US.utf8, en_ZA.utf8, en_ZM.utf8, en_ZW.utf8, es_AR.utf8, es_BO.utf8, es_CL.utf8, es_CO.utf8, es_CR.utf8, es_CU.utf8, es_DO.utf8, es_EC.utf8, es_ES.utf8, es_GT.utf8, es_HN.utf8, es_MX.utf8, es_NI.utf8, es_PA.utf8, es_PE.utf8, es_PR.utf8, es_PY.utf8, es_SV.utf8, es_US.utf8, es_UY.utf8, es_VE.utf8, et_EE.utf8, eu_ES.utf8, fa_IR.utf8, ff_SN.utf8, fi_FI.utf8, fil_PH.utf8, fo_FO.utf8, fr_BE.utf8, fr_CA.utf8, fr_CH.utf8, fr_FR.utf8, fr_LU.utf8, fur_IT.utf8, fy_DE.utf8, fy_NL.utf8, ga_IE.utf8, gd_GB.utf8, gez_ER.utf8, gez_ET.utf8, gl_ES.utf8, gu_IN.utf8, gv_GB.utf8, ha_NG.utf8, he_IL.utf8, hi_IN.utf8, hne_IN.utf8, hr_HR.utf8, hsb_DE.utf8, ht_HT.utf8, hu_HU.utf8, hy_AM.utf8, ia_FR.utf8, id_ID.utf8, ig_NG.utf8, ik_CA.utf8, is_IS.utf8, it_CH.utf8, it_IT.utf8, iu_CA.utf8, iw_IL.utf8, ja_JP.utf8, ka_GE.utf8, kk_KZ.utf8, kl_GL.utf8, km_KH.utf8, kn_IN.utf8, ko_KR.utf8, kok_IN.utf8, ks_IN.utf8, ku_TR.utf8, kw_GB.utf8, ky_KG.utf8, lb_LU.utf8, lg_UG.utf8, li_BE.utf8, li_NL.utf8, lij_IT.utf8, lo_LA.utf8, lt_LT.utf8, lv_LV.utf8, mag_IN.utf8, mai_IN.utf8, mg_MG.utf8, mhr_RU.utf8, mi_NZ.utf8, mk_MK.utf8, ml_IN.utf8, mn_MN.utf8, mni_IN.utf8, mr_IN.utf8, ms_MY.utf8, mt_MT.utf8, my_MM.utf8, nb_NO.utf8, nds_DE.utf8, nds_NL.utf8, ne_NP.utf8, nhn_MX.utf8, niu_NU.utf8, niu_NZ.utf8, nl_AW.utf8, nl_BE.utf8, nl_NL.utf8, nn_NO.utf8, nr_ZA.utf8, nso_ZA.utf8, oc_FR.utf8, om_ET.utf8, om_KE.utf8, or_IN.utf8, os_RU.utf8, pa_IN.utf8, pa_PK.utf8, pap_AN.utf8, pl_PL.utf8, ps_AF.utf8, pt_BR.utf8, pt_PT.utf8, ro_RO.utf8, ru_RU.utf8, ru_UA.utf8, rw_RW.utf8, sa_IN.utf8, sat_IN.utf8, sc_IT.utf8, sd_IN.utf8, se_NO.utf8, shs_CA.utf8, si_LK.utf8, sid_ET.utf8, sk_SK.utf8, sl_SI.utf8, so_DJ.utf8, so_ET.utf8, so_KE.utf8, so_SO.utf8, sq_AL.utf8, sq_MK.utf8, sr_ME.utf8, sr_RS.utf8, ss_ZA.utf8, st_ZA.utf8, sv_FI.utf8, sv_SE.utf8, sw_KE.utf8, sw_TZ.utf8, szl_PL.utf8, ta_IN.utf8, ta_LK.utf8, te_IN.utf8, tg_TJ.utf8, th_TH.utf8, ti_ER.utf8, ti_ET.utf8, tig_ER.utf8, tk_TM.utf8, tl_PH.utf8, tn_ZA.utf8, tr_CY.utf8, tr_TR.utf8, ts_ZA.utf8, tt_RU.utf8, ug_CN.utf8, uk_UA.utf8, unm_US.utf8, ur_IN.utf8, ur_PK.utf8, ve_ZA.utf8, vi_VN.utf8, wa_BE.utf8, wae_CH.utf8, wal_ET.utf8, wo_SN.utf8, xh_ZA.utf8, yi_US.utf8, yo_NG.utf8, yue_HK.utf8, zh_CN.utf8, zh_HK.utf8, zh_SG.utf8, zh_TW.utf8, zu_ZA.utf8
     
    When used with Python 3 (and the DEB and RPM packages of Curator are compiled
    and bundled with Python 3), Curator requires the locale to be unicode. Any of
    the above unicode definitions are acceptable.
    To set the locale to be unicode, try:
    $ export LC_ALL=en_US.utf8
    $ curator_cli [ARGS]
    Alternately, you should be able to specify the locale on the command-line:
    $ LC_ALL=en_US.utf8 curator_cli [ARGS]
    Be sure to substitute your unicode variant for en_US.utf8
    配置修正
    vim /etc/profile
    #添加以下内容
    export LC_ALL=en_US.utf-8
    export LANG=en_US.utf-8
     
    source /etc/profile
     

    四、应用


    主要利用其删除功能,先展示下其命令行功能表:

    $ curator_cli --help
    Usage: curator_cli [OPTIONS] COMMAND [ARGS]...
     
    Options:
      --config PATH       Path to configuration file. Default:
                          ~/.curator/curator.yml
      --host TEXT         Elasticsearch host.
      --url_prefix TEXT   Elasticsearch http url prefix.
      --port TEXT         Elasticsearch port.
      --use_ssl           Connect to Elasticsearch through SSL.
      --certificate TEXT  Path to certificate to use for SSL validation.
      --client-cert TEXT  Path to file containing SSL certificate for client auth.
      --client-key TEXT   Path to file containing SSL key for client auth.
      --ssl-no-validate   Do not validate SSL certificate
      --http_auth TEXT    Use Basic Authentication ex: user:pass
      --timeout INTEGER   Connection timeout in seconds.
      --master-only       Only operate on elected master node.
      --dry-run           Do not perform any changes.
      --loglevel TEXT     Log level
      --logfile TEXT      log file
      --logformat TEXT    Log output format [default|logstash|json].
      --version           Show the version and exit.
      --help              Show this message and exit.
     
    Commands:
      allocation        Shard Routing Allocation
      close             Close indices
      delete_indices    Delete indices
      delete_snapshots  Delete snapshots
      forcemerge        forceMerge index/shard segments
      open              Open indices
      replicas          Change replica count
      show_indices      Show indices
      show_snapshots    Show snapshots
      snapshot          Snapshot indices

    我用到的删除命令示例:

    #!/bin/bash

    dy=`date -d"1 weeks ago" +%Y.%m.%d`
    echo "start deleting elasticsearch index by $dy"
    curator_cli --host "127.0.0.1" --port "9200" --http_auth elastic:gl2016 delete_indices --filter_list "{"filtertype":"pattern","kind":"regex","value":"logstash-(.+)-$dy"}" >> /tmp/curator.log





  • 相关阅读:
    读书记录
    npm install 安装nodesass报错,因为nodejs版本的问题
    jeecgboot修改前端vue页⾯包括vue页⾯传递参数和获取参数
    dedecms织梦调用日期格式大全
    JS数组中every(),filter(),forEach(),map(),some()方法学习笔记!转
    Where条件的in里面放太多数据导致很慢
    谷歌浏览器 F12打开控制台面板缓慢问题
    linux sqlserver数据库连接问题The server selected protocol version TLS10 is not accepted by client preferences[TLS12]
    使用Java语言搭建一个简易的局域网直播(live)系统
    SpringBoot整合aspectj实现面向切面编程(即AOP)
  • 原文地址:https://www.cnblogs.com/snifferhu/p/6196662.html
Copyright © 2020-2023  润新知