使用Metricbeat监控zookeeper遇到的问题

1.metricbeat中启动自动加载模块

metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true
  reload.period: 10s

2.开启kibana中的dashboard功能

setup.dashboards.enabled: true

然后启动metricbeat

3.启动zookeeper模块

./metricbeat modules enable zookeeper

会看到目录modules.d下的zookeeper.yml.disabled变成zookeeper.yml

4.编辑zookeeper.yml,配置上监听的zookeeper地址

- module: zookeeper
  metricsets:
    - mntr
    - server
  period: 10s
  hosts: ["172.17.107.187:2181","172.17.107.187:2182","172.17.107.187:2183"]

5.在kibana web界面创建索引metricbeat-7.3.0*模式

6.在kibana左侧导航仪表盘搜索zookeeper,出现如下信息，点开查看：

注意：若是出现如下之类的错误提示,可以按照文章(https://www.cnblogs.com/sanduzxcvbnm/p/12172935.html)说的方法进行修改

[esaggs] > Request to Elasticsearch failed: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [zookeeper.server.version_date] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"metricbeat-7.3.0","node":"VD2R4MIiQEOh1rlL5YU2ow","reason":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [zookeeper.server.version_date] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [zookeeper.server.version_date] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.","caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [zookeeper.server.version_date] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}},"status":400}

7.但是在查看仪表盘中的[Metricbeat System] Overview ECS时出现的问题，却不知道要如何解决？
问题如下：

• 当刷新这个仪表盘界面时会出现如下的提示：

"reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [zookeeper.server.version_date

但是按照上一步说的执行如下命令后：

PUT metricbeat-7.3.0/_mapping
{
  "properties": {
    "zookeeper": {
      "properties": {
        "server": {
            "properties": {
                "version_date":{
                    "type": "text",
                    "fielddata": true,
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 256
                        }
                    }
                }
            }
        }
      }
    }
  }
}

这个错误不会出现了，但是会出现另一个错误：

"type":"illegal_argument_exception","reason":"Field [zookeeper.server.version_date] of type [text] does not support custom formats"

究其原因就是 [zookeeper.server.version_date] 的类型问题
官方给的该字段的参考值如下：

官方给的该字段的类型是：

创建索引时生成的mapping中该字段类型是text。
仪表盘中会使用到该字段进行聚合查询，因为默认的text类型没有开启fielddata=true。
开启后但是会出现字段类型的错误。

修改字段类型是date,这一步是无法进行操作的,报错：

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "mapper [zookeeper.server.version_date] of different type, current_type [text], merged_type [date]"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "mapper [zookeeper.server.version_date] of different type, current_type [text], merged_type [date]"
  },
  "status": 400
}

修改字段类型是date,并且开启fielddata=true。这一步也是无法进行操作的，报错：

{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "Mapping definition for [version_date] has unsupported parameters:  [fielddata : true]"
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "Mapping definition for [version_date] has unsupported parameters:  [fielddata : true]"
  },
  "status": 400
}

能做的操作就是给text类型开启fielddata=true。

感觉陷入死循环了，不修改不行，修改也不行。。。