zeebe 集成elasticsearch exporter

zeebe 目前还在一直的开发中，同时一些变动还是挺大的，比如simple monitor 的以前是不需要配置HazelcastExporter的
估计是为了进行集群功能处理，新添加的，以前写的配置基本都没法使用了，所以写了一个新的运行配置

说明运行的时候注意版本，我测试的是Zeebe 0.17.0，可能其他版本有变动

环境准备

• docker-compose 文件

  es 使用aws 开源的，同时禁用了ssl，使用http 请求

version: "3"
services:
  db:
    image: oscarfonts/h2
    container_name: zeebe_db
    ports:
      - "1521:1521"
      - "81:81"
  monitor:
    image: camunda/zeebe-simple-monitor
    environment:
      - spring.datasource.url=jdbc:h2:tcp://db:1521/zeebe-monitor
      - io.zeebe.monitor.connectionString=app:26500
      - io.zeebe.monitor.hazelcast.connection=app:5701
    ports:
    - "8080:8080"
  app:
    image: camunda/zeebe
    volumes:
      - ./zeebe-simple-monitor-exporter-0.13.0.jar:/usr/local/zeebe/lib/zeebe-simple-monitor-exporter-0.13.0.jar
      - ./zeebe-hazelcast-exporter-0.2.0.jar:/usr/local/zeebe/lib/zeebe-hazelcast-exporter-0.2.0.jar
      - ./zeebe.cfg.toml:/usr/local/zeebe/conf/zeebe.cfg.toml
    ports:
    - "26500:26500"
    - "26501:26501"
    - "26502:26502"
    - "26503:26503"
    - "26504:26504"
    - "5701:5701"
  odfe-node1:
    image: amazon/opendistro-for-elasticsearch:0.8.0
    container_name: odfe-node1
    environment:
      - opendistro_security.ssl.http.enabled=false
      - cluster.name=odfe-cluster
      - bootstrap.memory_lock=true # along with the memlock settings below, disables swapping
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m" # minimum and maximum Java heap size, recommend setting both to 50% of system RAM
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./odfe-data1:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
      - 9600:9600 # required for Performance Analyzer
  odfe-node2:
    image: amazon/opendistro-for-elasticsearch:0.8.0
    container_name: odfe-node2
    environment:
      - opendistro_security.ssl.http.enabled=false
      - cluster.name=odfe-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - discovery.zen.ping.unicast.hosts=odfe-node1
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - ./odfe-data2:/usr/share/elasticsearch/data
  kibana:
    image: amazon/opendistro-for-elasticsearch-kibana:0.8.0
    container_name: odfe-kibana
    ports:
      - 5601:5601
    expose:
      - "5601"
    environment:
      ELASTICSEARCH_URL: http://odfe-node1:9200

• 几个exporter 下载

  写好了一个简单的shell，执行就可以了，或者clone github 代码 camunda-zeebe-es-exporter-demo

#!/bin/bash
wget https://github.com/zeebe-io/zeebe-simple-monitor/releases/download/0.13.0/zeebe-simple-monitor-exporter-0.13.0.jar
wget https://github.com/zeebe-io/zeebe-hazelcast-exporter/releases/download/0.2.0/zeebe-hazelcast-exporter-0.2.0.jar

• zeebe 配置

  主要是几个exporter 的配置,es 以及simple monitor 的,配置有点多，只写添加的，完整参考githubcamunda-zeebe-es-exporter-demo

[[exporters]]
id = "simple-monitor"
className = "io.zeebe.monitor.SimpleMonitorExporter"
[exporters.args]
  jdbcUrl = "jdbc:h2:tcp://db:1521/zeebe-monitor"
  driverName = "org.h2.Driver"
  userName = "sa"
  password = ""
[[exporters]]
 id = "hazelcast"
 className = "io.zeebe.hazelcast.exporter.HazelcastExporter"
[[exporters]]
id = "elasticsearch"
className = "io.zeebe.exporter.ElasticsearchExporter"
[exporters.args]
  url = "http://odfe-node1:9200"
 [exporters.args.bulk]
 delay = 5
 size = 1_000

 [exporters.args.authentication] 
   username = "admin"
   password = "admin"
 [exporters.args.index]
  prefix = "zeebe-record"

  ## 这个参数挺重要，设置为true，有异常，false 是可以的，应该和es版本有关系
  createTemplate = false
  command = false
  event = true
  rejection = false
  deployment = true
  incident = true
  job = true
  message = false
  messageSubscription = false
  raft = false
  workflowInstance = true
  workflowInstanceSubscription = false

启动&&测试

• 启动

docker-compose up -d

• 几个问题
  我们需要配置宿主机几个参数
  vm.max_map_count：
  /etc/sysctl.conf 添加
  vm.max_map_count=262144 生效 sysctl -p
  容器启动失败，可能需要设置挂载目录权限，简单的方法是 chmod 777 odfe-data1 odfe-data2
• 效果
  es 集群

monitor

• 创建一个部署

  github 代码中包含一个简单的bpmn 的文件，是一个简单的流程

运行实例效果

es 集群数据，deploy

es 集群数据，job

说明

这个只是简单的测试，zeebe 的设计还是很灵活的，团队的规划是q2 可以产品可用，还是很期待的，实际上使用zeebe 做为我们微服务中的一个
编排工具，还是很强大的，而且集成opendistro for es 强大的功能，我们可以完成好多方便的应用开发

参考资料

https://github.com/zeebe-io/zeebe
https://github.com/rongfengliang/camunda-zeebe-es-exporter-demo
https://github.com/zeebe-io/zeebe-simple-monitor
https://github.com/zeebe-io/zeebe-hazelcast-exporter
https://opendistro.github.io/for-elasticsearch-docs/docs/install/