• CMAK(Kafka Manager)安装


    CMAK 是管理Kafka集群的常用工具,之前的名字叫Kafka Manager。CMAK功能很强大,它可以管理多个Kafka集群,查看集群内部状态,如:topic、broker、consumer、partition,也可用于创建topic,迁移分区等。笔者在安装CMAK过程中遇到并解决了一些问题,现记录下来分享给大家。

    1. 编译

    Linux服务器编译

    CMAK没有提供编译好的安装包,因此需要下载源码后编译出安装包。下载最新的CMAK源码并解压。

    wget https://github.com/yahoo/CMAK/archive/refs/tags/3.0.0.5.zip
    unzip CMAK-3.0.0.5.zip
    

    笔者首先尝试在Linux服务器上编译,CMAK依赖的JDK版本为Java 11+ ,因此需要先下载JDK。笔者从Oracle官网下载了JDK11(需要先注册账号,登录后才能下载),具体版本为:jdk-11.0.13_linux-x64_bin.tar.gz

    编译时需要添加java环境变量,并设置JAVA_HOME路径,参考官网提供的命令编译CMAK(JAVA的路径需要修改)。

    cd CMAK-3.0.0.5
    PATH=/usr/lib/jvm/zulu-11-amd64/bin:$PATH \
      JAVA_HOME=/usr/lib/jvm/zulu-11-amd64 \
      ./sbt -java-home /usr/lib/jvm/zulu-11-amd64 clean dist
    

    执行上面命令后,发现最终编译失败,报错信息如下:

    Cannot use JVMCI compiler: No JVMCI compiler found
    

    Mac本地编译

    查阅了相关资料,发现了在Mac上编译成功的例子(Problem in installing CMAK in Macos #849 )。因此笔者尝试在Mac上编译源码。

    首先从CMAK Github官网下载最新版源码的压缩包并解压,具体版本为:CMAK-3.0.0.5.zip, 在从Oracle官网下载JDK11并解压,具体版本为:jdk-11.0.13_osx-x64_bin.tar.gz.

    安装sbt

    brew install sbt
    

    进入到CMAK目录,执行编译命令,注意需要运行sbt clean dist,而不是./sbt clean dist,另外JDK目录修改成实际地址。

    cd CMAK-3.0.0.5
    
    PATH=jdk-11.0.13.jdk/Contents/Home/bin:$PATH \
      JAVA_HOME=jdk-11.0.13.jdk/Contents/Home \
      sbt  -java-home jdk-11.0.13.jdk/Contents/Home clean dist
    

    最后发现编译成功了,包位置:target/universal/cmak-3.0.0.5.zip

    将编译完成后的cmak包上传到Linux服务器,并解压。

    unzip cmak-3.0.0.5.zip
    

    2. 配置

    CMAK核心配置文件位置: conf/application.conf

    最少配置

    CMAK的最少配置是只需配置zk参数cmak.zkhosts,注意zk配置不能包含后缀。正确的配置示例:

    cmak.zkhosts="my.zookeeper.host.com:2181,other.zookeeper.host.com:2181"
    

    若zk配置包含后缀,如:cmak.zkhosts="my.zookeeper.host.com:2181/cmak"。那么启动cmak后会报错,报错信息如下:

    2022-03-10 17:19:52,585 - [ERROR] - from akka.actor.OneForOneStrategy in kafka-manager-system-akka.actor.default-dispatcher-3
    requirement failed: Kafka manager base path not found : /kafka-manager
    akka.actor.ActorInitializationException: akka://kafka-manager-system/user/kafka-manager: exception during creation
            at akka.actor.ActorInitializationException$.apply(Actor.scala:193)
            at akka.actor.ActorCell.create(ActorCell.scala:669)
            at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:523)
            at akka.actor.ActorCell.systemInvoke(ActorCell.scala:545)
            at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:283)
            ... 9 common frames omitted
    Caused by: java.lang.IllegalArgumentException: requirement failed: Kafka manager base path not found : /kafka-manager
            at scala.Predef$.require(Predef.scala:281)
            at kafka.manager.actor.KafkaManagerActor.<init>(KafkaManagerActor.scala:77)
            ... 18 common frames omitted
    

    报错信息显示没有找到CMAK的基路径/kafka-manager,可见CMAK需要在zk元数据中自动生成/kafka-manager路径,不能人为指定CMAK在zk中的路径。

    权限认证

    CMAK支持两种权限认证方式,一种是设置用户名和密码方式,需要设置参数basicAuthentication.enabled为true,详细配置如下:

    basicAuthentication.enabled=true
    
    basicAuthentication.username="admin"
    basicAuthentication.password="password"
    

    另外一种是LDAP方式,需要把参数basicAuthentication.enabledbasicAuthentication.ldap.enabled均设为true,在配置LDAP的相关参数。

    basicAuthentication.enabled=true
    basicAuthentication.ldap.enabled=true
    
    basicAuthentication.ldap.server=${?KAFKA_MANAGER_LDAP_SERVER}
    basicAuthentication.ldap.port=389
    basicAuthentication.ldap.username=${?KAFKA_MANAGER_LDAP_USERNAME}
    basicAuthentication.ldap.password=${?KAFKA_MANAGER_LDAP_PASSWORD}
    basicAuthentication.ldap.search-base-dn=${?KAFKA_MANAGER_LDAP_SEARCH_BASE_DN}
    basicAuthentication.ldap.search-filter=${?KAFKA_MANAGER_LDAP_SEARCH_FILTER}
    basicAuthentication.ldap.group-filter=${?KAFKA_MANAGER_LDAP_GROUP_FILTER}
    

    3. 启动

    如果服务器没有配置java路径,或者需要配置不同版本的java,那么需要添加-java-home 选项。CMAK默认配置路径为conf/application.conf, 默认端口为9000,可根据情况进行覆写。应用启动命令如下:

    cd /data0/workspace/cmak-3.0.0.5 && bin/cmak -java-home jdk-11.0.13 -Dconfig.file=conf/application.conf -Dhttp.port=9000 > /dev/null 2>&1 &
    

    服务启动后,在CMAK UI界面上添加kafka集群,发现无法添加,且出现以下报错信息:

    Yikes! KeeperErrorCode = Unimplemented for /kafka-manager/mutex
    

    查看CMAK日志,同样发现了Unimplemented for /kafka-manager/mutex 报错日志。

    2022-03-10 17:50:06,159 - [ERROR] - from kafka.manager.KafkaManager in pool-1-thread-1
    Failed on input : KMAddCluster(ClusterConfig()
    org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /kafka-manager/mutex
            at org.apache.zookeeper.KeeperException.create(KeeperException.java:106)
            at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
            at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1538)
            at org.apache.curator.utils.ZKPaths.mkdirs(ZKPaths.java:291)
            at org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:746)
    ....
    2022-03-10 17:50:06,162 - [ERROR] - from kafka.manager.ApiError$ in pool-1-thread-1
    error : KeeperErrorCode = Unimplemented for /kafka-manager/mutex
    org.apache.zookeeper.KeeperException$UnimplementedException: KeeperErrorCode = Unimplemented for /kafka-manager/mutex
            at org.apache.zookeeper.KeeperException.create(KeeperException.java:106)
            at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
            at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1538)
            at org.apache.curator.utils.ZKPaths.mkdirs(ZKPaths.java:291)
    ...
    

    通过查阅资料(https://github.com/yahoo/CMAK/issues/731#issuecomment-643880544), 发现需要在ZK集群上手动创建mutex目录,即可在CMAK UI界面添加Kafka集群。

    # ./bin/zkCli.sh
    [zk: localhost:2181(CONNECTED) 2] ls /kafka-manager
    [configs, deleteClusters, clusters]
    [zk: localhost:2181(CONNECTED) 3] create /kafka-manager/mutex ""
    Created /kafka-manager/mutex
    [zk: localhost:2181(CONNECTED) 5] create /kafka-manager/mutex/locks ""
    Created /kafka-manager/mutex/locks
    [zk: localhost:2181(CONNECTED) 6] create /kafka-manager/mutex/leases ""
    Created /kafka-manager/mutex/leases
    

    向zk集群创建目录/kafka-manager/mutex,/kafka-manager/mutex/locks, /kafka-manager/mutex/leases 后,重启CMAK,发现可以正常添加kafka集群了。

    4. 参考资料

    1. CMAK Github官网

    2. Problem in installing CMAK in Macos #849

    3. Failed to add cluster on 3.0.0.1 #731

  • 相关阅读:
    linux tomcat 突然验证码出不来
    使用open live writer客户端写博客
    创建自己的maven模板
    Dynamic Web Module 3.0 requires Java 1.6 or newer
    win10 操作配置备忘
    Maven使用
    ORA-12514: TNS:listener does not currently know of service …
    PlantUML——4.实例演示1
    C语言基础(一)
    Linux系统挂载FAT32的U盘
  • 原文地址:https://www.cnblogs.com/ljhbjehp/p/15999283.html
Copyright © 2020-2023  润新知