20200219 第 5 章 Kafka 监控

第 5 章 Kafka 监控

5.1 Kafka Eagle

1. 修改 kafka 启动命令

   修改 kafka-server-start.sh 命令中

   if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
   export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
   fi
   

   为

   if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
       export KAFKA_HEAP_OPTS="-server -Xms2G -Xmx2G -XX:PermSize=128m    -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:ParallelGCThreads=8 -XX:ConcGCThreads=5 -XX:InitiatingHeapOccupancyPercent=70"
       export JMX_PORT="9999"
       #export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
   fi
   

2. 上传压缩包 kafka-eagle-bin-1.3.7.tar.gz 到集群/opt/software 目录

3. 解压到本地

   [atguigu@hadoop102 software]$ tar -zxvf kafka-eagle-bin-1.3.7.tar.gz
   

4. 进入刚才解压的目录

   [atguigu@hadoop102 kafka-eagle-bin-1.3.7]$ ll
   总用量 82932
   -rw-rw-r--. 1 atguigu atguigu 84920710 8 月 13 23:00 kafka-eagleweb-1.3.7-bin.tar.gz
   

5. 将 kafka-eagle-web-1.3.7-bin.tar.gz 解压至/opt/module

   [atguigu@hadoop102 kafka-eagle-bin-1.3.7]$ tar -zxvf kafka-eagleweb-1.3.7-bin.tar.gz -C /opt/module/
   

6. 修改名称

   [atguigu@hadoop102 module]$ mv kafka-eagle-web-1.3.7/ eagle
   

7. 给启动文件执行权限

   [atguigu@hadoop102 eagle]$ cd bin/
   [atguigu@hadoop102 bin]$ ll
   总用量 12
   -rw-r--r--. 1 atguigu atguigu 1848 8 月 22 2017 ke.bat
   -rw-r--r--. 1 atguigu atguigu 7190 7 月 30 20:12 ke.sh
   [atguigu@hadoop102 bin]$ chmod 777 ke.sh
   

8. 修改配置文件 conf/system-config.properties

   ######################################
   # multi zookeeper&kafka cluster list
   ######################################
   kafka.eagle.zk.cluster.alias=cluster1
   cluster1.zk.list=hadoop102:2181,hadoop103:2181,hadoop104:2181
   ######################################
   # kafka offset storage
   ######################################
   cluster1.kafka.eagle.offset.storage=kafka
   ######################################
   # enable kafka metrics
   ######################################
   kafka.eagle.metrics.charts=true
   kafka.eagle.sql.fix.error=false
   ######################################
   # kafka jdbc driver address
   ######################################
   kafka.eagle.driver=com.mysql.jdbc.Driver
   kafka.eagle.url=jdbc:mysql://hadoop102:3306/ke?useUnicode=true&ch
   aracterEncoding=UTF-8&zeroDateTimeBehavior=convertToNull
   kafka.eagle.username=root
   kafka.eagle.password=000000
   

9. 在/etc/profile中添加环境变量

   export KE_HOME=/opt/module/eagle
   export PATH=$PATH:$KE_HOME/bin
   

   注意： source /etc/profile

10. 创建文件/opt/module/eagle/bin/ke.pid

11. 启动

    [atguigu@hadoop102 eagle]$ bin/ke.sh start
    

12. 登录页面查看监控数据