• kafka修改指定消费组的偏移量


    kafka针对每个消费组的消费信息都在zookeeper中做了记录,启动一个zookeeper客户端,以下可以看出kafka每个消费组在zookeeper中的消费每个kafka主题的信息
    [zk: localhost:2181(CONNECTED) 0] ls /consumers/kafkawarning0
    
    kafkawarning02 (消费组groupid)   kafkawarning01
    [zk: localhost:2181(CONNECTED) 0] ls /consumers/kafkawarning01/offsets
    [avro-log-bj-jingnei1-dev-proxy-websense]
    [zk: localhost:2181(CONNECTED) 1] ls /consumers/kafkawarning01/offsets/avro-log-bj-jingnei1-dev-proxy-websense(对应主题)
    [3, 2, 1, 0, 4]
    [zk: localhost:2181(CONNECTED) 2] get /consumers/kafkawarning01/offsets/avro-log-bj-jingnei1-dev-proxy-websense/0(对应分区)
    80492922(偏移量)
    cZxid = 0x80037a3fa
    ctime = Tue Mar 29 10:51:35 CST 2016
    mZxid = 0x80037a3fa
    mtime = Tue Mar 29 10:51:35 CST 2016
    pZxid = 0x80037a3fa
    cversion = 0
    dataVersion = 0
    aclVersion = 0
    ephemeralOwner = 0x0
    dataLength = 8
    numChildren = 0

    针对这个我们可以通过修改zookeeper中的信息来重置一个topic的消费组的offset,以下是代码

     public static void main(String[] args) {
          String zkConnect = "11.11.184.162:2181,11.11.184.165:2181,11.11.184.178:2181";
              int sessionTimeout = 5000;  
              int connectionTimeout = 5000; 
              ZkClient zkClient = new ZkClient(zkConnect, sessionTimeout, connectionTimeout,new ZkSerializerSuper());
              //修改指定消费组在zookeeper中的偏移量
          map.put("0","12345");
          map.put(2,"23451");
          map.put(1,"23541"); setPersistentPathInZk(map,zkClient);//map存放的是每个分区对应的偏移量 }
    //调用ZKUtils中的updatePersistentPath方法进行修改
    private static void setPersistentPathInZk(Map map, ZkClient zkClient) {
        String group_id = "kafkawarning01";
        String topic = "topic1";
        Set set = map.keySet(); for (Object key : set) { ZkUtils.updatePersistentPath(zkClient, "/consumers/"+group_id+"/offsets/"+topic+"/"+key, map.get(key).toString()); } }
  • 相关阅读:
    多线程编程(2):线程的同步
    C#中listview实现排序
    [PLC]S7-300的数据类型
    C# 多线程编程(4):多线程与UI操作
    Thunderbird 80 column FIX 发出的邮件也需要在80列处line break
    vsftp 500 OOPS: vsftpd: refusing to run with writable anonymous root
    科普 What is YUV
    转载:网站真的可以无密码登录么?
    Ubuntu 12.04安装Microsoft lifecam studio摄像头
    Thunderbird on Ubuntu 12.04 调整邮件列表行间距
  • 原文地址:https://www.cnblogs.com/ygwx/p/5337730.html
Copyright © 2020-2023  润新知