• zookeeper删除kafka元数据


    问题:卸载kafka前未删除kafka topic,重新安装kafka后,生成跟之前topic名字相同的topic时报错,显示topic已存在

    [root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --create  --topic hive-test-table --partitions 5 --replication-factor 2
    WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
    Error while executing topic command : Topic "hive-test-table" already exists.
    [2017-01-16 10:48:30,766] ERROR kafka.common.TopicExistsException: Topic "hive-test-table" already exists.
    	at kafka.admin.AdminUtils$.createOrUpdateTopicPartitionAssignmentPathInZK(AdminUtils.scala:261)
    	at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:245)
    	at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:107)
    	at kafka.admin.TopicCommand$.main(TopicCommand.scala:60)
    	at kafka.admin.TopicCommand.main(TopicCommand.scala)
     (kafka.admin.TopicCommand$)
    
    

    原因:kafka topic的元数据存储在zookeeper里,卸载kafka前未删除topic,之前topic的元数据仍存在zk里,需要手动删除zk里的元数据

    解决:

    zookeeper shell命令

    [zk: d65:2181/kafka_hive(CONNECTED) 7] ll
    ZooKeeper -server host:port cmd args
    	connect host:port
    	get path [watch]
    	ls path [watch]
    	set path data [version]
    	rmr path
    	delquota [-n|-b] path
    	quit 
    	printwatches on|off
    	create [-s] [-e] path data acl
    	stat path [watch]
    	close 
    	ls2 path [watch]
    	history 
    	listquota path
    	setAcl path acl
    	getAcl path
    	sync path
    	redo cmdno
    	addauth scheme auth
    	delete path [version]
    	setquota -n|-b val path
    

    进入zookeeper目录,删除kafka topic 元数据

    [root@d100 bin]# sh zkCli.sh -server  d65:2181/kafka_hive
    
    [zk: d65:2181/kafka_hive(CONNECTED) 8] ls /
    [consumers, config, controller, isr_change_notification, brokers, admin, controller_epoch]
    
    [zk: d65:2181/kafka_hive(CONNECTED) 9] ls /brokers
    [seqid, topics, ids]
    
    [zk: d65:2181/kafka_hive(CONNECTED) 10] ls /brokers/topics
    [略...]
    
    [zk: d65:2181/kafka_hive(CONNECTED) 11] rmr /brokers/topics/hive-test-table
    
    [zk: d65:2181/kafka_hive(CONNECTED) 13] ls /brokers/topics
    [略...]
    

    重新创建topic成功

    [root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --create  --topic hive-test-table --partitions 5 --replication-factor 2
    Created topic "hive-test-table".
    
    [root@d96 ~]# kafka-topics --zookeeper d65:2181/kafka_hive --desc  --topic hive-test-table
    Topic:hive-test-table	PartitionCount:5	ReplicationFactor:2	Configs:
    	Topic: hive-test-table	Partition: 0	Leader: 1285	Replicas: 1285,1281	Isr: 1285,1281
    	Topic: hive-test-table	Partition: 1	Leader: 1281	Replicas: 1281,1282	Isr: 1281,1282
    	Topic: hive-test-table	Partition: 2	Leader: 1282	Replicas: 1282,1283	Isr: 1282,1283
    	Topic: hive-test-table	Partition: 3	Leader: 1283	Replicas: 1283,1284	Isr: 1283,1284
    	Topic: hive-test-table	Partition: 4	Leader: 1284	Replicas: 1284,1285	Isr: 1284,1285
    
  • 相关阅读:
    面试题25二叉树中和为某一值得路径+递归函数详细解析
    [Codeforces 1199D]Welfare State(线段树)
    [Codeforces 1199C]MP3(离散化+二分答案)
    [Codeforces 1191D] Tokitsukaze, CSL and Stone Game(博弈论)
    [Codeforces 639F] Bear and Chemistry (Tarjan+虚树)(有详细注释)
    [HDU5807] [BestCoder Round #86 1004] Keep In Touch (DP)
    [Codeforces712D] Memory and Scores(DP+前缀和优化)(不用单调队列)
    [Codeforces722E] Research Rover (dp+组合数学)
    [POJ3612] Telephone Wire(暴力dp+剪枝)
    [Codeforces600E] Lomsat gelral(树上启发式合并)
  • 原文地址:https://www.cnblogs.com/xiaodf/p/6289383.html
Copyright © 2020-2023  润新知