解决kafka集群由于默认的__consumer_offsets这个topic的默认的副本数为1而存在的单点故障问题

抛出问题：

__consumer_offsets这个topic是由kafka自动创建的，默认50个，但是都存在一台kafka服务器上，这是不是就存在很明显的单点故障？
经测试，如果将存储consumer_offsets的这台机器kill掉，所有的消费者都停止消费了。请问这个问题是怎么解决的呢？

原因分析：

由于__consumer_offsets这个用于存储offset的分区是由kafka服务器默认自动创建的，那么它在创建该分区的时候，分区数和副本数的依据是什么？
分区数是固定的50，这个没什么可怀疑的，副本数呢？应该是一个默认值1，依据是，如果我们没有在server.properties文件中指定topic分区的副本数的话，它的默认值就是1。

__consumer_offsets是一个非常重要的topic，我们怎么能允许它只有一个副本呢？这样就存在单点故障，也就是如果该分区所在的集群宕机了的话，
我们的消费者就无法正常消费数据了。

我在笔记本上搭建了kafka集群，共3个Broker，来解决这个问题。下面是一些记录。

 说明：如果你的__consumer_offsets这个topic已经被创建了，而且只存在一台broker上，如果你直接使用命令删除这个topic是会报错了，提示这是kafka内置的topic，禁止删除。可以在zookeeper中删除（我是使用ZooInspector这个客户端连上zookeeper，删除和__consumer_offsets这个topic有关的目录或节点）。

然后就是修改kafka的核心配置文件server.properties，下面是第一台Broker的配置文件：

[root@hadoop01 kafka-logs]# cat /opt/kafka/config/server.properties
broker.id=0
listeners=PLAINTEXT://:9092
port=9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/opt/logs/kafka-logs
num.partitions=3
num.recovery.threads.per.data.dir=1
log.retention.hours=168
log.segment.bytes=536870912
log.retention.check.interval.ms=300000
zookeeper.connect=192.168.71.11:2181,192.168.71.12:2181,192.168.71.13:2181
izookeeper.connection.timeout.ms=6000
delete.topic.enable=true
host.name=192.168.71.11
advertised.host.name=192.168.71.11
auto.create.topics.enable=true
default.replication.factor=3

在上面的配置中，我开启了自动创建topic，并指定了topic的分区数为3，副本数为3(因为我有3台Broker，我们希望每台机器上都有一个副本，从而保证分区的高可用)。

下面是第二台Broker的server.properties文件的配置：

 1 [root@hadoop02 kafka-logs]# cat /opt/kafka/config/server.properties
 2 broker.id=1
 3 listeners=PLAINTEXT://:9092
 4 port=9092
 5 num.network.threads=3
 6 num.io.threads=8
 7 socket.send.buffer.bytes=102400
 8 socket.receive.buffer.bytes=102400
 9 socket.request.max.bytes=104857600
10 log.dirs=/opt/logs/kafka-logs
11 num.partitions=3
12 num.recovery.threads.per.data.dir=1
13 log.retention.hours=168
14 log.segment.bytes=536870912
15 log.retention.check.interval.ms=300000
16 zookeeper.connect=192.168.71.11:2181,192.168.71.12:2181,192.168.71.13:2181
17 izookeeper.connection.timeout.ms=6000
18 delete.topic.enable=true
19 host.name=192.168.71.12
20 advertised.host.name=192.168.71.12
21 auto.create.topics.enable=true
22 default.replication.factor=3
23 [root@hadoop02 kafka-logs]#

第3台Broker的server.properties文件的配置：

 1 [root@hadoop03 kafka-logs]# cat /opt/kafka/config/server.properties
 2 broker.id=2
 3 listeners=PLAINTEXT://:9092
 4 port=9092
 5 num.network.threads=3
 6 num.io.threads=8
 7 socket.send.buffer.bytes=102400
 8 socket.receive.buffer.bytes=102400
 9 socket.request.max.bytes=104857600
10 log.dirs=/opt/logs/kafka-logs
11 num.partitions=3
12 num.recovery.threads.per.data.dir=1
13 log.retention.hours=168
14 log.segment.bytes=536870912
15 log.retention.check.interval.ms=300000
16 zookeeper.connect=192.168.71.11:2181,192.168.71.12:2181,192.168.71.13:2181
17 izookeeper.connection.timeout.ms=6000
18 delete.topic.enable=true
19 host.name=192.168.71.13
20 advertised.host.name=192.168.71.13
21 auto.create.topics.enable=true
22 default.replication.factor=3
23 [root@hadoop03 kafka-logs]# 

配置好之后，启动3个Broker。此时__consumer_offsets分区不会被创建，它会在消费者开始消费数据的时候被创建。

首先启动一个消费者客户端来消费数据。然后启动一个生产者客户端来发送数据到kafka集群。

下面查看第一个Broker的topic，发现有50个__consumer_offsets分区：

[root@hadoop01 kafka-logs]# ll
total 224
-rw-r--r--. 1 root root    0 Mar 24 13:19 cleaner-offset-checkpoint
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-0
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-1
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-10
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-11
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-12
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-13
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-14
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-15
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-16
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-17
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-18
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-19
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-2
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-20
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-21
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-22
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-23
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-24
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-25
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-26
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-27
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-28
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-29
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-3
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-30
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-31
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-32
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-33
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-34
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-35
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-36
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-37
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-38
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-39
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-4
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-40
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-41
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-42
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-43
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-44
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-45
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-46
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-47
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-48
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-49
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-5
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-6
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-7
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-8
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-9
drwxr-xr-x. 2 root root 4096 Mar 24 13:54 friend-0
drwxr-xr-x. 2 root root 4096 Mar 24 13:54 friend-1
drwxr-xr-x. 2 root root 4096 Mar 24 13:54 friend-2
-rw-r--r--. 1 root root   54 Mar 24 13:19 meta.properties
-rw-r--r--. 1 root root 1228 Mar 24 13:54 recovery-point-offset-checkpoint
-rw-r--r--. 1 root root 1228 Mar 24 13:54 replication-offset-checkpoint
[root@hadoop01 kafka-logs]# 

启动第二台Broker服务器，它也有50个"__consumer_offsets"分区：

[root@hadoop02 kafka-logs]# ll
total 224
-rw-r--r--. 1 root root    0 Mar 24 13:19 cleaner-offset-checkpoint
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-0
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-1
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-10
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-11
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-12
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-13
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-14
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-15
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-16
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-17
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-18
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-19
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-2
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-20
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-21
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-22
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-23
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-24
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-25
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-26
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-27
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-28
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-29
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-3
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-30
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-31
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-32
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-33
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-34
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-35
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-36
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-37
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-38
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-39
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-4
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-40
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-41
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-42
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-43
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-44
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-45
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-46
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-47
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-48
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-49
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-5
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-6
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-7
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-8
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-9
drwxr-xr-x. 2 root root 4096 Mar 24 13:54 friend-0
drwxr-xr-x. 2 root root 4096 Mar 24 13:54 friend-1
drwxr-xr-x. 2 root root 4096 Mar 24 13:54 friend-2
-rw-r--r--. 1 root root   54 Mar 24 13:19 meta.properties
-rw-r--r--. 1 root root 1228 Mar 24 13:54 recovery-point-offset-checkpoint
-rw-r--r--. 1 root root 1228 Mar 24 13:54 replication-offset-checkpoint
[root@hadoop02 kafka-logs]# 

第3台Broker上的"__consumer_offsets"分区情况：

[root@hadoop03 kafka-logs]# ll
total 224
-rw-r--r--. 1 root root    0 Mar 24 13:19 cleaner-offset-checkpoint
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-0
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-1
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-10
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-11
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-12
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-13
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-14
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-15
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-16
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-17
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-18
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-19
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-2
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-20
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-21
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-22
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-23
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-24
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-25
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-26
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-27
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-28
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-29
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-3
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-30
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-31
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-32
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-33
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-34
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-35
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-36
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-37
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-38
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-39
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-4
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-40
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-41
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-42
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-43
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-44
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-45
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-46
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-47
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-48
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-49
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-5
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-6
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-7
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-8
drwxr-xr-x. 2 root root 4096 Mar 24 13:46 __consumer_offsets-9
drwxr-xr-x. 2 root root 4096 Mar 24 13:54 friend-0
drwxr-xr-x. 2 root root 4096 Mar 24 13:54 friend-1
drwxr-xr-x. 2 root root 4096 Mar 24 13:54 friend-2
-rw-r--r--. 1 root root   54 Mar 24 13:19 meta.properties
-rw-r--r--. 1 root root 1228 Mar 24 13:54 recovery-point-offset-checkpoint
-rw-r--r--. 1 root root 1228 Mar 24 13:54 replication-offset-checkpoint
[root@hadoop03 kafka-logs]# /opt/kafka/bin/kafka-topics.sh --describe --topic __consumer_offsets --zookeeper localhost:2181
Topic:__consumer_offsets        PartitionCount:50       ReplicationFactor:3     Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=producer
        Topic: __consumer_offsets       Partition: 0    Leader: 1       Replicas: 1,2,0 Isr: 1,2,0
        Topic: __consumer_offsets       Partition: 1    Leader: 2       Replicas: 2,0,1 Isr: 2,0,1
        Topic: __consumer_offsets       Partition: 2    Leader: 0       Replicas: 0,1,2 Isr: 1,2,0
        Topic: __consumer_offsets       Partition: 3    Leader: 1       Replicas: 1,0,2 Isr: 1,0,2
        Topic: __consumer_offsets       Partition: 4    Leader: 2       Replicas: 2,1,0 Isr: 2,1,0
        Topic: __consumer_offsets       Partition: 5    Leader: 0       Replicas: 0,2,1 Isr: 2,1,0
        Topic: __consumer_offsets       Partition: 6    Leader: 1       Replicas: 1,2,0 Isr: 1,2,0
        Topic: __consumer_offsets       Partition: 7    Leader: 2       Replicas: 2,0,1 Isr: 2,0,1
        Topic: __consumer_offsets       Partition: 8    Leader: 0       Replicas: 0,1,2 Isr: 1,2,0
        Topic: __consumer_offsets       Partition: 9    Leader: 1       Replicas: 1,0,2 Isr: 1,0,2
        Topic: __consumer_offsets       Partition: 10   Leader: 2       Replicas: 2,1,0 Isr: 2,1,0
        Topic: __consumer_offsets       Partition: 11   Leader: 0       Replicas: 0,2,1 Isr: 2,1,0
        Topic: __consumer_offsets       Partition: 12   Leader: 1       Replicas: 1,2,0 Isr: 1,0,2
        Topic: __consumer_offsets       Partition: 13   Leader: 2       Replicas: 2,0,1 Isr: 2,0,1
        Topic: __consumer_offsets       Partition: 14   Leader: 0       Replicas: 0,1,2 Isr: 1,2,0
        Topic: __consumer_offsets       Partition: 15   Leader: 1       Replicas: 1,0,2 Isr: 1,0,2
        Topic: __consumer_offsets       Partition: 16   Leader: 2       Replicas: 2,1,0 Isr: 2,1,0
        Topic: __consumer_offsets       Partition: 17   Leader: 0       Replicas: 0,2,1 Isr: 2,1,0
        Topic: __consumer_offsets       Partition: 18   Leader: 1       Replicas: 1,2,0 Isr: 1,2,0
        Topic: __consumer_offsets       Partition: 19   Leader: 2       Replicas: 2,0,1 Isr: 2,0,1
        Topic: __consumer_offsets       Partition: 20   Leader: 0       Replicas: 0,1,2 Isr: 1,2,0
        Topic: __consumer_offsets       Partition: 21   Leader: 1       Replicas: 1,0,2 Isr: 1,0,2
        Topic: __consumer_offsets       Partition: 22   Leader: 2       Replicas: 2,1,0 Isr: 2,1,0
        Topic: __consumer_offsets       Partition: 23   Leader: 0       Replicas: 0,2,1 Isr: 2,1,0
        Topic: __consumer_offsets       Partition: 24   Leader: 1       Replicas: 1,2,0 Isr: 1,0,2
        Topic: __consumer_offsets       Partition: 25   Leader: 2       Replicas: 2,0,1 Isr: 2,0,1
        Topic: __consumer_offsets       Partition: 26   Leader: 0       Replicas: 0,1,2 Isr: 1,2,0
        Topic: __consumer_offsets       Partition: 27   Leader: 1       Replicas: 1,0,2 Isr: 1,0,2
        Topic: __consumer_offsets       Partition: 28   Leader: 2       Replicas: 2,1,0 Isr: 2,1,0
        Topic: __consumer_offsets       Partition: 29   Leader: 0       Replicas: 0,2,1 Isr: 2,1,0
        Topic: __consumer_offsets       Partition: 30   Leader: 1       Replicas: 1,2,0 Isr: 1,2,0
        Topic: __consumer_offsets       Partition: 31   Leader: 2       Replicas: 2,0,1 Isr: 2,0,1
        Topic: __consumer_offsets       Partition: 32   Leader: 0       Replicas: 0,1,2 Isr: 1,2,0
        Topic: __consumer_offsets       Partition: 33   Leader: 1       Replicas: 1,0,2 Isr: 1,0,2
        Topic: __consumer_offsets       Partition: 34   Leader: 2       Replicas: 2,1,0 Isr: 2,1,0
        Topic: __consumer_offsets       Partition: 35   Leader: 0       Replicas: 0,2,1 Isr: 2,1,0
        Topic: __consumer_offsets       Partition: 36   Leader: 1       Replicas: 1,2,0 Isr: 1,2,0
        Topic: __consumer_offsets       Partition: 37   Leader: 2       Replicas: 2,0,1 Isr: 2,0,1
        Topic: __consumer_offsets       Partition: 38   Leader: 0       Replicas: 0,1,2 Isr: 1,2,0
        Topic: __consumer_offsets       Partition: 39   Leader: 1       Replicas: 1,0,2 Isr: 1,0,2
        Topic: __consumer_offsets       Partition: 40   Leader: 2       Replicas: 2,1,0 Isr: 2,1,0
        Topic: __consumer_offsets       Partition: 41   Leader: 0       Replicas: 0,2,1 Isr: 2,1,0
        Topic: __consumer_offsets       Partition: 42   Leader: 1       Replicas: 1,2,0 Isr: 1,0,2
        Topic: __consumer_offsets       Partition: 43   Leader: 2       Replicas: 2,0,1 Isr: 2,0,1
        Topic: __consumer_offsets       Partition: 44   Leader: 0       Replicas: 0,1,2 Isr: 1,2,0
        Topic: __consumer_offsets       Partition: 45   Leader: 1       Replicas: 1,0,2 Isr: 1,0,2
        Topic: __consumer_offsets       Partition: 46   Leader: 2       Replicas: 2,1,0 Isr: 2,1,0
        Topic: __consumer_offsets       Partition: 47   Leader: 0       Replicas: 0,2,1 Isr: 2,1,0
        Topic: __consumer_offsets       Partition: 48   Leader: 1       Replicas: 1,2,0 Isr: 1,2,0
        Topic: __consumer_offsets       Partition: 49   Leader: 2       Replicas: 2,0,1 Isr: 2,0,1
[root@hadoop03 kafka-logs]# 
[root@hadoop03 kafka-logs]# /opt/kafka/bin/kafka-topics.sh --describe --topic friend --zookeeper localhost:2181                  
Topic:friend    PartitionCount:3        ReplicationFactor:3     Configs:
        Topic: friend   Partition: 0    Leader: 2       Replicas: 2,0,1 Isr: 2,0,1
        Topic: friend   Partition: 1    Leader: 0       Replicas: 0,1,2 Isr: 0,1,2
        Topic: friend   Partition: 2    Leader: 1       Replicas: 1,2,0 Isr: 1,2,0
[root@hadoop03 kafka-logs]# 

下面是启动消费者消费数据时的日志：

======启动一个消费者后的日志情况======================================================================
D:javajdk1.8.0_121injava -Didea.launcher.port=7532 -Didea.launcher.bin.path=D:javaIDEA-14.1.4in -Dfile.encoding=UTF-8 -classpath D:javajdk1.8.0_121jrelibcharsets.jar;D:javajdk1.8.0_121jrelibdeploy.jar;D:javajdk1.8.0_121jrelibjavaws.jar;D:javajdk1.8.0_121jrelibjce.jar;D:javajdk1.8.0_121jrelibjfr.jar;D:javajdk1.8.0_121jrelibjfxswt.jar;D:javajdk1.8.0_121jrelibjsse.jar;D:javajdk1.8.0_121jrelibmanagement-agent.jar;D:javajdk1.8.0_121jrelibplugin.jar;D:javajdk1.8.0_121jrelib
esources.jar;D:javajdk1.8.0_121jrelib
t.jar;D:javajdk1.8.0_121jrelibextaccess-bridge-64.jar;D:javajdk1.8.0_121jrelibextcldrdata.jar;D:javajdk1.8.0_121jrelibextdnsns.jar;D:javajdk1.8.0_121jrelibextjaccess.jar;D:javajdk1.8.0_121jrelibextjfxrt.jar;D:javajdk1.8.0_121jrelibextlocaledata.jar;D:javajdk1.8.0_121jrelibext
ashorn.jar;D:javajdk1.8.0_121jrelibextsunec.jar;D:javajdk1.8.0_121jrelibextsunjce_provider.jar;D:javajdk1.8.0_121jrelibextsunmscapi.jar;D:javajdk1.8.0_121jrelibextsunpkcs11.jar;D:javajdk1.8.0_121jrelibextzipfs.jar;D:zpgitzp-kafkazp-consumer-frienduildclassesmain;D:zpgitzp-kafkazp-consumer-frienduild
esourcesmain;C:UsersSYJ.gradlecachesmodules-2files-2.1org.postgresqlpostgresql9.4-1201-jdbc41870b0e689b514304461a9c1aba11920dc5de4321postgresql-9.4-1201-jdbc41.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1joda-timejoda-time2.5c73038a3688525aad5cf33409df483178290cd64joda-time-2.5.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.apache.commonscommons-lang33.3.290a3822c38ec8c996e84c16a3477ef632cbc87a3commons-lang3-3.3.2.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.apache.commonscommons-io1.3.26dde38349ba9bb5e6ea6320531eae969985dae5commons-io-1.3.2.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1commons-netcommons-net3.3cd0d5510908225f76c5fe5a3f1df4fa44866f81ecommons-net-3.3.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1com.fasterxml.jackson.corejackson-databind2.4.28e31266a272ad25ac4c089734d93e8d811652c1fjackson-databind-2.4.2.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.apache.httpcomponentshttpclient4.3.59783d89b8eea20a517a4afc5f979bd2882b54c44httpclient-4.3.5.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.mybatismybatis3.2.87b6bf82cea13570b5290d6ed841283a1fcce170mybatis-3.2.8.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.mybatismybatis-spring1.2.21e40a7f5373e4242075a1d386817e7dd49b1697dmybatis-spring-1.2.2.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1com.github.miemiedevmybatis-paginator1.2.15d5d9891d2d89b13b0856a00f04ff60dd1f95ffdbmybatis-paginator-1.2.15.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1com.github.pagehelperpagehelper3.4.2173072cf0dab08b102d7420932f678d0c60138b7pagehelper-3.4.2.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1mysqlmysql-connector-java5.1.32d28c9a6cf0810fd0e2180e44029c10a54ca26de8mysql-connector-java-5.1.32.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1com.alibabadruid1.0.9f91e47c9018578e5ca4d2e808cc3351505ae3ebbdruid-1.0.9.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1jstljstl1.274aca283cd4f4b4f3e425f5820cda58f44409547jstl-1.2.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1commons-fileuploadcommons-fileupload1.3.1c621b54583719ac0310404463d6d99db27e1052ccommons-fileupload-1.3.1.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1
edis.clientsjedis2.7.2f2f47f1025ea5090263820e8598e56eb47f5c88ajedis-2.7.2.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.apache.solrsolr-solrj4.10.36e5ee2f18a6615d5419b61e60e3cae274ac66085solr-solrj-4.10.3.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1com.alibabafastjson1.2.4fbcf8415e32859b473b336e5ac6422ee69b1185bfastjson-1.2.4.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.springframework.integrationspring-integration-kafka1.3.0.RELEASE6d46351ea70084d51cd43f62cd80bb20af1c9d96spring-integration-kafka-1.3.0.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.springframework.kafkaspring-kafka1.1.3.RELEASEa45832c6a155383b2a54659926c1dc325052d883spring-kafka-1.1.3.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.apache.kafkakafka_2.10.10.0.037899467b805929a2ae898d312cd789dbd902c3dkafka_2.10-0.10.0.0.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1com.fasterxml.jackson.corejackson-annotations2.4.0d6a66c7a5f01cf500377bd669507a08cfeba882ajackson-annotations-2.4.0.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1com.fasterxml.jackson.corejackson-core2.4.2ceb72830d95c512b4b300a38f29febc85bdf6e4bjackson-core-2.4.2.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.apache.httpcomponentshttpcore4.3.231fbbff1ddbf98f3aa7377c94d33b0447c646b6ehttpcore-4.3.2.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1commons-codeccommons-codec1.67f0fc8f61ecadeb3695f0b9464755eee44374d4commons-codec-1.6.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.apache.commonscommons-pool22.362a559a025fd890c30364296ece14643ba9c8c5bcommons-pool2-2.3.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.apache.httpcomponentshttpmime4.3.1f7899276dddd01d8a42ecfe27e7031fcf9824422httpmime-4.3.1.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.apache.zookeeperzookeeper3.4.61b2502e29da1ebaade2357cd1de35a855fa3755zookeeper-3.4.6.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.codehaus.woodstoxwstx-asl3.2.7252c7faae9ce98cb9c9d29f02db88f7373e7f407wstx-asl-3.2.7.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.noggit
oggit.58e6e65624d2e09a30190c6434abe23b7d4e5413c
oggit-0.5.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.springframework.integrationspring-integration-core4.1.6.RELEASE7b95eb7f4c08070c345a723720fd5dbccf71b140spring-integration-core-4.1.6.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1com.yammer.metricsmetrics-annotation2.2.062962b54c490a95c0bb255fa93b0ddd6cc36dd4bmetrics-annotation-2.2.0.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1com.yammer.metricsmetrics-core2.2.0f82c035cfa786d3cbec362c38c22a5f5b1bc8724metrics-core-2.2.0.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1io.projectreactor
eactor-core2.0.6.RELEASEa36e58c9d2d0ac1f47e17e8db60a7ebb9d09789c
eactor-core-2.0.6.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.springframeworkspring-messaging4.3.5.RELEASE480f1116f2060107493b91e72e21359b02aca776spring-messaging-4.3.5.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.springframework.retryspring-retry1.1.3.RELEASEf9517754a9990194ed0daecb5653e48564d557eespring-retry-1.1.3.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.projectreactor
eactor-core1.1.4.RELEASEda621f1aef5f8cd5c22ae78afec53f1e8659caed
eactor-core-1.1.4.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.reactivestreams
eactive-streams1.0.014b8c877d98005ba3941c9257cfe09f6ed0e0d74
eactive-streams-1.0.0.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1com.lmaxdisruptor3.2.1db375f499e32c3f06549e9addf8b1647123d6426disruptor-3.2.1.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1io.gatlingjsr166e1.0d1bf191a18dfe6e3157a4fbf6b527390d906ace6jsr166e-1.0.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1commons-iocommons-io2.3cd8d6ffc833cc63c30d712a180f4663d8f55799bcommons-io-2.3.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1com.101teczkclient.8c0f700a4a3b386279d7d8dd164edecbe836cbfdbzkclient-0.8.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.scala-langscala-library2.10.6421989aa8f95a05a4f894630aad96b8c7b828732scala-library-2.10.6.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1
et.sf.jopt-simplejopt-simple4.9ee9e9eaa0a35360dcfeac129ff4923215fd65904jopt-simple-4.9.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1jlinejline.9.9499a18e9a44834afdebc467294e1138364c207402jline-0.9.94.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1io.netty
etty3.7.0.Final7a8c35599c68c0bf383df74469aa3e03d9aca87
etty-3.7.0.Final.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1junitjunit3.8.199129f16442844f6a4a11ae22fbbee40b14d774fjunit-3.8.1.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.apache.kafkakafka-clients.10.1.152f03b809c26f9676ddfcf130f13c80dfc929b98kafka-clients-0.10.1.1.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1
et.jpountz.lz4lz41.3.0c708bb2590c0652a642236ef45d9f99ff842a2celz4-1.3.0.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.xerial.snappysnappy-java1.1.2.648d92871ca286a47f230feb375f0bbffa83b85f6snappy-java-1.1.2.6.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1commons-loggingcommons-logging1.24bfc12adfe4842bf07b657f0369c4cb522955686commons-logging-1.2.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1com.goldmansachsgs-collections5.1.07114c5349c816ea645b1ea3ffcc21fa073cbabcgs-collections-5.1.0.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.slf4jslf4j-log4j121.7.217238b064d1aba20da2ac03217d700d91e02460faslf4j-log4j12-1.7.21.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.slf4jslf4j-api1.7.21139535a69a4239db087de9bab0bee568bf8e0b70slf4j-api-1.7.21.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1log4jlog4j1.2.175af35056b4d257e4b64b9e8069c0746e8b08629flog4j-1.2.17.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.springframeworkspring-beans4.3.5.RELEASEe12bbc3277da28e2e2608a187f83091dc6c300bfspring-beans-4.3.5.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.springframeworkspring-core4.3.5.RELEASE80299e3f80e8c6d5c076db2ba6adf44a4b52f578spring-core-4.3.5.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.springframeworkspring-context4.3.5.RELEASEca3391c0e17d0138335ba51b51371661d20d56a8spring-context-4.3.5.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1com.goldmansachsgs-collections-api5.1.0ea605cdf64cab5fc7b48c99f061d4c8db05b6ff1gs-collections-api-5.1.0.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.springframeworkspring-aop4.3.5.RELEASE4f113218af716bd8d174c411f19b26418b5a70f6spring-aop-4.3.5.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.springframeworkspring-expression4.3.5.RELEASE3689dc6c5b942ecde4122eac889ed87977d6f287spring-expression-4.3.5.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.springframeworkspring-webmvc4.3.5.RELEASEc624659217edab07d8279fb0f90462136f089220spring-webmvc-4.3.5.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.springframeworkspring-jdbc4.3.5.RELEASE7c09e38b6f6e0b178973dea06bb8fdc6d19aa596spring-jdbc-4.3.5.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.springframeworkspring-aspects4.3.5.RELEASE8c00c15865a1da0a1ba143a50622a8436a56e097spring-aspects-4.3.5.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.springframeworkspring-web4.3.5.RELEASE6641daccf2fddafc8358144f3a4f999130fdf144spring-web-4.3.5.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.springframeworkspring-tx4.3.5.RELEASE296d8ae21fadc0f115dae7d1e8d5c4f8c7de2c3espring-tx-4.3.5.RELEASE.jar;C:UsersSYJ.gradlecachesmodules-2files-2.1org.aspectjaspectjweaver1.8.9db28774f477f07220eac18d5ec9c4e01f48589d7aspectjweaver-1.8.9.jar;D:javaIDEA-14.1.4libidea_rt.jar com.intellij.rt.execution.application.AppMain com.zhaopin.AppMain
[17/03/24 20:46:36:702][org.springframework.context.support.AbstractApplicationContext-prepareRefresh] Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@6bc168e5: startup date [Fri Mar 24 20:46:36 CST 2017]; root of context hierarchy
[17/03/24 20:46:36:971][org.springframework.beans.factory.xml.XmlBeanDefinitionReader-loadBeanDefinitions] Loading XML bean definitions from file [D:zpgitzp-kafkazp-consumer-frienduild
esourcesmainspringapplicationContext-consumer.xml]
[17/03/24 20:46:37:472][org.springframework.beans.factory.xml.XmlBeanDefinitionReader-loadBeanDefinitions] Loading XML bean definitions from file [D:zpgitzp-kafkazp-consumer-frienduild
esourcesmainspringapplicationContext-dao.xml]
[17/03/24 20:46:37:871][org.springframework.beans.factory.xml.XmlBeanDefinitionReader-loadBeanDefinitions] Loading XML bean definitions from file [D:zpgitzp-kafkazp-consumer-frienduild
esourcesmainspringapplicationContext-service.xml]
[17/03/24 20:46:39:580][org.springframework.core.io.support.PropertiesLoaderSupport-loadProperties] Loading properties file from file [D:zpgitzp-kafkazp-consumer-frienduild
esourcesmain
esourcedb.properties]
[17/03/24 20:46:39:580][org.springframework.core.io.support.PropertiesLoaderSupport-loadProperties] Loading properties file from file [D:zpgitzp-kafkazp-consumer-frienduild
esourcesmain
esource
esource.properties]
[17/03/24 20:46:41:058][org.apache.kafka.common.config.AbstractConfig-logAll] ConsumerConfig values: 
    auto.commit.interval.ms = 1000
    auto.offset.reset = latest
    bootstrap.servers = [192.168.71.11:9092, 192.168.71.12:9092, 192.168.71.13:9092]
    check.crcs = true
    client.id = 
    connections.max.idle.ms = 540000
    enable.auto.commit = true
    exclude.internal.topics = true
    fetch.max.bytes = 52428800
    fetch.max.wait.ms = 500
    fetch.min.bytes = 1
    group.id = friend-group
    heartbeat.interval.ms = 3000
    interceptor.classes = null
    key.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
    max.partition.fetch.bytes = 1048576
    max.poll.interval.ms = 300000
    max.poll.records = 1
    metadata.max.age.ms = 300000
    metric.reporters = []
    metrics.num.samples = 2
    metrics.sample.window.ms = 30000
    partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
    receive.buffer.bytes = 65536
    reconnect.backoff.ms = 50
    request.timeout.ms = 305000
    retry.backoff.ms = 100
    sasl.kerberos.kinit.cmd = /usr/bin/kinit
    sasl.kerberos.min.time.before.relogin = 60000
    sasl.kerberos.service.name = null
    sasl.kerberos.ticket.renew.jitter = 0.05
    sasl.kerberos.ticket.renew.window.factor = 0.8
    sasl.mechanism = GSSAPI
    security.protocol = PLAINTEXT
    send.buffer.bytes = 131072
    session.timeout.ms = 15000
    ssl.cipher.suites = null
    ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
    ssl.endpoint.identification.algorithm = null
    ssl.key.password = null
    ssl.keymanager.algorithm = SunX509
    ssl.keystore.location = null
    ssl.keystore.password = null
    ssl.keystore.type = JKS
    ssl.protocol = TLS
    ssl.provider = null
    ssl.secure.random.implementation = null
    ssl.trustmanager.algorithm = PKIX
    ssl.truststore.location = null
    ssl.truststore.password = null
    ssl.truststore.type = JKS
    value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer

[17/03/24 20:46:41:066][org.apache.kafka.common.config.AbstractConfig-logAll] ConsumerConfig values: 
    auto.commit.interval.ms = 1000
    auto.offset.reset = latest
    bootstrap.servers = [192.168.71.11:9092, 192.168.71.12:9092, 192.168.71.13:9092]
    check.crcs = true
    client.id = consumer-1
    connections.max.idle.ms = 540000
    enable.auto.commit = true
    exclude.internal.topics = true
    fetch.max.bytes = 52428800
    fetch.max.wait.ms = 500
    fetch.min.bytes = 1
    group.id = friend-group
    heartbeat.interval.ms = 3000
    interceptor.classes = null
    key.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
    max.partition.fetch.bytes = 1048576
    max.poll.interval.ms = 300000
    max.poll.records = 1
    metadata.max.age.ms = 300000
    metric.reporters = []
    metrics.num.samples = 2
    metrics.sample.window.ms = 30000
    partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
    receive.buffer.bytes = 65536
    reconnect.backoff.ms = 50
    request.timeout.ms = 305000
    retry.backoff.ms = 100
    sasl.kerberos.kinit.cmd = /usr/bin/kinit
    sasl.kerberos.min.time.before.relogin = 60000
    sasl.kerberos.service.name = null
    sasl.kerberos.ticket.renew.jitter = 0.05
    sasl.kerberos.ticket.renew.window.factor = 0.8
    sasl.mechanism = GSSAPI
    security.protocol = PLAINTEXT
    send.buffer.bytes = 131072
    session.timeout.ms = 15000
    ssl.cipher.suites = null
    ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
    ssl.endpoint.identification.algorithm = null
    ssl.key.password = null
    ssl.keymanager.algorithm = SunX509
    ssl.keystore.location = null
    ssl.keystore.password = null
    ssl.keystore.type = JKS
    ssl.protocol = TLS
    ssl.provider = null
    ssl.secure.random.implementation = null
    ssl.trustmanager.algorithm = PKIX
    ssl.truststore.location = null
    ssl.truststore.password = null
    ssl.truststore.type = JKS
    value.deserializer = class org.apache.kafka.common.serialization.StringDeserializer

[17/03/24 20:46:41:480][org.apache.kafka.common.utils.AppInfoParser$AppInfo-<init>] Kafka version : 0.10.1.1
[17/03/24 20:46:41:480][org.apache.kafka.common.utils.AppInfoParser$AppInfo-<init>] Kafka commitId : f10ef2720b03b247
[17/03/24 20:46:41:625][org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup-start] Starting beans in phase 0
[17/03/24 20:46:41:630][org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup-start] Starting beans in phase 0
[17/03/24 20:46:58:908][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:46:58:911][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator-onJoinPrepare] Revoking previously assigned partitions [] for group friend-group
[17/03/24 20:46:58:912][org.springframework.kafka.listener.AbstractMessageListenerContainer$2-onPartitionsRevoked] partitions revoked:[]
[17/03/24 20:46:58:912][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:46:59:240][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:46:59:558][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:46:59:559][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:46:59:566][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:46:59:688][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:46:59:689][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:46:59:692][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:46:59:807][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:46:59:808][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:46:59:814][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:46:59:941][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:46:59:942][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:47:00:520][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:47:00:828][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:47:00:829][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:47:00:841][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:47:01:182][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:47:01:183][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:47:02:911][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:47:03:352][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:47:03:354][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:47:03:946][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:47:04:394][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:47:04:398][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:47:04:450][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:47:04:676][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:47:04:678][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:47:04:687][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:47:05:307][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:47:05:308][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:47:05:655][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:47:06:182][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:47:06:183][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:47:06:190][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:47:08:076][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:47:08:077][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:47:11:343][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:47:11:514][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:47:11:516][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:47:11:548][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:47:11:698][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:47:11:700][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:47:11:714][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:47:13:814][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:47:13:816][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:47:17:025][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$1-onSuccess] Successfully joined group friend-group with generation 1
[17/03/24 20:47:17:026][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator-onJoinComplete] Setting newly assigned partitions [] for group friend-group
[17/03/24 20:47:17:027][org.springframework.kafka.listener.AbstractMessageListenerContainer$2-onPartitionsAssigned] partitions assigned:[]
[17/03/24 20:47:23:055][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:47:23:237][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.12:9092 (id: 2147483646 rack: null) for group friend-group.
[17/03/24 20:47:35:384][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.12:9092 (id: 2147483646 rack: null) dead for group friend-group
[17/03/24 20:47:35:407][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:47:39:366][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-coordinatorDead] Marking the coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) dead for group friend-group
[17/03/24 20:47:39:408][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$GroupCoordinatorResponseHandler-onSuccess] Discovered coordinator 192.168.71.11:9092 (id: 2147483647 rack: null) for group friend-group.
[17/03/24 20:55:43:272][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator-onJoinPrepare] Revoking previously assigned partitions [] for group friend-group

====发送一条消息到生产者之后，消费者输出日志如下=========================================
[17/03/24 20:55:43:273][org.springframework.kafka.listener.AbstractMessageListenerContainer$2-onPartitionsRevoked] partitions revoked:[]
[17/03/24 20:55:43:273][org.apache.kafka.clients.consumer.internals.AbstractCoordinator-sendJoinGroupRequest] (Re-)joining group friend-group
[17/03/24 20:55:43:420][org.apache.kafka.clients.consumer.internals.AbstractCoordinator$1-onSuccess] Successfully joined group friend-group with generation 2
[17/03/24 20:55:43:424][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator-onJoinComplete] Setting newly assigned partitions [friend-1, friend-0, friend-2] for group friend-group
[17/03/24 20:55:43:425][org.springframework.kafka.listener.AbstractMessageListenerContainer$2-onPartitionsAssigned] partitions assigned:[friend-1, friend-0, friend-2]
====发送第二条消息到生产者后，消费者输出日志如下======================================================
[17/03/24 21:05:05:408][com.zhaopin.consumer.ConsumerService-onMessage] ====onMessage====ConsumerRecord(topic = friend, partition = 0, offset = 0, CreateTime = 1490360705152, checksum = 2832597311, serialized key size = 48, serialized value size = 172, key = 5f974fa3-676c-4474-a260-b4417c36bd34||zhaopin123, value = {"data":{"friends":[{"friendName":"55445","friendPhone":"55445"},{"friendName":"55445","friendPhone":"55445"}],"userId":55555,"userPhone":"55445"},"requestId":"zhaopin123"})
[17/03/24 21:05:06:137][com.alibaba.druid.pool.DruidDataSource-validationQueryCheck] testWhileIdle is true, validationQuery not set
[17/03/24 21:05:06:181][com.alibaba.druid.pool.DruidDataSource-init] {dataSource-1} inited


=====数据库friend表中增加了2条数据===============================================
287835    55555    55445    1    0    55445    0    55445
287836    55555    55445    1    0    55445    0    55445