Kafka replication
kafka_replication_detailed_design_v2.pdf
kafka Detailed Replication Design V3
Apache Kafka中Follower如何从Leader fetch消息
KIP-1 - Remove support of request.required.acks
Hands-free Kafka Replication: A lesson in operational simplicity
Intra-cluster Replication in Apache Kafka
The current leader of a partition further maintains 3 sets: AR, ISR, CUR and RAR, which correspond to the set of replicas that are assigned to the partition, in-sync with the leader, catching up with the leader, and being reassigned to other brokers. Normally, ISR ⊆ AR and AR = ISR + CUR.
Replica
Replica { // a replica of a partition
broker_id : int
partition : Partition
isLocal : Boolean // is this replica local to this broker
log : Log // local log associated with this replica
hw : long // offset of the last committed message
leo : long // log end offset
}
Partition
Partition { //a partition in a topic
topic : string
partition_id : int
leader: Replica // the leader replica of this partition
commitQ: Queue // produce requests pending commit at the leader
AR: Set[Replica] // replicas assigned to this partition
ISR: Set[Replica] // In-sync replica set, maintained at the leader
CUR: Set[Replica] // Catch-up replica set, maintained at the leader
RAR: Set[Replica] // Reassigned replica set, maintained at the leader
}