kafka集群安装

0. Kafka源码包下载

http://mirror.bit.edu.cn/apache/kafka/

 

1. 集群环境

Master 172.16.11.97

Slave1 172.16.11.98

Slave2 172.16.11.99

 

2. 下载软件包

#Master

wget http://mirror.bit.edu.cn/apache/kafka/0.10.2.1/kafka_2.11-0.10.2.1.tgz

tar zxvf kafka_2.11-0.10.2.1.tgz

 

3. 修改Kafka配置文件

#Master

cd kafka_2.11-0.10.2.1/config

vim server.propertie

log.dirs=/tmp/kafka-logs

zookeeper.connect=master:2181,slave1:2181,slave2:2181

 

4. 增加环境变量

#Master、Slave1、Slave2

vim ~/.bashrc

export KAFKA_HOME=/usr/local/src/kafka_2.11-0.10.2.1

export PATH=$KAFKA_HOME/bin:$PATH

#刷新环境变量

source ~/.bashrc

 

5. 拷贝软件包

scp -r /usr/local/src/kafka_2.11-0.10.2.1 root@slave1:/usr/local/src/kafka_2.11-0.10.2.1

scp -r /usr/local/src/kafka_2.11-0.10.2.1 root@slave2:/usr/local/src/kafka_2.11-0.10.2.1

 

6. 修改Kafka配置文件

#Master

vim config/server.propertie

broker.id=0

#Slave1

vim config/server.propertie

broker.id=1

#Slave2

vim config/server.propertie

broker.id=2

 

7. 启动Kafka-Zookeeper集群

#如果启动了Zookeeper集群则跳过此步骤

vim /usr/local/src/kafka_2.11-0.10.2.1/bin/start-kafka-zookeeper.sh

/usr/local/src/kafka_2.11-0.10.2.1/bin/zookeeper-server-start.sh /usr/local/src/kafka_2.11-0.10.2.1/config/zookeeper.propeties

chmod +x /usr/local/src/kafka_2.11-0.10.2.1/bin/start-kafka-zookeeper.sh

start-kafka-zookeeper.sh

 

8.启动Kafka集群

#Master、Slave1、Slave2

vim /usr/local/src/kafka_2.11-0.10.2.1/bin/start-kafka.sh

/usr/local/src/kafka_2.11-0.10.2.1/bin/kafka-server-start.sh /usr/local/src/kafka_2.11-0.10.2.1/config/server.properties

chmod +x /usr/local/src/kafka_2.11-0.10.2.1/bin/start-kafka.sh

start-kafka-zookeeper.sh

 

9. 进程状态

jps

#Master

#Slave1

#Slave2

 

10. 关闭集群

kafka-server-stop.sh