Zookeeper 分布式服务框架是 Apache Hadoop 的一个子项目，它主要是用来解决分布式应用中经常遇到的一些数据管理问题，如：统一命名服务、状态同步服务、集群管理、分布式应用配置项的管理等。现在openstack的项目已经有开始使用zookeeper来实现分布式开发，例如Ceilometer的Central Agent。在SDN控制器开发领域，对于控制器的高可用和负载均衡的需求也需要引用zookeeper工具。因此作为openstack架构师和开发人员，有必要深入研究zookeeper的原理和开发技巧。由于zookeeper是基于java开发项目，作为openstack的python开发语言，现在已经有一些成熟的接口api和项目，例如kazoo、tooz等。
首先我们要搭建一套zookeeper的学习研究环境
1.获取zookeeper包
wget http://mirror.nus.edu.sg/apache/zookeeper/zookeeper-3.4.6/zookeeper-3.4.6.tar.gz
2.创建三节点环境
server1
├── data
├── dataLog
├── logs
├── zookeeper-3.4.6
└── zookeeper-3.4.6.tar.gz
server2
├── data
├── dataLog
├── logs
├── zookeeper-3.4.6
└── zookeeper-3.4.6.tar.gz
server3
├── data
├── dataLog
├── logs
├── zookeeper-3.4.6
└── zookeeper-3.4.6.tar.gz
Server主配置zoo.cfg
# 心跳间隔时间(ms)
tickTime=2000
# Follower与Leader初始化时能容忍的时间(initLimit * tickTime)
initLimit=10
# Follower与Leader请求响应时能容忍的时间(syncLimit * tickTime)
syncLimit=5
# server1数据目录，每个server各自配置
dataDir=/path/to/server1/data
# 客户端连接端口
clientPort=2182
# server列表:
#
# .n(标记server)=(server地址):(投票时所用端口):(选举Leader时所用端口)
server.1=127.0.0.1:2889:3889
server.2=127.0.0.1:2990:3990
server.3=127.0.0.1:2991:3991
在每个Server/data目录下建立一个myid文件，标识自己是哪个Server(对应zoo.cfg的server列表)
现在启动server1
server1/zookeeper-3.4.6/bin/zkServer.sh start-foreground
#
# 输出
2015-03-02 19:56:47,485 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2182:QuorumPeer@714] - LOOKING
2015-03-02 19:56:47,486 [myid:1] - INFO [QuorumPeer[myid=1]/0:0:0:0:0:0:0:0:2182:FastLeaderElection@815] - New election. My id = 1, proposed zxid=0x1a00000006
2015-03-02 19:56:47,488 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 1 (n.leader), 0x1a00000006 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x1a (n.peerEpoch) LOOKING (my state)
2015-03-02 19:56:47,510 [myid:1] - WARN [WorkerSender[myid=1]:QuorumCnxManager@382] - Cannot open channel to 2 at election address /127.0.0.1:3990
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:368)
at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:341)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:449)
at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:430)
at java.lang.Thread.run(Thread.java:744)
2015-03-02 19:56:47,560 [myid:1] - WARN [WorkerSender[myid=1]:QuorumCnxManager@382] - Cannot open channel to 3 at election address /127.0.0.1:3991
server1尝试连接zoo.cfg中配置的server列表，这时server2，server3还没启动，因此连接失败， 将server2，server3一并启动后，可以看到server1已经成为Follower
2015-03-02 20:02:41,487 [myid:1] - INFO [/127.0.0.1:3889:QuorumCnxManager$Listener@511] - Received connection request /127.0.0.1:65469
2015-03-02 20:02:41,492 [myid:1] - INFO [WorkerReceiver[myid=1]:FastLeaderElection@597] - Notification: 1 (message format version), 3 (n.leader), 0x1a00000006 (n.zxid)0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x1a (n.peerEpoch) FOLLOWING (my state)
于是，可以使用客户端尝试连接这个集群，并且客户端会被随机分配到一个server
zookeeper-3.4.6/bin/zkCli.sh -server 127.0.0.1:2182,127.0.0.1:2183,127.0.0.1:2184
3.简单测试
列出节点信息：
[zk: 127.0.0.1:2182,127.0.0.1:2183,127.0.0.1:2184(CONNECTED) 11] ls /
[dubbo, zookeeper, pampas, pubsub]
创建/workers节点：
[zk: 127.0.0.1:2182,127.0.0.1:2183,127.0.0.1:2184(CONNECTED) 11] create /workers ""
[zk: 127.0.0.1:2182,127.0.0.1:2183,127.0.0.1:2184(CONNECTED) 11] ls /
[dubbo, zookeeper, workers, pampas, pubsub]
删除/workers节点：
[zk: 127.0.0.1:2182,127.0.0.1:2183,127.0.0.1:2184(CONNECTED) 11] delete /workers
[zk: localhost:2181(CONNECTED) 4] ls /
[zk: 127.0.0.1:2182,127.0.0.1:2183,127.0.0.1:2184(CONNECTED) 11]
[zk: localhost:2181(CONNECTED) 5] quit
通过以上步骤，我们就成功搭建出一套三节点zookeeper伪集群环境，为后面原理验证和python接口开发打好基础。