上两篇介绍了zookeeper服务器端的安装和配置,今天分享下利用zookeeper客户端编程来实现配置文件的统一管理,包括文件添加、删除、更新的同步。
比如,连接数据库信息的配置文件,一般每个应用服务器代码上都会存放。某个时候如果我想添加一个新数据库用户连接,那么对应用到的服务器上配置文件都要修改一遍,对于上百台的服务器,一一修改显然不现实,这时,我们可以把配置文件统一放到zookeeper服务器上,我们只需要更改zookeeper服务器上的配置文件,然后所有应用服务器上的zookeeper客户端监听到zookeeper服务器上的配置文件被改变了,进行实时同步就可以了。
主要的结构图如下:
【环境介绍】:为了看得清楚,我们这次引入两台机器,分别扮演服务器端和客户端
服务器端(liunx系统):10.126.101.153 客户端(windows): 10.249.9.19
编程环境:java+eclipse+maven+cruator
一、启动zookeeper服务(10.126.101.153)
[root@test bin]# ./zkServer.sh start JMX enabled by default Using config: /home/mysql/darren/software/zookeeper-3.4.6/bin/../conf/zoo.cfg Starting zookeeper ... STARTED
二、编写客户端代码(java)
由于ZooKeeper自带的客户端API太底层, 程序员在使用的时候需要自己处理很多事情,更为关键的是,使用过程中如果不当会产生很多问题,如:
- 初始化连接的问题: 在client与server之间握手建立连接的过程中, 如果握手失败, 执行所有的同步方法(比如create, getData等)将抛出异常
- 自动恢复(failover)的问题: 当client与一台server的连接丢失,并试图去连接另外一台server时, client将回到初始连接模式
- session过期的问题: 在极端情况下, 出现ZooKeeper session过期, 客户端需要自己去监听该状态并重新创建
为了方便我们编程和简化应用,这里使用cruator框架。关于cruator框架的一些介绍可以参考这里curator解决了zookeeper client哪些问题
使用cruator时,建议采用maven管理项目,maven可以为我们很容易的引入cruator依赖包,关于maven安装和配置参考这里。eclipse4.5版本以上一般都已经安装m2了,只需要配置即可。
1、采用eclipse创建maven项目
New->Other->Maven->Maven Project,然后一直默认next,填写Group Id,Artifact Id,Package,点击finish即可。
2、配置pom.xml文件,引入cruator依赖包
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.darren</groupId> <artifactId>zk_client</artifactId> <version>0.0.1-SNAPSHOT</version> <packaging>jar</packaging> <name>zk_client</name> <url>http://maven.apache.org</url> <properties> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> </properties> <dependencies> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>3.8.1</version> <scope>test</scope> </dependency> <dependency> <groupId>com.netflix.curator</groupId> <artifactId>curator-recipes</artifactId> <version>1.3.0</version> </dependency> <dependency> <groupId>com.googlecode.json-simple</groupId> <artifactId>json-simple</artifactId> <version>1.1.1</version> </dependency> <dependency> <groupId>commons-daemon</groupId> <artifactId>commons-daemon</artifactId> <version>1.0.10</version> </dependency> </dependencies> </project>
3、编写zookeeper 客户端操作代码
使用cruator编写zk客户端,一般分为以下几个步骤:
1)使用CuratorFrameworkFactory建立客户端到zookeeper服务器端的连接;
2)创建PathChildrenCache对象,表示zk路径节点及数据
3)为PathChildrenCache对象建立监听,这里需要实现PathChildrenCacheListener接口,用于处理监听事件逻辑。
代码如下:
1 package com.darren.zk_client; 2 3 import com.netflix.curator.framework.CuratorFramework; 4 import com.netflix.curator.framework.CuratorFrameworkFactory; 5 import com.netflix.curator.framework.recipes.cache.PathChildrenCache; 6 import com.netflix.curator.framework.recipes.cache.PathChildrenCacheEvent; 7 import com.netflix.curator.framework.recipes.cache.PathChildrenCacheListener; 8 import com.netflix.curator.retry.ExponentialBackoffRetry; 9 import com.netflix.curator.retry.RetryUntilElapsed; 10 import com.netflix.curator.utils.ZKPaths; 11 12 public class App 13 { 14 private static final String PATH = "/db/pools"; 15 16 @SuppressWarnings("resource") 17 public static void main( String[] args ) throws Exception 18 { 19 CuratorFramework client = null; 20 PathChildrenCache cache = null; 21 try{
//使用CuratorFrameworkFactory建立客户端到zookeeper服务器端的连接并启动 22 client = CuratorFrameworkFactory.newClient("127.0.0.1:2181", new RetryUntilElapsed(3000, 1000)); 23 client.start();
//创建PathChildrenCache对象,表示zk路径节点及数据 24 cache = new PathChildrenCache(client,PATH,true);
//为PathChildrenCache对象建立监听,这里需要实现PathChildrenCacheListener接口,用于处理监听事件逻辑 25 cache.getListenable().addListener(new Listener()); 26 cache.start(); 27 } catch(Exception e){ 28 29 } 30 //表示该java程序一直运行,如果没有,每次java启动后就自动退出了 31 while (true) { 32 Thread.sleep(500); 33 } 34 } 35 36 //内部类,用户实现监听接口 37 private static class Listener implements PathChildrenCacheListener{ 38 39 @Override 40 public void childEvent(CuratorFramework client, PathChildrenCacheEvent event) throws Exception { 41 // TODO Auto-generated method stub 42 switch (event.getType()){ 43 case CHILD_ADDED:{ 44 System.out.println("Node added: " + ZKPaths.getNodeFromPath(event.getData().getPath())); 45 break; 46 } 47 case CHILD_UPDATED:{ 48 System.out.println("Node changed: " + ZKPaths.getNodeFromPath(event.getData().getPath())); 49 break; 50 } 51 case CHILD_REMOVED:{ 52 System.out.println("Node removed: " + ZKPaths.getNodeFromPath(event.getData().getPath())); 53 break; 54 } 55 default: 56 System.out.println("no node changed!!!"); 57 break; 58 } 59 } 60 61 } 62 63 }
运行效果:
2016-12-01 16:52:42,394 INFO com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting 2016-12-01 16:52:42,482 INFO org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT 2016-12-01 16:52:42,482 INFO org.apache.zookeeper.ZooKeeper - Client environment:host.name=507B9D97E083.anjuke.net 2016-12-01 16:52:42,482 INFO org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.7.0_79 2016-12-01 16:52:42,482 INFO org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Oracle Corporation 2016-12-01 16:52:42,482 INFO org.apache.zookeeper.ZooKeeper - Client environment:java.home=C:Program FilesJavajdk1.7.0_79jre 2016-12-01 16:52:42,482 INFO org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=E:workdirjavazk_client argetclasses;E:workdirjava epocom etflixcuratorcurator-recipes1.3.0curator-recipes-1.3.0.jar;E:workdirjava epocomgoogleguavaguava11.0.1guava-11.0.1.jar;E:workdirjava epocomgooglecodefindbugsjsr3051.3.9jsr305-1.3.9.jar;E:workdirjava epocom etflixcuratorcurator-framework1.3.0curator-framework-1.3.0.jar;E:workdirjava epocom etflixcuratorcurator-client1.3.0curator-client-1.3.0.jar;E:workdirjava epoorgapachezookeeperzookeeper3.4.5zookeeper-3.4.5.jar;E:workdirjava epoorgslf4jslf4j-api1.6.1slf4j-api-1.6.1.jar;E:workdirjava epoorgslf4jslf4j-log4j121.6.1slf4j-log4j12-1.6.1.jar;E:workdirjava epolog4jlog4j1.2.15log4j-1.2.15.jar;E:workdirjava epojavaxmailmail1.4mail-1.4.jar;E:workdirjava epojavaxactivationactivation1.1activation-1.1.jar;E:workdirjava epojlinejline .9.94jline-0.9.94.jar;E:workdirjava epoorgjboss etty etty3.2.2.Final etty-3.2.2.Final.jar;E:workdirjava epocomgooglecodejson-simplejson-simple1.1.1json-simple-1.1.1.jar;E:workdirjava epocommons-daemoncommons-daemon1.0.10commons-daemon-1.0.10.jar 2016-12-01 16:52:42,483 INFO org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=C:Program FilesJavajdk1.7.0_79in;C:WindowsSunJavain;C:Windowssystem32;C:Windows;C:/Program Files/Java/jdk1.7.0_79/bin/../jre/bin/server;C:/Program Files/Java/jdk1.7.0_79/bin/../jre/bin;C:/Program Files/Java/jdk1.7.0_79/bin/../jre/lib/amd64;C:Program FilesJavajdk1.7.0_79in;C:Program FilesJavajdk1.7.0_79jrein;C:Perl64sitein;C:Perl64in;C:Python27;C:Python27Scripts;C:Program Files (x86)InteliCLS Client;C:Program FilesInteliCLS Client;C:Windowssystem32;C:Windows;C:WindowsSystem32Wbem;C:WindowsSystem32WindowsPowerShellv1.0;C:Program FilesIntelIntel(R) Management Engine ComponentsDAL;C:Program Files (x86)IntelIntel(R) Management Engine ComponentsDAL;C:Program FilesIntelIntel(R) Management Engine ComponentsIPT;C:Program Files (x86)IntelIntel(R) Management Engine ComponentsIPT;C:Program Files (x86)MITKerberosin;C:CygwinPortableAppCygwinin;C:Program FilesTortoiseGitin;C:Program FilesGitcmd;D:Program filesapache-maven-3.3.9in;D:Program fileseclipse;;. 2016-12-01 16:52:42,483 INFO org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=C:UsersHAILIA~1AppDataLocalTemp 2016-12-01 16:52:42,483 INFO org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA> 2016-12-01 16:52:42,483 INFO org.apache.zookeeper.ZooKeeper - Client environment:os.name=Windows 7 2016-12-01 16:52:42,484 INFO org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64 2016-12-01 16:52:42,484 INFO org.apache.zookeeper.ZooKeeper - Client environment:os.version=6.1 2016-12-01 16:52:42,484 INFO org.apache.zookeeper.ZooKeeper - Client environment:user.name=hailiangzhao 2016-12-01 16:52:42,484 INFO org.apache.zookeeper.ZooKeeper - Client environment:user.home=C:Usershailiangzhao 2016-12-01 16:52:42,484 INFO org.apache.zookeeper.ZooKeeper - Client environment:user.dir=E:workdirjavazk_client 2016-12-01 16:52:42,486 INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=10.126.101.153:2181 sessionTimeout=60000 watcher=com.netflix.curator.ConnectionState@271816d1 2016-12-01 16:52:42,718 INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server tjtx-101-153.58os.org/10.126.101.153:2181. Will not attempt to authenticate using SASL (unknown error) 2016-12-01 16:52:42,752 INFO org.apache.zookeeper.ClientCnxn - Socket connection established to tjtx-101-153.58os.org/10.126.101.153:2181, initiating session 2016-12-01 16:52:42,796 INFO org.apache.zookeeper.ClientCnxn - Session establishment complete on server tjtx-101-153.58os.org/10.126.101.153:2181, sessionid = 0x158b95fadec0001, negotiated timeout = 40000 2016-12-01 16:52:42,801 INFO com.netflix.curator.framework.state.ConnectionStateManager - State change: CONNECTED Node added: darren.php
上面的控制台信息表示client已经与10.126.101.153:2181 zookeeper服务端建立了连接,并且对服务器上的/db/pools节点进行了监控,只要这下面的数据有任何变化,都会输出变化的node和内容。
这时,我们可以在服务器端进行如下操作:
[zk: 10.126.101.153:2181(CONNECTED) 5] create /db/pools/darren1.php "darren1.php" Created /db/pools/darren1.php [zk: 10.126.101.153:2181(CONNECTED) 6] set /db/pools/darren1.php "xxxxxx" cZxid = 0x7 ctime = Thu Dec 01 16:58:09 CST 2016 mZxid = 0x8 mtime = Thu Dec 01 16:58:30 CST 2016 pZxid = 0x7 cversion = 0 dataVersion = 1 aclVersion = 0 ephemeralOwner = 0x0 dataLength = 8 numChildren = 0 [zk: 10.126.101.153:2181(CONNECTED) 7]
这时客户端控制台追加输出,表示已经监控到了darren1.php内容改变:
Node added: darren1.php
Node changed: darren1.php
okay,使用cruator框架编写zk客户端是不是很容易呢。
当然,这个小例子很简单,还没有进行配置文件的同步,我们可以对以上程序进行如下几个改进即可实现:
1、在处理监听事件的内部类里,我们可以对变化的节点内容写入本地文件中,实现了文件同步。
2、使用daemon接口,把该java程序封装成windows或者linux服务,永久运行,保证实时同步文件。
【遇到的错误:】
【Multiple markers at this line @Override的解决方法】http://blog.csdn.net/mazhaojuan/article/details/28931375
【maven编译时出现读取XXX时出错invalid LOC header (bad signature)】http://blog.csdn.net/woshixuye/article/details/14669929
【log4j:WARN No appenders could be found for logger】http://www.cnblogs.com/jbelial/archive/2012/06/05/2536814.html
【moven参考资料】
【maven安装配置】http://jingyan.baidu.com/article/1709ad808ad49f4634c4f00d.html
【maven profile配置】http://elim.iteye.com/blog/1900568
【maven处理资源文件】http://www.cnblogs.com/now-fighting/p/4888343.html
【maven pom.xml配置说明】http://www.cnblogs.com/qq78292959/p/3711501.html
【java参考资料】
【java api官方文档】http://docs.oracle.com/javase/7/docs/api/
【java程序以服务方式运行】http://blog.csdn.net/jason5186/article/details/9146167
【cruator参考资料】
【cruator简介】http://macrochen.iteye.com/blog/1366136/
【cruator java api】http://curator.apache.org/apidocs/index.html
【zookeeper客户端编程API】http://zookeeper.apache.org/doc/r3.4.6/javaExample.html
【cruator 案例】http://blog.sina.com.cn/s/blog_616e189f01018axz.html