• zookeeper 实战案例分享:cruator客户端编程


    上两篇介绍了zookeeper服务器端的安装和配置,今天分享下利用zookeeper客户端编程来实现配置文件的统一管理,包括文件添加、删除、更新的同步。

    比如,连接数据库信息的配置文件,一般每个应用服务器代码上都会存放。某个时候如果我想添加一个新数据库用户连接,那么对应用到的服务器上配置文件都要修改一遍,对于上百台的服务器,一一修改显然不现实,这时,我们可以把配置文件统一放到zookeeper服务器上,我们只需要更改zookeeper服务器上的配置文件,然后所有应用服务器上的zookeeper客户端监听到zookeeper服务器上的配置文件被改变了,进行实时同步就可以了。

    主要的结构图如下:

    【环境介绍】:为了看得清楚,我们这次引入两台机器,分别扮演服务器端和客户端

       服务器端(liunx系统):10.126.101.153
    
       客户端(windows):  10.249.9.19

    编程环境:java+eclipse+maven+cruator

    一、启动zookeeper服务(10.126.101.153)

    [root@test bin]# ./zkServer.sh start
    JMX enabled by default
    Using config: /home/mysql/darren/software/zookeeper-3.4.6/bin/../conf/zoo.cfg
    Starting zookeeper ... STARTED

    二、编写客户端代码(java)

    由于ZooKeeper自带的客户端API太底层, 程序员在使用的时候需要自己处理很多事情,更为关键的是,使用过程中如果不当会产生很多问题,如:

    • 初始化连接的问题: 在client与server之间握手建立连接的过程中, 如果握手失败, 执行所有的同步方法(比如create, getData等)将抛出异常
    • 自动恢复(failover)的问题: 当client与一台server的连接丢失,并试图去连接另外一台server时, client将回到初始连接模式
    • session过期的问题: 在极端情况下, 出现ZooKeeper session过期, 客户端需要自己去监听该状态并重新创建

    为了方便我们编程和简化应用,这里使用cruator框架。关于cruator框架的一些介绍可以参考这里curator解决了zookeeper client哪些问题

    使用cruator时,建议采用maven管理项目,maven可以为我们很容易的引入cruator依赖包,关于maven安装和配置参考这里。eclipse4.5版本以上一般都已经安装m2了,只需要配置即可。 

    1、采用eclipse创建maven项目

    New->Other->Maven->Maven Project,然后一直默认next,填写Group Id,Artifact Id,Package,点击finish即可。

    2、配置pom.xml文件,引入cruator依赖包

    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
      <modelVersion>4.0.0</modelVersion>
    
      <groupId>com.darren</groupId>
      <artifactId>zk_client</artifactId>
      <version>0.0.1-SNAPSHOT</version>
      <packaging>jar</packaging>
    
      <name>zk_client</name>
      <url>http://maven.apache.org</url>
    
      <properties>
            <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        </properties>
    
        <dependencies>
    
            <dependency>
                <groupId>junit</groupId>
                <artifactId>junit</artifactId>
                <version>3.8.1</version>
                <scope>test</scope>
            </dependency>
            <dependency>
                <groupId>com.netflix.curator</groupId>
                <artifactId>curator-recipes</artifactId>
                <version>1.3.0</version>
            </dependency>
            <dependency>
                <groupId>com.googlecode.json-simple</groupId>
                <artifactId>json-simple</artifactId>
                <version>1.1.1</version>
            </dependency>
            <dependency>
                <groupId>commons-daemon</groupId>
                <artifactId>commons-daemon</artifactId>
                <version>1.0.10</version>
            </dependency>
        </dependencies>  
    </project>

    3、编写zookeeper 客户端操作代码

    使用cruator编写zk客户端,一般分为以下几个步骤:

     1)使用CuratorFrameworkFactory建立客户端到zookeeper服务器端的连接;

     2)创建PathChildrenCache对象,表示zk路径节点及数据

     3)为PathChildrenCache对象建立监听,这里需要实现PathChildrenCacheListener接口,用于处理监听事件逻辑。

    代码如下:

     1 package com.darren.zk_client;
     2 
     3 import com.netflix.curator.framework.CuratorFramework;
     4 import com.netflix.curator.framework.CuratorFrameworkFactory;
     5 import com.netflix.curator.framework.recipes.cache.PathChildrenCache;
     6 import com.netflix.curator.framework.recipes.cache.PathChildrenCacheEvent;
     7 import com.netflix.curator.framework.recipes.cache.PathChildrenCacheListener;
     8 import com.netflix.curator.retry.ExponentialBackoffRetry;
     9 import com.netflix.curator.retry.RetryUntilElapsed;
    10 import com.netflix.curator.utils.ZKPaths;
    11 
    12 public class App 
    13 {
    14     private static final String PATH = "/db/pools";
    15     
    16     @SuppressWarnings("resource")
    17     public static void main( String[] args ) throws Exception
    18     {
    19         CuratorFramework client = null;
    20         PathChildrenCache cache = null;
    21         try{
    //使用CuratorFrameworkFactory建立客户端到zookeeper服务器端的连接并启动
    22 client = CuratorFrameworkFactory.newClient("127.0.0.1:2181", new RetryUntilElapsed(3000, 1000)); 23 client.start();
    //创建PathChildrenCache对象,表示zk路径节点及数据
    24 cache = new PathChildrenCache(client,PATH,true);
    //为PathChildrenCache对象建立监听,这里需要实现PathChildrenCacheListener接口,用于处理监听事件逻辑
    25 cache.getListenable().addListener(new Listener()); 26 cache.start(); 27 } catch(Exception e){ 28 29 } 30 //表示该java程序一直运行,如果没有,每次java启动后就自动退出了 31 while (true) { 32 Thread.sleep(500); 33 } 34 } 35 36 //内部类,用户实现监听接口 37 private static class Listener implements PathChildrenCacheListener{ 38 39 @Override 40 public void childEvent(CuratorFramework client, PathChildrenCacheEvent event) throws Exception { 41 // TODO Auto-generated method stub 42 switch (event.getType()){ 43 case CHILD_ADDED:{ 44 System.out.println("Node added: " + ZKPaths.getNodeFromPath(event.getData().getPath())); 45 break; 46 } 47 case CHILD_UPDATED:{ 48 System.out.println("Node changed: " + ZKPaths.getNodeFromPath(event.getData().getPath())); 49 break; 50 } 51 case CHILD_REMOVED:{ 52 System.out.println("Node removed: " + ZKPaths.getNodeFromPath(event.getData().getPath())); 53 break; 54 } 55 default: 56 System.out.println("no node changed!!!"); 57 break; 58 } 59 } 60 61 } 62 63 }

    运行效果:

    2016-12-01 16:52:42,394 INFO  com.netflix.curator.framework.imps.CuratorFrameworkImpl - Starting
    2016-12-01 16:52:42,482 INFO  org.apache.zookeeper.ZooKeeper - Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
    2016-12-01 16:52:42,482 INFO  org.apache.zookeeper.ZooKeeper - Client environment:host.name=507B9D97E083.anjuke.net
    2016-12-01 16:52:42,482 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.version=1.7.0_79
    2016-12-01 16:52:42,482 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.vendor=Oracle Corporation
    2016-12-01 16:52:42,482 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.home=C:Program FilesJavajdk1.7.0_79jre
    2016-12-01 16:52:42,482 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.class.path=E:workdirjavazk_client	argetclasses;E:workdirjava
    epocom
    etflixcuratorcurator-recipes1.3.0curator-recipes-1.3.0.jar;E:workdirjava
    epocomgoogleguavaguava11.0.1guava-11.0.1.jar;E:workdirjava
    epocomgooglecodefindbugsjsr3051.3.9jsr305-1.3.9.jar;E:workdirjava
    epocom
    etflixcuratorcurator-framework1.3.0curator-framework-1.3.0.jar;E:workdirjava
    epocom
    etflixcuratorcurator-client1.3.0curator-client-1.3.0.jar;E:workdirjava
    epoorgapachezookeeperzookeeper3.4.5zookeeper-3.4.5.jar;E:workdirjava
    epoorgslf4jslf4j-api1.6.1slf4j-api-1.6.1.jar;E:workdirjava
    epoorgslf4jslf4j-log4j121.6.1slf4j-log4j12-1.6.1.jar;E:workdirjava
    epolog4jlog4j1.2.15log4j-1.2.15.jar;E:workdirjava
    epojavaxmailmail1.4mail-1.4.jar;E:workdirjava
    epojavaxactivationactivation1.1activation-1.1.jar;E:workdirjava
    epojlinejline.9.94jline-0.9.94.jar;E:workdirjava
    epoorgjboss
    etty
    etty3.2.2.Final
    etty-3.2.2.Final.jar;E:workdirjava
    epocomgooglecodejson-simplejson-simple1.1.1json-simple-1.1.1.jar;E:workdirjava
    epocommons-daemoncommons-daemon1.0.10commons-daemon-1.0.10.jar
    2016-12-01 16:52:42,483 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.library.path=C:Program FilesJavajdk1.7.0_79in;C:WindowsSunJavain;C:Windowssystem32;C:Windows;C:/Program Files/Java/jdk1.7.0_79/bin/../jre/bin/server;C:/Program Files/Java/jdk1.7.0_79/bin/../jre/bin;C:/Program Files/Java/jdk1.7.0_79/bin/../jre/lib/amd64;C:Program FilesJavajdk1.7.0_79in;C:Program FilesJavajdk1.7.0_79jrein;C:Perl64sitein;C:Perl64in;C:Python27;C:Python27Scripts;C:Program Files (x86)InteliCLS Client;C:Program FilesInteliCLS Client;C:Windowssystem32;C:Windows;C:WindowsSystem32Wbem;C:WindowsSystem32WindowsPowerShellv1.0;C:Program FilesIntelIntel(R) Management Engine ComponentsDAL;C:Program Files (x86)IntelIntel(R) Management Engine ComponentsDAL;C:Program FilesIntelIntel(R) Management Engine ComponentsIPT;C:Program Files (x86)IntelIntel(R) Management Engine ComponentsIPT;C:Program Files (x86)MITKerberosin;C:CygwinPortableAppCygwinin;C:Program FilesTortoiseGitin;C:Program FilesGitcmd;D:Program filesapache-maven-3.3.9in;D:Program fileseclipse;;.
    2016-12-01 16:52:42,483 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.io.tmpdir=C:UsersHAILIA~1AppDataLocalTemp
    2016-12-01 16:52:42,483 INFO  org.apache.zookeeper.ZooKeeper - Client environment:java.compiler=<NA>
    2016-12-01 16:52:42,483 INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.name=Windows 7
    2016-12-01 16:52:42,484 INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.arch=amd64
    2016-12-01 16:52:42,484 INFO  org.apache.zookeeper.ZooKeeper - Client environment:os.version=6.1
    2016-12-01 16:52:42,484 INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.name=hailiangzhao
    2016-12-01 16:52:42,484 INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.home=C:Usershailiangzhao
    2016-12-01 16:52:42,484 INFO  org.apache.zookeeper.ZooKeeper - Client environment:user.dir=E:workdirjavazk_client
    2016-12-01 16:52:42,486 INFO  org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=10.126.101.153:2181 sessionTimeout=60000 watcher=com.netflix.curator.ConnectionState@271816d1
    2016-12-01 16:52:42,718 INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server tjtx-101-153.58os.org/10.126.101.153:2181. Will not attempt to authenticate using SASL (unknown error)
    2016-12-01 16:52:42,752 INFO  org.apache.zookeeper.ClientCnxn - Socket connection established to tjtx-101-153.58os.org/10.126.101.153:2181, initiating session
    2016-12-01 16:52:42,796 INFO  org.apache.zookeeper.ClientCnxn - Session establishment complete on server tjtx-101-153.58os.org/10.126.101.153:2181, sessionid = 0x158b95fadec0001, negotiated timeout = 40000
    2016-12-01 16:52:42,801 INFO  com.netflix.curator.framework.state.ConnectionStateManager - State change: CONNECTED
    Node added: darren.php

    上面的控制台信息表示client已经与10.126.101.153:2181 zookeeper服务端建立了连接,并且对服务器上的/db/pools节点进行了监控,只要这下面的数据有任何变化,都会输出变化的node和内容。

    这时,我们可以在服务器端进行如下操作:

    [zk: 10.126.101.153:2181(CONNECTED) 5] create /db/pools/darren1.php "darren1.php"
    Created /db/pools/darren1.php
    [zk: 10.126.101.153:2181(CONNECTED) 6] set  /db/pools/darren1.php "xxxxxx"       
    cZxid = 0x7
    ctime = Thu Dec 01 16:58:09 CST 2016
    mZxid = 0x8
    mtime = Thu Dec 01 16:58:30 CST 2016
    pZxid = 0x7
    cversion = 0
    dataVersion = 1
    aclVersion = 0
    ephemeralOwner = 0x0
    dataLength = 8
    numChildren = 0
    [zk: 10.126.101.153:2181(CONNECTED) 7] 

    这时客户端控制台追加输出,表示已经监控到了darren1.php内容改变:

    Node added: darren1.php
    Node changed: darren1.php

    okay,使用cruator框架编写zk客户端是不是很容易呢。

    当然,这个小例子很简单,还没有进行配置文件的同步,我们可以对以上程序进行如下几个改进即可实现:

    1、在处理监听事件的内部类里,我们可以对变化的节点内容写入本地文件中,实现了文件同步。

    2、使用daemon接口,把该java程序封装成windows或者linux服务,永久运行,保证实时同步文件。

    遇到的错误:

      【Multiple markers at this line @Override的解决方法】http://blog.csdn.net/mazhaojuan/article/details/28931375
      【maven编译时出现读取XXX时出错invalid LOC header (bad signature)】http://blog.csdn.net/woshixuye/article/details/14669929
      【log4j:WARN No appenders could be found for loggerhttp://www.cnblogs.com/jbelial/archive/2012/06/05/2536814.html 

    【moven参考资料】

      【maven安装配置】http://jingyan.baidu.com/article/1709ad808ad49f4634c4f00d.html

      【maven profile配置】http://elim.iteye.com/blog/1900568 
      【maven处理资源文件】http://www.cnblogs.com/now-fighting/p/4888343.html 
      【maven pom.xml配置说明】http://www.cnblogs.com/qq78292959/p/3711501.html

    【java参考资料】

      【java api官方文档】http://docs.oracle.com/javase/7/docs/api/                    
      【java程序以服务方式运行】http://blog.csdn.net/jason5186/article/details/9146167 

    【cruator参考资料】

      【cruator简介】http://macrochen.iteye.com/blog/1366136/
      【cruator java api】http://curator.apache.org/apidocs/index.html
      【zookeeper客户端编程API】http://zookeeper.apache.org/doc/r3.4.6/javaExample.html
      【cruator 案例】http://blog.sina.com.cn/s/blog_616e189f01018axz.html




  • 相关阅读:
    python中range函数与列表中删除元素
    python碎片记录(一)
    Coursera在线学习---第八节.K-means聚类算法与主成分分析(PCA)
    Coursera在线学习---第七节.支持向量机(SVM)
    Coursera在线学习---第六节.构建机器学习系统
    Coursera在线学习---第五节.Logistic Regression
    梯度下降法与迭代结束条件
    利用并查集求最大生成树和最小生成树(nlogn)
    字符串二分查找
    map容器
  • 原文地址:https://www.cnblogs.com/mysql-dba/p/6122809.html
Copyright © 2020-2023  润新知