Dubbo发布过程中，注册中心监听服务变化

目录

• 前言
• 监听服务的绑定时机
  • OverrideListener
  • ZookeeperRegistry.subscribe
    • ZookeeperRegistry
    • FailbackRegistry
    • AbstractRegistry
      • zkClient的addChildListener为配置路径添加监听器
        • AbstractZookeeperClient
        • CuratorZookeeperClient
  • FailbackRegistry.notify
    • FailbackRegistry
    • • saveProperties保存配置到本地文件
      • OverrideListener的notify
  • DestroyableExporter

前言

在上一节中，我们分析了Dubbo根据RegistryUrl获取到了Registry实例对象，并根据providerUrl注册了Provider的服务地址信息节点，现在我们继续查看服务地址变化的情况下的监听服务是如何工作的？

监听服务的绑定时机

第一个红框中是服务地址注册，第二个红框是监听服务的注册

首先调用了getSubscribedOverrideUrl方法来获取overrideSubscribeUrl ,随后调用OverrideListener方法构建一个监听者，监听者url为overrideSubscribeUrl =provider://192.168.31.199:20881/com.bail.user.service.IUserService?XXX;随后将overrideSubscribeListener 放到通知者缓存中；最后调用了ZookeeperRegistry.subscribe的订阅方法进行监听。
执行完registry.subscribe方法，构建了一个DestroyableExporter对象并返回，并保存到了ServiceConfig的缓存对象List中

    private final Map<URL, NotifyListener> overrideListeners = new ConcurrentHashMap<URL, NotifyListener>();
        final URL overrideSubscribeUrl = getSubscribedOverrideUrl(registedProviderUrl);
        final OverrideListener overrideSubscribeListener = new OverrideListener(overrideSubscribeUrl, originInvoker);
        overrideListeners.put(overrideSubscribeUrl, overrideSubscribeListener);
        registry.subscribe(overrideSubscribeUrl, overrideSubscribeListener);
        //Ensure that a new exporter instance is returned every time export
        return new DestroyableExporter<T>(exporter, originInvoker, overrideSubscribeUrl, registedProviderUrl);

getSubscribedOverrideUrl方法中注册设置了protocol为provider,并添加了category参数，值为configurators，得到的结果为

provider://192.168.31.199:20881/com.bail.user.service.IUserService?anyhost=true&application=user-provider&category=configurators&check=false&dubbo=2.6.2&generic=false&getUserById.retries=3&getUserById.timeout=3000&interface=com.bail.user.service.IUserService&methods=getUserById,queryList&pid=12672&retries=2&revision=1.0.0&side=provider&timeout=8000&timestamp=1642302952925&version=1.0.0

    public static final String CATEGORY_KEY = "category";
    public static final String CONFIGURATORS_CATEGORY = "configurators";
    private URL getSubscribedOverrideUrl(URL registedProviderUrl) {
        return registedProviderUrl.setProtocol(Constants.PROVIDER_PROTOCOL)
                .addParameters(Constants.CATEGORY_KEY, Constants.CONFIGURATORS_CATEGORY,
                        Constants.CHECK_KEY, String.valueOf(false));
    }

OverrideListener

OverrideListener是RegistryProtocol的一个内部类

public class RegistryProtocol implements Protocol {
    private class OverrideListener implements NotifyListener {
        private final URL subscribeUrl;
        private final Invoker originInvoker;

        public OverrideListener(URL subscribeUrl, Invoker originalInvoker) {
            this.subscribeUrl = subscribeUrl;
            this.originInvoker = originalInvoker;
        }
  }
}

ZookeeperRegistry.subscribe

ZookeeperRegistry

父类的subscribe调用了ZookeeperRegistry 的doSubscribe方法，看一下当前都有哪些动作方法？
在else逻辑中，当前url =

provider://10.9.233.26:20881/com.bail.user.service.IUserService?anyhost=true&application=user-provider&category=configurators&check=false&dubbo=2.6.2&generic=false&getUserById.retries=3&getUserById.timeout=3000&interface=com.bail.user.service.IUserService&methods=getUserById,queryList&pid=24288&retries=2&revision=1.0.0&side=provider&timeout=8000&timestamp=1642126934044&version=1.0.0

调用toCategoriesPath方法得到categoriesPath pahts[0] = "/dubbo/com.bail.user.service.IUserService/configurators"
根据path创建配置节点，为当前路径添加监听者，调用zkClient的addChildListener方法，
根据url, path, children三个参数得到的url如下图

empty://10.9.233.26:20881/com.bail.user.service.IUserService?anyhost=true&application=user-provider&category=configurators&check=false&dubbo=2.6.2&generic=false&getUserById.retries=3&getUserById.timeout=3000&interface=com.bail.user.service.IUserService&methods=getUserById,queryList&pid=24288&retries=2&revision=1.0.0&side=provider&timeout=8000&timestamp=1642126934044&version=1.0.0

添加完addChildListener监听之后，将url转为empty url并放到了一个URL缓存里面，作为后续使用。

获取到urls之后，调用父类FailbackRegistry 的notify方法

public class ZookeeperRegistry extends FailbackRegistry {
    // url对应的订阅者
    private final ConcurrentMap<URL, ConcurrentMap<NotifyListener, ChildListener>> zkListeners = new ConcurrentHashMap<URL, ConcurrentMap<NotifyListener, ChildListener>>();
    protected void doSubscribe(final URL url, final NotifyListener listener) {
        try {
            if (Constants.ANY_VALUE.equals(url.getServiceInterface())) {
                String root = toRootPath();
                ConcurrentMap<NotifyListener, ChildListener> listeners = zkListeners.get(url);
                if (listeners == null) {
                    zkListeners.putIfAbsent(url, new ConcurrentHashMap<NotifyListener, ChildListener>());
                    listeners = zkListeners.get(url);
                }
                ChildListener zkListener = listeners.get(listener);
                if (zkListener == null) {
                    listeners.putIfAbsent(listener, new ChildListener() {
                        @Override
                        public void childChanged(String parentPath, List<String> currentChilds) {
                            for (String child : currentChilds) {
                                child = URL.decode(child);
                                if (!anyServices.contains(child)) {
                                    anyServices.add(child);
                                    subscribe(url.setPath(child).addParameters(Constants.INTERFACE_KEY, child,
                                            Constants.CHECK_KEY, String.valueOf(false)), listener);
                                }
                            }
                        }
                    });
                    zkListener = listeners.get(listener);
                }
                zkClient.create(root, false);
                List<String> services = zkClient.addChildListener(root, zkListener);
                if (services != null && !services.isEmpty()) {
                    for (String service : services) {
                        service = URL.decode(service);
                        anyServices.add(service);
                        subscribe(url.setPath(service).addParameters(Constants.INTERFACE_KEY, service,
                                Constants.CHECK_KEY, String.valueOf(false)), listener);
                    }
                }
            } else {
                // url.getServiceInterface() = com.bail.user.service.IUserService,进入else逻辑
                List<URL> urls = new ArrayList<URL>();
                for (String path : toCategoriesPath(url)) {
                    ConcurrentMap<NotifyListener, ChildListener> listeners = zkListeners.get(url);
                    if (listeners == null) {
                        zkListeners.putIfAbsent(url, new ConcurrentHashMap<NotifyListener, ChildListener>());
                        listeners = zkListeners.get(url);
                    }
                    // 初始zkListener为空，添加listener对应的子事件，并添加了一个childChanged事件动作，在动作中触发notify方法，通知监听者
                    ChildListener zkListener = listeners.get(listener);
                    if (zkListener == null) {
                        listeners.putIfAbsent(listener, new ChildListener() {
                            @Override
                            public void childChanged(String parentPath, List<String> currentChilds) {
                                ZookeeperRegistry.this.notify(url, listener, toUrlsWithEmpty(url, parentPath, currentChilds));
                            }
                        });
                        zkListener = listeners.get(listener);
                    }
                    // 根据path创建配置节点
                    zkClient.create(path, false);
                    // 为当前路径添加监听者，调用zkClient的addChildListener方法
                    List<String> children = zkClient.addChildListener(path, zkListener);
                    if (children != null) {
                        urls.addAll(toUrlsWithEmpty(url, path, children));
                    }
                }
                notify(url, listener, urls);
            }
        } catch (Throwable e) {
            throw new RpcException("Failed to subscribe " + url + " to zookeeper " + getUrl() + ", cause: " + e.getMessage(), e);
        }
    }

    private String[] toCategoriesPath(URL url) {
        String[] categories;
        if (Constants.ANY_VALUE.equals(url.getParameter(Constants.CATEGORY_KEY))) {
            categories = new String[]{Constants.PROVIDERS_CATEGORY, Constants.CONSUMERS_CATEGORY,
                    Constants.ROUTERS_CATEGORY, Constants.CONFIGURATORS_CATEGORY};
        } else {
            categories = url.getParameter(Constants.CATEGORY_KEY, new String[]{Constants.DEFAULT_CATEGORY});
        }
        String[] paths = new String[categories.length];
        for (int i = 0; i < categories.length; i++) {
            paths[i] = toServicePath(url) + Constants.PATH_SEPARATOR + categories[i];
        }
        return paths;
    }
}

notify方法调用了父类的doNotify方法

FailbackRegistry

public abstract class FailbackRegistry extends AbstractRegistry {
    public void subscribe(URL url, NotifyListener listener) {
        // 添加监听者
        super.subscribe(url, listener);
        removeFailedSubscribed(url, listener);
        try {
            // Sending a subscription request to the server side
            // 发布一个订阅请求到服务端，调用子类的具体实现方法
            doSubscribe(url, listener);
        } catch (Exception e) {
            Throwable t = e;

            List<URL> urls = getCacheUrls(url);
            if (urls != null && !urls.isEmpty()) {
                notify(url, listener, urls);
                logger.error("Failed to subscribe " + url + ", Using cached list: " + urls + " from cache file: " + getUrl().getParameter(Constants.FILE_KEY, System.getProperty("user.home") + "/dubbo-registry-" + url.getHost() + ".cache") + ", cause: " + t.getMessage(), t);
            } else {
                // If the startup detection is opened, the Exception is thrown directly.
                boolean check = getUrl().getParameter(Constants.CHECK_KEY, true)
                        && url.getParameter(Constants.CHECK_KEY, true);
                boolean skipFailback = t instanceof SkipFailbackWrapperException;
                if (check || skipFailback) {
                    if (skipFailback) {
                        t = t.getCause();
                    }
                    throw new IllegalStateException("Failed to subscribe " + url + ", cause: " + t.getMessage(), t);
                } else {
                    logger.error("Failed to subscribe " + url + ", waiting for retry, cause: " + t.getMessage(), t);
                }
            }

            // Record a failed registration request to a failed list, retry regularly
            addFailedSubscribed(url, listener);
        }
    }

    // ==== Template method ====

    protected abstract void doRegister(URL url);

    protected abstract void doUnregister(URL url);

    protected abstract void doSubscribe(URL url, NotifyListener listener);

    protected abstract void doUnsubscribe(URL url, NotifyListener listener);
}

AbstractRegistry

在父类中添加前面构建的监听者
public abstract class AbstractRegistry implements Registry {
    public void subscribe(URL url, NotifyListener listener) {
        if (url == null) {
            throw new IllegalArgumentException("subscribe url == null");
        }
        if (listener == null) {
            throw new IllegalArgumentException("subscribe listener == null");
        }
        if (logger.isInfoEnabled()) {
            logger.info("Subscribe: " + url);
        }
        Set<NotifyListener> listeners = subscribed.get(url);
        if (listeners == null) {
            subscribed.putIfAbsent(url, new ConcurrentHashSet<NotifyListener>());
            listeners = subscribed.get(url);
        }
        listeners.add(listener);
    }
}

consumerUrl

provider://10.9.233.26:20881/com.bail.user.service.IUserService?anyhost=true&application=user-provider&category=configurators&check=false&dubbo=2.6.2&generic=false&getUserById.retries=3&getUserById.timeout=3000&interface=com.bail.user.service.IUserService&methods=getUserById,queryList&pid=24288&retries=2&revision=1.0.0&side=provider&timeout=8000&timestamp=1642126934044&version=1.0.0

providerUrl

empty://10.9.233.26:20881/com.bail.user.service.IUserService?anyhost=true&application=user-provider&category=configurators&check=false&dubbo=2.6.2&generic=false&getUserById.retries=3&getUserById.timeout=3000&interface=com.bail.user.service.IUserService&methods=getUserById,queryList&pid=24288&retries=2&revision=1.0.0&side=provider&timeout=8000&timestamp=1642126934044&version=1.0.0

isMatch方法返回true

    public static boolean isMatch(URL consumerUrl, URL providerUrl) {
        // consumerInterface  = providerInterface = com.bail.user.service.IUserService
        String consumerInterface = consumerUrl.getServiceInterface();
        String providerInterface = providerUrl.getServiceInterface();
        if (!(Constants.ANY_VALUE.equals(consumerInterface) || StringUtils.isEquals(consumerInterface, providerInterface)))
            return false;

        if (!isMatchCategory(providerUrl.getParameter(Constants.CATEGORY_KEY, Constants.DEFAULT_CATEGORY),
                consumerUrl.getParameter(Constants.CATEGORY_KEY, Constants.DEFAULT_CATEGORY))) {
            return false;
        }
        if (!providerUrl.getParameter(Constants.ENABLED_KEY, true)
                && !Constants.ANY_VALUE.equals(consumerUrl.getParameter(Constants.ENABLED_KEY))) {
            return false;
        }

        String consumerGroup = consumerUrl.getParameter(Constants.GROUP_KEY);
        String consumerVersion = consumerUrl.getParameter(Constants.VERSION_KEY);
        String consumerClassifier = consumerUrl.getParameter(Constants.CLASSIFIER_KEY, Constants.ANY_VALUE);

        String providerGroup = providerUrl.getParameter(Constants.GROUP_KEY);
        String providerVersion = providerUrl.getParameter(Constants.VERSION_KEY);
        String providerClassifier = providerUrl.getParameter(Constants.CLASSIFIER_KEY, Constants.ANY_VALUE);
        return (Constants.ANY_VALUE.equals(consumerGroup) || StringUtils.isEquals(consumerGroup, providerGroup) || StringUtils.isContains(consumerGroup, providerGroup))
                && (Constants.ANY_VALUE.equals(consumerVersion) || StringUtils.isEquals(consumerVersion, providerVersion))
                && (consumerClassifier == null || Constants.ANY_VALUE.equals(consumerClassifier) || StringUtils.isEquals(consumerClassifier, providerClassifier));
    }

zkClient的addChildListener为配置路径添加监听器

AbstractZookeeperClient

createTargetChildListener(path, listener)方法返回一个CuratorWatcherImpl 实例对象。然后调用addTargetChildListener方法

public abstract class AbstractZookeeperClient<TargetChildListener> implements ZookeeperClient {
    // 缓存监听器
    private final ConcurrentMap<String, ConcurrentMap<ChildListener, TargetChildListener>> childListeners = new ConcurrentHashMap<String, ConcurrentMap<ChildListener, TargetChildListener>>();

    // 添加子监听器
    public List<String> addChildListener(String path, final ChildListener listener) {
        // listeners当前为空
        ConcurrentMap<ChildListener, TargetChildListener> listeners = childListeners.get(path);
        if (listeners == null) {
            childListeners.putIfAbsent(path, new ConcurrentHashMap<ChildListener, TargetChildListener>());
            listeners = childListeners.get(path);
        }
        // 当前的目标监听器为空
        TargetChildListener targetListener = listeners.get(listener);
        if (targetListener == null) {
            listeners.putIfAbsent(listener, createTargetChildListener(path, listener));
            targetListener = listeners.get(listener);
        }
        // addTargetChildListener以当前dubbo.com.bail.user.service.IUserService.configurators路径为key,
        // CuratorWatcherImpl 为动作相应添加了一个目标子路径监听器
        return addTargetChildListener(path, targetListener);
    }

    protected abstract void doClose();

    protected abstract void createPersistent(String path);

    protected abstract void createEphemeral(String path);

    protected abstract boolean checkExists(String path);

    protected abstract TargetChildListener createTargetChildListener(String path, ChildListener listener);

    protected abstract List<String> addTargetChildListener(String path, TargetChildListener listener);

    protected abstract void removeTargetChildListener(String path, TargetChildListener listener);
}

CuratorZookeeperClient

addTargetChildListener 为当前路径添加事件响应动作并监听

public class CuratorZookeeperClient extends AbstractZookeeperClient<CuratorWatcher> {
    private final CuratorFramework client;

    public CuratorWatcher createTargetChildListener(String path, ChildListener listener) {
        return new CuratorWatcherImpl(listener);
    }

    @Override
    public List<String> addTargetChildListener(String path, CuratorWatcher listener) {
        try {
            return client.getChildren().usingWatcher(listener).forPath(path);
        } catch (NoNodeException e) {
            return null;
        } catch (Exception e) {
            throw new IllegalStateException(e.getMessage(), e);
        }
    }
private class CuratorWatcherImpl implements CuratorWatcher {

        private volatile ChildListener listener;

        public CuratorWatcherImpl(ChildListener listener) {
            this.listener = listener;
        }

        public void unwatch() {
            this.listener = null;
        }

        @Override
        public void process(WatchedEvent event) throws Exception {
            if (listener != null) {
                String path = event.getPath() == null ? "" : event.getPath();
                listener.childChanged(path,
                        // if path is null, curator using watcher will throw NullPointerException.
                        // if client connect or disconnect to server, zookeeper will queue
                        // watched event(Watcher.Event.EventType.None, .., path = null).
                        StringUtils.isNotEmpty(path)
                                ? client.getChildren().usingWatcher(this).forPath(path)
                                : Collections.<String>emptyList());
            }
        }
    }
}

FailbackRegistry.notify

FailbackRegistry

方法内部首先调用了自身的doNotify方法，doNotify方法内调用了父类的notify方法

public abstract class FailbackRegistry extends AbstractRegistry {
    protected void notify(URL url, NotifyListener listener, List<URL> urls) {
        if (url == null) {
            throw new IllegalArgumentException("notify url == null");
        }
        if (listener == null) {
            throw new IllegalArgumentException("notify listener == null");
        }
        try {
            doNotify(url, listener, urls);
        } catch (Exception t) {
            // Record a failed registration request to a failed list, retry regularly
            Map<NotifyListener, List<URL>> listeners = failedNotified.get(url);
            if (listeners == null) {
                failedNotified.putIfAbsent(url, new ConcurrentHashMap<NotifyListener, List<URL>>());
                listeners = failedNotified.get(url);
            }
            listeners.put(listener, urls);
            logger.error("Failed to notify for subscribe " + url + ", waiting for retry, cause: " + t.getMessage(), t);
        }
    }

    protected void doNotify(URL url, NotifyListener listener, List<URL> urls) {
        super.notify(url, listener, urls);
    }
}

notify方法中，首先调用了UrlUtils.isMatch(url, u)来判断提供者url和urls中的empty url路径是否匹配，categoryList.add(u)添加当前empty路径,并将整个categoryList添加到了以configurators为key的map中；
接下来从notified缓存中获取到了以providerUrl为key的map，并将当前的路径信息result添加到了notified中，调用了saveProperties进行属性文件保存，最后调用了前面添加的OverrideListener的notify方法

public abstract class AbstractRegistry implements Registry {
    private File file;
    private final ConcurrentMap<URL, Map<String, List<URL>>> notified = new ConcurrentHashMap<URL, Map<String, List<URL>>>();
    protected void notify(URL url, NotifyListener listener, List<URL> urls) {
       ......
        Map<String, List<URL>> result = new HashMap<String, List<URL>>();
        for (URL u : urls) {
            if (UrlUtils.isMatch(url, u)) {
                String category = u.getParameter(Constants.CATEGORY_KEY, Constants.DEFAULT_CATEGORY);
                List<URL> categoryList = result.get(category);
                if (categoryList == null) {
                    categoryList = new ArrayList<URL>();
                    result.put(category, categoryList);
                }
                categoryList.add(u);
            }
        }
        if (result.size() == 0) {
            return;
        }
        Map<String, List<URL>> categoryNotified = notified.get(url);
        if (categoryNotified == null) {
            notified.putIfAbsent(url, new ConcurrentHashMap<String, List<URL>>());
            categoryNotified = notified.get(url);
        }
        for (Map.Entry<String, List<URL>> entry : result.entrySet()) {
            String category = entry.getKey();
            List<URL> categoryList = entry.getValue();
            categoryNotified.put(category, categoryList);
            saveProperties(url);
            listener.notify(categoryList);
        }
    }
}

saveProperties保存配置到本地文件

saveProperties

    private File file;
    private void saveProperties(URL url) {
        if (file == null) {
            return;
        }

        try {
            StringBuilder buf = new StringBuilder();
            Map<String, List<URL>> categoryNotified = notified.get(url);
            if (categoryNotified != null) {
                for (List<URL> us : categoryNotified.values()) {
                    for (URL u : us) {
                        if (buf.length() > 0) {
                            buf.append(URL_SEPARATOR);
                        }
                        buf.append(u.toFullString());
                    }
                }
            }
            properties.setProperty(url.getServiceKey(), buf.toString());
            long version = lastCacheChanged.incrementAndGet();
            if (syncSaveFile) {
                doSaveProperties(version);
            } else {
                registryCacheExecutor.execute(new SaveProperties(version));
            }
        } catch (Throwable t) {
            logger.warn(t.getMessage(), t);
        }
    }

OverrideListener的notify

notify动作中，如果当前发布的provider Url和容器中绑定的provider Url不一致，会调用RegistryProtocol.this.doChangeLocalExport(originInvoker, newUrl)方法进行一次重新发布

public class RegistryProtocol implements Protocol {
    private class OverrideListener implements NotifyListener {
        public synchronized void notify(List<URL> urls) {
            logger.debug("original override urls: " + urls);
            List<URL> matchedUrls = getMatchedUrls(urls, subscribeUrl);
            logger.debug("subscribe url: " + subscribeUrl + ", override urls: " + matchedUrls);
            // No matching results
            if (matchedUrls.isEmpty()) {
                return;
            }
            // 将得到的符合条件的emptyUrl转化为ConfiguratorUrl
            List<Configurator> configurators = RegistryDirectory.toConfigurators(matchedUrls);

            final Invoker<?> invoker;
            if (originInvoker instanceof InvokerDelegete) {
                invoker = ((InvokerDelegete<?>) originInvoker).getInvoker();
            } else {
                invoker = originInvoker;
            }
            //The origin invoker
            URL originUrl = RegistryProtocol.this.getProviderUrl(invoker);
            String key = getCacheKey(originInvoker);
            ExporterChangeableWrapper<?> exporter = bounds.get(key);
            if (exporter == null) {
                logger.warn(new IllegalStateException("error state, exporter should not be null"));
                return;
            }
            //The current, may have been merged many times
            URL currentUrl = exporter.getInvoker().getUrl();
            //Merged with this configuration
            URL newUrl = getConfigedInvokerUrl(configurators, originUrl);
            if (!currentUrl.equals(newUrl)) {
                RegistryProtocol.this.doChangeLocalExport(originInvoker, newUrl);
                logger.info("exported provider url changed, origin url: " + originUrl + ", old export url: " + currentUrl + ", new export url: " + newUrl);
            }
        }
    }
}

DestroyableExporter

入参值为：

    static private class DestroyableExporter<T> implements Exporter<T> {
        public static final ExecutorService executor = Executors.newSingleThreadExecutor(new NamedThreadFactory("Exporter-Unexport", true));

        private Exporter<T> exporter;
        private Invoker<T> originInvoker;
        private URL subscribeUrl;
        private URL registerUrl;

        public DestroyableExporter(Exporter<T> exporter, Invoker<T> originInvoker, URL subscribeUrl, URL registerUrl) {
            this.exporter = exporter;
            this.originInvoker = originInvoker;
            this.subscribeUrl = subscribeUrl;
            this.registerUrl = registerUrl;
        }

    }