7.7 服务远程暴露

为了安全：服务启动的ip全部使用10.10.10.10

远程服务的暴露总体步骤：

• 将ref封装为invoker
• 将invoker转换为exporter
• 启动netty
• 注册服务到zookeeper
• 订阅与通知
• 返回新的exporter实例

在7.4 服务远程暴露 - 创建Exporter与启动netty服务端中，实现了前三步，在7.6 服务远程暴露 - 注册服务到zookeeper实现了第四步。本节实现第五步：订阅。总体代码如下：RegistryProtocol.export(final Invoker<T> originInvoker)

        // 订阅override数据
        // FIXME 提供者订阅时，会影响同一JVM即暴露服务，又引用同一服务的的场景，因为subscribed以服务名为缓存的key，导致订阅信息覆盖。
        final URL overrideSubscribeUrl = getSubscribedOverrideUrl(registedProviderUrl);
        final OverrideListener overrideSubscribeListener = new OverrideListener(overrideSubscribeUrl, originInvoker);
        overrideListeners.put(overrideSubscribeUrl, overrideSubscribeListener);
        registry.subscribe(overrideSubscribeUrl, overrideSubscribeListener);

说明：

• 第一句代码根据registedProviderUrl来获取overrideSubscribeUrl。
• 第二句代码创建overrideSubscribeListener
• 第三句代码将{ overrideSubscribeUrl : overrideSubscribeListener放入缓存 }
• 第四句代码实现真正的订阅与通知

一  获取overrideSubscribeUrl

final URL overrideSubscribeUrl = getSubscribedOverrideUrl(registedProviderUrl);

    /**
     * 1 将协议改为provider；
     * 2 添加参数：category=configurators和check=false；
     */
    private URL getSubscribedOverrideUrl(URL registedProviderUrl) {
        return registedProviderUrl.setProtocol(Constants.PROVIDER_PROTOCOL)
                                  .addParameters(Constants.CATEGORY_KEY, Constants.CONFIGURATORS_CATEGORY, Constants.CHECK_KEY, String.valueOf(false));
    }

开始时的registedProviderUrl如下：

• dubbo://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=5259&side=provider&timestamp=1507294508053

最终的overrideSubscribeUrl如下：

• provider://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=5259&side=provider&timestamp=1507294508053

二  创建overrideSubscribeListener

final OverrideListener overrideSubscribeListener = new OverrideListener(overrideSubscribeUrl, originInvoker);

overrideSubscribeListener是RegistryProtocol的内部类，来看一下声明和属性：

    private class OverrideListener implements NotifyListener {
        private final URL subscribeUrl;
        private final Invoker originInvoker;

        public OverrideListener(URL subscribeUrl, Invoker originalInvoker) {
            this.subscribeUrl = subscribeUrl;
            this.originInvoker = originalInvoker;
        }

这里创建出来的OverrideListener实例属性如下：

• subscribeUrl：provider://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=1818&side=provider&timestamp=1507366969962
• originInvoker：该实例还是在ServiceConfig.doExportUrlsFor1Protocol(ProtocolConfig protocolConfig, List<URL> registryURLs)创建出来的AbstractProxyInvoker实例（具体见7.4 服务远程暴露 - 创建Exporter与启动netty服务端）
  • proxy：DemoServiceImpl实例
  • type：Class<com.alibaba.dubbo.demo.DemoService>
  • url：registry://10.211.55.5:2181/com.alibaba.dubbo.registry.RegistryService?application=demo-provider&client=curator&dubbo=2.0.0&export=dubbo%3A%2F%2F10.10.10.10%3A20880%2Fcom.alibaba.dubbo.demo.DemoService%3Fanyhost%3Dtrue%26application%3Ddemo-provider%26dubbo%3D2.0.0%26generic%3Dfalse%26interface%3Dcom.alibaba.dubbo.demo.DemoService%26methods%3DsayHello%26pid%3D993%26side%3Dprovider%26timestamp%3D1507100322516&pid=993&registry=zookeeper&timestamp=1507100319830

最后，将创建出来的OverrideListener实例存储在RegistryProtocol的属性Map<URL, NotifyListener> overrideListeners中：

• key: （overrideSubscribeUrl，也就是subscribeUrl） provider://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=1818&side=provider&timestamp=1507366969962
• value:  上述的OverrideListener实例

三  真正的订阅

registry.subscribe(overrideSubscribeUrl, overrideSubscribeListener);

这里的registry是ZookeeperRegistry实例，subscribe(URL url, NotifyListener listener)方法在其父类FailbackRegistry中，如下：

    @Override
    public void subscribe(URL url, NotifyListener listener) {
        if (destroyed.get()){
            return;
        }
        super.subscribe(url, listener);
        removeFailedSubscribed(url, listener);
        try {
            // 向服务器端发送订阅请求
            doSubscribe(url, listener);
        } catch (Exception e) {
            Throwable t = e;

            List<URL> urls = getCacheUrls(url);
            if (urls != null && urls.size() > 0) {
                notify(url, listener, urls);
                logger.error("Failed to subscribe " + url + ", Using cached list: " + urls + " from cache file: " + getUrl().getParameter(Constants.FILE_KEY, System.getProperty("user.home") + "/dubbo-registry-" + url.getHost() + ".cache") + ", cause: " + t.getMessage(), t);
            } else {
                // 如果开启了启动时检测check=true，则直接抛出异常
                boolean check = getUrl().getParameter(Constants.CHECK_KEY, true)
                        && url.getParameter(Constants.CHECK_KEY, true);
                boolean skipFailback = t instanceof SkipFailbackWrapperException;
                if (check || skipFailback) {
                    if (skipFailback) {
                        t = t.getCause();
                    }
                    throw new IllegalStateException("Failed to subscribe " + url + ", cause: " + t.getMessage(), t);
                } else {
                    logger.error("Failed to subscribe " + url + ", waiting for retry, cause: " + t.getMessage(), t);
                }
            }
            // 将失败的订阅请求记录到失败列表，定时重试
            addFailedSubscribed(url, listener);
        }
    }

步骤：

• 首先调用其父类AbstractRegistry的方法，将之前创建出来的overrideSubscribeListener实例加入到overrideSubscribeUrl所对应的监听器集合中；
• 然后从failedSubscribed／failedUnsubscribed中overrideSubscribeUrl所对应的监听器集合中删除overrideSubscribeListener实例；从failedNotified获取当前url的通知失败map Map<NotifyListener, List<URL>>，之后从中删除掉该NotifyListener实例以及其需要通知的所有的url。
• 之后使用具体的子类（这里是ZookeeperRegistry）向服务器端发送订阅请求
• 如果在订阅的过程中抛出了异常，那么尝试获取缓存url，如果有缓存url，则进行失败通知，之后“将失败的订阅请求记录到失败列表，定时重试”，如果没有缓存url，如果开启了启动时检测或者直接抛出的异常是SkipFailbackWrapperException，则直接抛出异常，不会“将失败的订阅请求记录到失败列表，定时重试”

将之前创建出来的overrideSubscribeListener实例加入到overrideSubscribeUrl所对应的监听器集合中

   private final ConcurrentMap<URL, Set<NotifyListener>> subscribed = new ConcurrentHashMap<URL, Set<NotifyListener>>();//已经订阅的<URL, Set<NotifyListener>>
    
     /**
     * 首先从ConcurrentMap<URL, Set<NotifyListener>> subscribed中获取key为url的集合Set<NotifyListener>，
     * 如果该集合存在，直接将当前的NotifyListener实例存入该集合，
     * 如果集合不存在，先创建，之后放入subscribed中，并将当前的NotifyListener实例存入刚刚创建的集合
     *
     * @param url      订阅条件，不允许为空，如：consumer://10.10.10.10/com.alibaba.foo.BarService?version=1.0.0&application=kylin
     * @param listener 变更事件监听器，不允许为空
     */
    public void subscribe(URL url, NotifyListener listener) {
        if (url == null) {
            throw new IllegalArgumentException("subscribe url == null");
        }
        if (listener == null) {
            throw new IllegalArgumentException("subscribe listener == null");
        }
        if (logger.isInfoEnabled()) {
            logger.info("Subscribe: " + url);
        }
        Set<NotifyListener> listeners = subscribed.get(url);
        if (listeners == null) {
            subscribed.putIfAbsent(url, new ConcurrentHashSet<NotifyListener>());
            listeners = subscribed.get(url);
        }
        listeners.add(listener);
    }

从失败集合移除overrideSubscribeListener实例

    /**
     * 1 从ConcurrentMap<URL, Set<NotifyListener>> failedSubscribed 中获取当前url的订阅失败列表Set<NotifyListener>，之后从中删除掉该NotifyListener实例；
     * 2 从ConcurrentMap<URL, Set<NotifyListener>> failedUnsubscribed 中获取当前url的反订阅失败列表Set<NotifyListener>，之后从中删除掉该NotifyListener实例；
     * 3 从ConcurrentMap<URL, Map<NotifyListener, List<URL>>> failedNotified 中获取当前url的通知失败map Map<NotifyListener, List<URL>>，之后从中删除掉该NotifyListener实例以及其需要通知的所有的url。
     *
     * @param url
     * @param listener
     */
    private void removeFailedSubscribed(URL url, NotifyListener listener) {
        Set<NotifyListener> listeners = failedSubscribed.get(url);
        if (listeners != null) {
            listeners.remove(listener);
        }
        listeners = failedUnsubscribed.get(url);
        if (listeners != null) {
            listeners.remove(listener);
        }
        Map<NotifyListener, List<URL>> notified = failedNotified.get(url);
        if (notified != null) {
            notified.remove(listener);
        }
    }

ZookeeperRegistry.doSubscribe(final URL url, final NotifyListener listener)

protected void doSubscribe(final URL url, final NotifyListener listener) {
        try {
            if (Constants.ANY_VALUE.equals(url.getServiceInterface())) {//这条分支先不说
                String root = toRootPath();
                ConcurrentMap<NotifyListener, ChildListener> listeners = zkListeners.get(url);
                if (listeners == null) {
                    zkListeners.putIfAbsent(url, new ConcurrentHashMap<NotifyListener, ChildListener>());
                    listeners = zkListeners.get(url);
                }
                ChildListener zkListener = listeners.get(listener);
                if (zkListener == null) {
                    listeners.putIfAbsent(listener, new ChildListener() {
                        public void childChanged(String parentPath, List<String> currentChilds) {
                            for (String child : currentChilds) {
                                child = URL.decode(child);
                                if (!anyServices.contains(child)) {
                                    anyServices.add(child);
                                    subscribe(url.setPath(child).addParameters(Constants.INTERFACE_KEY, child,
                                            Constants.CHECK_KEY, String.valueOf(false)), listener);
                                }
                            }
                        }
                    });
                    zkListener = listeners.get(listener);
                }
                zkClient.create(root, false);
                List<String> services = zkClient.addChildListener(root, zkListener);
                if (services != null && services.size() > 0) {
                    for (String service : services) {
                        service = URL.decode(service);
                        anyServices.add(service);
                        subscribe(url.setPath(service).addParameters(Constants.INTERFACE_KEY, service,
                                Constants.CHECK_KEY, String.valueOf(false)), listener);
                    }
                }
            } else {
                /**
                 * ConcurrentMap<URL, ConcurrentMap<NotifyListener, ChildListener>> zkListeners
                 * 1 根据url获取ConcurrentMap<NotifyListener, ChildListener>，没有就创建
                 * 2 根据listener从ConcurrentMap<NotifyListener, ChildListener>获取ChildListener，没有就创建（创建的ChildListener用来监听子节点的变化）
                 * 3 创建path持久化节点
                 * 4 创建path子节点监听器
                 */
                List<URL> urls = new ArrayList<URL>();
                for (String path : toCategoriesPath(url)) {
                    ConcurrentMap<NotifyListener, ChildListener> listeners = zkListeners.get(url);
                    if (listeners == null) {
                        zkListeners.putIfAbsent(url, new ConcurrentHashMap<NotifyListener, ChildListener>());
                        listeners = zkListeners.get(url);
                    }
                    ChildListener zkListener = listeners.get(listener);
                    if (zkListener == null) {
                        listeners.putIfAbsent(listener, new ChildListener() {
                            //监听子节点列表的变化
                            public void childChanged(String parentPath, List<String> currentChilds) {
                                ZookeeperRegistry.this.notify(url, listener, toUrlsWithEmpty(url, parentPath, currentChilds));
                            }
                        });
                        zkListener = listeners.get(listener);
                    }
                    zkClient.create(path, false);//创建持久化节点/dubbo/com.alibaba.dubbo.demo.DemoService/configurators
                    List<String> children = zkClient.addChildListener(path, zkListener);
                    if (children != null) {
                        urls.addAll(toUrlsWithEmpty(url, path, children));
                    }
                }
                notify(url, listener, urls);
            }
        } catch (Throwable e) {
            throw new RpcException("Failed to subscribe " + url + " to zookeeper " + getUrl() + ", cause: " + e.getMessage(), e);
        }
    }

说明：

• url（overrideSubscribeUrl）：provider://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=9544&side=provider&timestamp=1507643800076
• listener：之前创建出来的overrideSubscribeListener实例

步骤：

• 首先获取categorypath：实际上就是获取/dubbo/{servicename}/{url中的category参数，默认是providers，这里是final URL overrideSubscribeUrl = getSubscribedOverrideUrl(registedProviderUrl);这句代码中添加到overrideSubscribeUrl上的category=configurators}

    private String[] toCategoriesPath(URL url) {
        String[] categroies;
        if (Constants.ANY_VALUE.equals(url.getParameter(Constants.CATEGORY_KEY))) {
            categroies = new String[]{Constants.PROVIDERS_CATEGORY, Constants.CONSUMERS_CATEGORY,
                    Constants.ROUTERS_CATEGORY, Constants.CONFIGURATORS_CATEGORY};
        } else {
            categroies = url.getParameter(Constants.CATEGORY_KEY, new String[]{Constants.DEFAULT_CATEGORY});
        }
        String[] paths = new String[categroies.length];
        for (int i = 0; i < categroies.length; i++) {
            paths[i] = toServicePath(url) + Constants.PATH_SEPARATOR + categroies[i];
        }
        return paths; // /dubbo/com.alibaba.dubbo.demo.DemoService/configurators
    }

• 然后就是获取并创建：ConcurrentMap<overrideSubscribeUrl, ConcurrentMap<overrideSubscribeListener实例, ChildListener>> zkListeners，这里创建出来的ChildListener实例中的childChanged(String parentPath, List<String> currentChilds)方法实际上就是最终当parentPath（实际上就是上边的categorypath）下的currentChilds发生变化时，执行的逻辑。
• 之后创建持久化节点：/dubbo/com.alibaba.dubbo.demo.DemoService/configurators
• 然后使用AbstractZookeeperClient<TargetChildListener>的addChildListener(String path, final ChildListener listener)方法为path下的子节点添加上边创建出来的内部类ChildListener实例
• 最后进行通知

AbstractZookeeperClient<TargetChildListener>.addChildListener(String path, final ChildListener listener)

    /**
     *  1 根据path从ConcurrentMap<String, ConcurrentMap<ChildListener, TargetChildListener>> childListeners获取ConcurrentMap<ChildListener, TargetChildListener>，没有就创建
     *  2 根据ChildListener获取TargetChildListener，没有就创建，TargetChildListener是真正的监听path的子节点变化的监听器
     *  createTargetChildListener(String path, final ChildListener listener)：创建一个真正的用来执行当path节点的子节点发生变化时的逻辑
     *  3 addTargetChildListener(path, targetListener)：将刚刚创建出来的子节点监听器订阅path的变化，这样之后，path的子节点发生了变化时，TargetChildListener才会执行相应的逻辑。
     *  而实际上TargetChildListener又会调用ChildListener的实现类的childChanged(String parentPath, List<String> currentChilds)方法，而该实现类，正好是ZookeeperRegistry中实现的匿名内部类，
     *  在该匿名内部类的childChanged(String parentPath, List<String> currentChilds)方法中，调用了ZookeeperRegistry.notify(URL url, NotifyListener listener, List<URL> urls)
     */
    public List<String> addChildListener(String path, final ChildListener listener) {
        ConcurrentMap<ChildListener, TargetChildListener> listeners = childListeners.get(path);
        if (listeners == null) {
            childListeners.putIfAbsent(path, new ConcurrentHashMap<ChildListener, TargetChildListener>());
            listeners = childListeners.get(path);
        }
        TargetChildListener targetListener = listeners.get(listener);
        if (targetListener == null) {
            listeners.putIfAbsent(listener, createTargetChildListener(path, listener));
            targetListener = listeners.get(listener);
        }
        return addTargetChildListener(path, targetListener);
    }

步骤：

• 首先是一顿获取和创建：ConcurrentMap<categorypath, ConcurrentMap<ZookeeperRegistry的内部类ChildListener实例, TargetChildListener>> childListeners，这里主要是创建TargetChildListener；
• 之后是真正的为path添加TargetChildListener实例。

CuratorZookeeperClient.createTargetChildListener(path, listener)

    public CuratorWatcher createTargetChildListener(String path, ChildListener listener) {
        return new CuratorWatcherImpl(listener);
    }

    private class CuratorWatcherImpl implements CuratorWatcher {

        private volatile ChildListener listener;

        public CuratorWatcherImpl(ChildListener listener) {
            this.listener = listener;
        }

        public void unwatch() {
            this.listener = null;
        }

        public void process(WatchedEvent event) throws Exception {
            if (listener != null) {
                listener.childChanged(event.getPath(), client.getChildren().usingWatcher(this).forPath(event.getPath()));
            }
        }
    }

很简单，就是创建一个监听path子节点的watcher，当path下有子节点变化时，调用listener（即传入的ZookeeperRegistry的内部类ChildListener实例的childChanged(String parentPath, List<String> currentChilds)方法）。

CuratorZookeeperClient.addTargetChildListener(String path, CuratorWatcher targetChildListener)

    public List<String> addTargetChildListener(String path, CuratorWatcher listener) {
        try {
            return client.getChildren().usingWatcher(listener).forPath(path);
        } catch (NoNodeException e) {
            return null;
        } catch (Exception e) {
            throw new IllegalStateException(e.getMessage(), e);
        }
    }

从上边的分析我们可以看出，当path节点下的子节点发生变化的时候，会首先调用TargetChildListener的process(WatchedEvent event)方法，在该方法中又会调用ChildListener实例的childChanged(String parentPath, List<String> currentChilds)方法，那么我们来分析一下该方法：

                            //监听子节点列表的变化
                            public void childChanged(String parentPath, List<String> currentChilds) {
                                ZookeeperRegistry.this.notify(url, listener, toUrlsWithEmpty(url, parentPath, currentChilds));
                            }

步骤：

• 首先获取子节点urls或者是一个consumer的empty协议的url

      /**
       * 过滤出providers中与consumer匹配的url集合
       */
      private List<URL> toUrlsWithoutEmpty(URL consumer, List<String> providers) {
          List<URL> urls = new ArrayList<URL>();
          if (providers != null && providers.size() > 0) {
              for (String provider : providers) {
                  provider = URL.decode(provider);
                  if (provider.contains("://")) {
                      URL url = URL.valueOf(provider);
                      if (UrlUtils.isMatch(consumer, url)) {
                          urls.add(url);
                      }
                  }
              }
          }
          return urls;
      }
  
      /**
       * 1 首先过滤出providers中与consumer匹配的providerUrl集合
       * 2 如果providerUrl集合不为空，直接返回这个集合
       * 3 如果为空，首先从path中获取category，然后将consumer的协议换成empty，添加参数category=configurators
       * @param consumer provider://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=9544&side=provider&timestamp=1507643800076
       * @param path /dubbo/com.alibaba.dubbo.demo.DemoService/configurators
       * @param providers 
       */
      private List<URL> toUrlsWithEmpty(URL consumer, String path, List<String> providers) {
          List<URL> urls = toUrlsWithoutEmpty(consumer, providers);
          if (urls == null || urls.isEmpty()) {
              int i = path.lastIndexOf('/');
              String category = i < 0 ? path : path.substring(i + 1);//configurators
              URL empty = consumer.setProtocol(Constants.EMPTY_PROTOCOL).addParameter(Constants.CATEGORY_KEY, category);
              urls.add(empty);
          }
          return urls; // empty://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=1237&side=provider&timestamp=1507352638483
      }

• 之后调用ZookeeperRegistry的父类FailbackRegistry.notify(URL url, NotifyListener listener, List<URL> urls)

      @Override
      protected void notify(URL url, NotifyListener listener, List<URL> urls) {
          if (url == null) {
              throw new IllegalArgumentException("notify url == null");
          }
          if (listener == null) {
              throw new IllegalArgumentException("notify listener == null");
          }
          try {
              doNotify(url, listener, urls);
          } catch (Exception t) {
              // 将失败的通知请求记录到失败列表，定时重试
              Map<NotifyListener, List<URL>> listeners = failedNotified.get(url);
              if (listeners == null) {
                  failedNotified.putIfAbsent(url, new ConcurrentHashMap<NotifyListener, List<URL>>());
                  listeners = failedNotified.get(url);
              }
              listeners.put(listener, urls);
              logger.error("Failed to notify for subscribe " + url + ", waiting for retry, cause: " + t.getMessage(), t);
          }
      }
  
      protected void doNotify(URL url, NotifyListener listener, List<URL> urls) {
          super.notify(url, listener, urls);
      }

  说明：这里传入的

  • url（overrideSubscribeUrl）：provider://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=9544&side=provider&timestamp=1507643800076

  • listener：之前创建出来的overrideSubscribeListener实例
  • urls：[ empty://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=9544&side=provider&timestamp=1507643800076 ]
  • 这里首先执行父类的AbstractRegistry.notify(URL url, NotifyListener listener, List<URL> urls)，如果失败，则获取或创建ConcurrentMap<overrideSubscribeUrl, Map<overrideSubscribeListener实例, urls>> failedNotified，后续做重试

来看一下通知的最核心部分：

AbstractRegistry.notify(URL url, NotifyListener listener, List<URL> urls)

    /**
     * 1 首先遍历List<URL> urls，将urls按照category进行分类，存储在Map<"categoryName", List<URL>> result中；
     * 2 之后遍历result：（每遍历一次，都是一个新的category）
     * （1）将Map<"categoryName", List<URL>>存储在ConcurrentMap<URL, Map<String, List<URL>>> notified的Map<String, List<URL>>中
     * （2）进行properties设置和文件保存
     * （3）调用传入放入listener的notify()方法。
     * @param url
     * @param listener
     * @param urls
     */
    protected void notify(URL url, NotifyListener listener, List<URL> urls) {
        if (url == null) {
            throw new IllegalArgumentException("notify url == null");
        }
        if (listener == null) {
            throw new IllegalArgumentException("notify listener == null");
        }
        if ((urls == null || urls.size() == 0)
                && !Constants.ANY_VALUE.equals(url.getServiceInterface())) {
            logger.warn("Ignore empty notify urls for subscribe url " + url);
            return;
        }
        if (logger.isInfoEnabled()) {
            logger.info("Notify urls for subscribe url " + url + ", urls: " + urls);
        }
        /**
         * 遍历List<URL> urls，将urls按照category进行分类
         */
        Map<String, List<URL>> result = new HashMap<String, List<URL>>(); //{ "categoryName" : List<URL> }
        for (URL u : urls) {
            if (UrlUtils.isMatch(url, u)) {
                String category = u.getParameter(Constants.CATEGORY_KEY, Constants.DEFAULT_CATEGORY);
                List<URL> categoryList = result.get(category);
                if (categoryList == null) {
                    categoryList = new ArrayList<URL>();
                    result.put(category, categoryList);
                }
                categoryList.add(u);
            }
        }
        if (result.size() == 0) {
            return;
        }
        Map<String, List<URL>> categoryNotified = notified.get(url);
        if (categoryNotified == null) {
            notified.putIfAbsent(url, new ConcurrentHashMap<String, List<URL>>());
            categoryNotified = notified.get(url);
        }
        for (Map.Entry<String, List<URL>> entry : result.entrySet()) {
            String category = entry.getKey();
            List<URL> categoryList = entry.getValue();
            categoryNotified.put(category, categoryList);//填充notified集合
            saveProperties(url);//该行代码为什么不写在循环体外边
            listener.notify(categoryList);
        }
    }

说明：这里传入的

• url（overrideSubscribeUrl）：provider://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=9544&side=provider&timestamp=1507643800076

• listener：之前创建出来的overrideSubscribeListener实例
• urls：[ empty://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=9544&side=provider&timestamp=1507643800076 ]

步骤：

• 首先遍历List<URL> urls，将urls按照category进行分类，存储在Map<"categoryName", List<URL>> result中；
• 然后获取或创建ConcurrentMap<overrideSubscribeUrl, Map<"categoryName", subList(urls)>> notified
• 最后遍历Map<"categoryName", List<URL>> result
  • 去填充notified集合
  • 保存传入的url到Properties properties（本地磁盘缓存中）
  • 调用传入的listener的notify方法（注意：这里调用的正是文章开头创建的overrideSubscribeListener实例的notify方法）

AbstractRegistry.saveProperties(URL url)

    /**
     * 1 按照url从将ConcurrentMap<URL, Map<String, List<URL>>> notified中将Map<String, List<URL>>拿出来，之后将所有category的list组成一串buf（以空格分隔）
     * 2 将< serviceKey<->buf >写入本地磁盘缓存中：Properties properties
     * 3 将AtomicLong lastCacheChanged加1
     * 4 之后根据syncSaveFile判断时同步保存properties到文件，还是异步保存properties到文件
     * @param url
     */
    private void saveProperties(URL url) {
        if (file == null) {
            return;
        }

        try {
            StringBuilder buf = new StringBuilder();
            Map<String, List<URL>> categoryNotified = notified.get(url);
            if (categoryNotified != null) {
                for (List<URL> us : categoryNotified.values()) {
                    for (URL u : us) {
                        if (buf.length() > 0) {
                            buf.append(URL_SEPARATOR);
                        }
                        buf.append(u.toFullString());
                    }
                }
            }
            properties.setProperty(url.getServiceKey(), buf.toString());
            long version = lastCacheChanged.incrementAndGet();
            if (syncSaveFile) {
                doSaveProperties(version);
            } else {
                registryCacheExecutor.execute(new SaveProperties(version));
            }
        } catch (Throwable t) {
            logger.warn(t.getMessage(), t);
        }
    }

说明：

• 入参：url：provider://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=5033&side=provider&timestamp=1507720343596
• properties：{ "com.alibaba.dubbo.demo.DemoService" -> "empty://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=5033&side=provider&timestamp=1507720343596" }
• 最后采用异步线程将properties中的内容写入到文件中

AbstractRegistry$SaveProperties

    private class SaveProperties implements Runnable {
        private long version;

        private SaveProperties(long version) {
            this.version = version;
        }

        public void run() {
            doSaveProperties(version);
        }
    }

AbstractRegistry.doSaveProperties(long version)

    /**
     * 1 先将文件中的内容读取到一个新的Properties newProperties中；
     * 2 之后将properties中的信息写入这个newProperties中；
     * 3 之后创建dubbo-registry-10.211.55.5.cache.lock文件；
     * 4 最后将这个newProperties中的内容写入到文件中
     */
    public void doSaveProperties(long version) {
        if (version < lastCacheChanged.get()) {
            return;
        }
        if (file == null) {
            return;
        }
        Properties newProperties = new Properties();
        // 保存之前先读取一遍，防止多个注册中心之间冲突
        InputStream in = null;
        try {
            if (file.exists()) {
                in = new FileInputStream(file);
                newProperties.load(in);
            }
        } catch (Throwable e) {
            logger.warn("Failed to load registry store file, cause: " + e.getMessage(), e);
        } finally {
            if (in != null) {
                try {
                    in.close();
                } catch (IOException e) {
                    logger.warn(e.getMessage(), e);
                }
            }
        }
        // 保存
        try {
            newProperties.putAll(properties);
            File lockfile = new File(file.getAbsolutePath() + ".lock");
            if (!lockfile.exists()) {
                lockfile.createNewFile();//创建lock文件
            }
            RandomAccessFile raf = new RandomAccessFile(lockfile, "rw");
            try {
                FileChannel channel = raf.getChannel();
                try {
                    FileLock lock = channel.tryLock();
                    if (lock == null) {
                        throw new IOException("Can not lock the registry cache file " + file.getAbsolutePath() + ", ignore and retry later, maybe multi java process use the file, please config: dubbo.registry.file=xxx.properties");
                    }
                    // 保存
                    try {
                        if (!file.exists()) {
                            file.createNewFile();
                        }
                        FileOutputStream outputFile = new FileOutputStream(file);
                        try {
                            newProperties.store(outputFile, "Dubbo Registry Cache");
                        } finally {
                            outputFile.close();
                        }
                    } finally {
                        lock.release();
                    }
                } finally {
                    channel.close();
                }
            } finally {
                raf.close();
            }
        } catch (Throwable e) {
            if (version < lastCacheChanged.get()) {
                return;
            } else {
                registryCacheExecutor.execute(new SaveProperties(lastCacheChanged.incrementAndGet()));
            }
            logger.warn("Failed to save registry store file, cause: " + e.getMessage(), e);
        }
    }

步骤见注释。这里有一个version，实际上是一个CAS判断，我们在saveProperties(URL url)方法中执行了long version = lastCacheChanged.incrementAndGet();之后在doSaveProperties(long version)进行if (version < lastCacheChanged.get())判断，如果满足这个条件，说明当前线程在进行doSaveProperties(long version)时，已经有其他线程执行了saveProperties(URL url)，马上就要执行doSaveProperties(long version)，所以当前线程放弃操作，让后边的这个线程来做保存操作。

保存操作执行之后，会在文件夹/Users/jigangzhao/.dubbo下生成两个文件：

• dubbo-registry-10.211.55.5.cache
• dubbo-registry-10.211.55.5.cache.lock

前者的内容：

#Wed Oct 11 19:42:29 CST 2017
com.alibaba.dubbo.demo.DemoService=empty://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&category=configurators&check=false&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=5165&side=provider&timestamp=1507722024953

最后就是OverrideListener.notify(List<URL> urls)

    /**
     * 重新export
     * 1.protocol中的exporter destroy问题
     * 1.要求registryprotocol返回的exporter可以正常destroy
     * 2.notify后不需要重新向注册中心注册
     * 3.export 方法传入的invoker最好能一直作为exporter的invoker.
     */
    private class OverrideListener implements NotifyListener {
        private final URL subscribeUrl;
        private final Invoker originInvoker;

        public OverrideListener(URL subscribeUrl, Invoker originalInvoker) {
            this.subscribeUrl = subscribeUrl;
            this.originInvoker = originalInvoker;
        }

        /**
         * 目的：
         * 对原本注册了的providerUrl进行校验，如果url发生了变化，那么要重新export
         *
         * @param urls 已注册信息列表，总不为空，含义同{@link com.alibaba.dubbo.registry.RegistryService#lookup(URL)}的返回值。
         */
        public synchronized void notify(List<URL> urls) {
            logger.debug("original override urls: " + urls);
            List<URL> matchedUrls = getMatchedUrls(urls, subscribeUrl);
            logger.debug("subscribe url: " + subscribeUrl + ", override urls: " + matchedUrls);
            //没有匹配的
            if (matchedUrls.isEmpty()) {
                return;
            }

            List<Configurator> configurators = RegistryDirectory.toConfigurators(matchedUrls);//这里是一个空列表

            final Invoker<?> invoker;
            if (originInvoker instanceof InvokerDelegete) {
                invoker = ((InvokerDelegete<?>) originInvoker).getInvoker();
            } else {
                invoker = originInvoker;
            }
            //最原始的invoker
            URL originUrl = RegistryProtocol.this.getProviderUrl(invoker);//dubbo://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=5279&side=provider&timestamp=1507723571451
            String key = getCacheKey(originInvoker);//dubbo://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=5279&side=provider&timestamp=1507723571451
            ExporterChangeableWrapper<?> exporter = bounds.get(key);//在doLocalExport方法中已经存放在这里了
            if (exporter == null) {
                logger.warn(new IllegalStateException("error state, exporter should not be null"));
                return;
            }
            //当前的，可能经过了多次merge
            URL currentUrl = exporter.getInvoker().getUrl();//dubbo://10.10.10.10:20880/com.alibaba.dubbo.demo.DemoService?anyhost=true&application=demo-provider&dubbo=2.0.0&generic=false&interface=com.alibaba.dubbo.demo.DemoService&methods=sayHello&pid=5279&side=provider&timestamp=1507723571451
            //与本次配置merge的
            URL newUrl = getConfigedInvokerUrl(configurators, originUrl);
            if (!currentUrl.equals(newUrl)) {
                RegistryProtocol.this.doChangeLocalExport(originInvoker, newUrl);//重新将invoker暴露为exporter
                logger.info("exported provider url changed, origin url: " + originUrl + ", old export url: " + currentUrl + ", new export url: " + newUrl);
            }
        }

        private List<URL> getMatchedUrls(List<URL> configuratorUrls, URL currentSubscribe) {
            List<URL> result = new ArrayList<URL>();
            for (URL url : configuratorUrls) {
                URL overrideUrl = url;
                // 兼容旧版本
                if (url.getParameter(Constants.CATEGORY_KEY) == null
                        && Constants.OVERRIDE_PROTOCOL.equals(url.getProtocol())) {
                    overrideUrl = url.addParameter(Constants.CATEGORY_KEY, Constants.CONFIGURATORS_CATEGORY);
                }

                //检查是不是要应用到当前服务上
                if (UrlUtils.isMatch(currentSubscribe, overrideUrl)) {
                    result.add(url);
                }
            }
            return result;
        }

        //合并配置的url
        private URL getConfigedInvokerUrl(List<Configurator> configurators, URL url) {
            for (Configurator configurator : configurators) {
                url = configurator.configure(url);
            }
            return url;
        }
    }

最后：总结一下：

当前的provider订阅了/dubbo/com.alibaba.dubbo.demo.DemoService/configurators，当其下的子节点发生变化时，如果其下的子节点的url或者当前的providerUrl发生了变化，需要重新暴露。

重新暴露：

    /**
     * 对修改了url的invoker重新export
     *
     * @param originInvoker
     * @param newInvokerUrl
     */
    @SuppressWarnings("unchecked")
    private <T> void doChangeLocalExport(final Invoker<T> originInvoker, URL newInvokerUrl) {
        String key = getCacheKey(originInvoker);
        final ExporterChangeableWrapper<T> exporter = (ExporterChangeableWrapper<T>) bounds.get(key);
        if (exporter == null) {
            logger.warn(new IllegalStateException("error state, exporter should not be null"));
        } else {
            final Invoker<T> invokerDelegete = new InvokerDelegete<T>(originInvoker, newInvokerUrl);
            exporter.setExporter(protocol.export(invokerDelegete));
        }
    }