加载heat resource的代码分析

Heat的核心是stack，stack又是由各种各样的资源组成的，heat除了自定义的大量资源外，还允许用户自定义自己需要的资源。

heat资源加载流程

我们先从heat-engine的启动脚本来看:

heat/cmd/engine.py
if __name__ == '__main__':  
  
    cfg.CONF(project='heat', prog='heat-engine')  
    logging.setup('heat')  
    messaging.setup()  
  
    mgr = None  
    try:  
	#加载heat的模板类  
        mgr = template._get_template_extension_manager()  
    except template.TemplatePluginNotRegistered as ex:  
        LOG.critical(_LC("%s"), ex)  
    if not mgr or not mgr.names():  
        sys.exit("ERROR: No template format plugins registered")  
  
    from heat.engine import service as engine  
  
    srv = engine.EngineService(cfg.CONF.host, rpc_api.ENGINE_TOPIC)  

上面为heat-egine进程启动的代码，在创建EngineService的时候，这里面有个resources初始化的步骤,

这里应该就是完成资源加载的关键，如下面的代码所示

heat/engine/service.py EngineService.__init__():

def __init__(self, host, topic, manager=None):  
        super(EngineService, self).__init__()  
        resources.initialise() #对资源进行初始化，分析代码可以除了初始化资源，此处还会初始化clients. 
        self.host = host  
        self.topic = topic  

首先是有个全局变量_environment，由于是初次加载，所以会执行clients.initialise()

heat/engine/resources/__init__.py
def initialise():  
    global _environment  
    if _environment is not None:  
        return  
  
    clients.initialise()  
  
    global_env = environment.Environment({}, user_env=False)  
    _load_global_environment(global_env)  
    _environment = global_env  

clients其实就是heat与各个openstack组件通信的关键，类似的，这里采用了全局_mgr来管理clients的加载，

初次加载的时候由于全局变量_mgr为None，所以采用stevedore加载heat需要用到的各种clients，

_mgr = None  
  
  
def has_client(name):  
    return _mgr and name in _mgr.names()  
  
heat/engine/clients/__init__.py 
def initialise():  
    global _mgr  
    if _mgr:  
        return  
  
    _mgr = extension.ExtensionManager(  
        namespace='heat.clients',  
        invoke_on_load=False,  
        verify_requirements=False) 

这里的clients为entrypoints中定义的名称空间为heat.clients中的各种client,具体可以查看heat的api-paste.ini文件。

加载完这些clients后，会定义一个Environment对象：

heat/engine/environment.py
def __init__(self, env=None, user_env=True):  
        """Create an Environment from a dict of varying format. 
        1) old-school flat parameters 
        2) or newer {resource_registry: bla, parameters: foo} 
 
        :param env: the json environment 
        :param user_env: boolean, if false then we manage python resources too. 
        """  
        if env is None:  
            env = {}  
        if user_env:  
            from heat.engine import resources  
            global_registry = resources.global_env().registry  
        else:  
            global_registry = None  
  
        self.registry = ResourceRegistry(global_registry)  
        self.registry.load(env.get(env_fmt.RESOURCE_REGISTRY, {}))  
  （部分代码有删减）

后面我们加载资源的关键了

self.registry = ResourceRegistry(global_registry)  def _load_global_environment(env):  
    _load_global_resources(env)  
    environment.read_global_environment(env)  
  
  
def _load_global_resources(env):  
    _register_constraints(env, _get_mapping('heat.constraints'))  
    _register_stack_lifecycle_plugins(  
        env,  
        _get_mapping('heat.stack_lifecycle_plugins'))  
  
    manager = plugin_manager.PluginManager(__name__)  
    # Sometimes resources should not be available for registration in Heat due  
    # to unsatisfied dependencies. We look first for the function  
    # 'available_resource_mapping', which should return the filtered resources.  
    # If it is not found, we look for the legacy 'resource_mapping'.  
    resource_mapping = plugin_manager.PluginMapping(['available_resource',  
                                                     'resource'])  
    constraint_mapping = plugin_manager.PluginMapping('constraint')  
  
    _register_resources(env, resource_mapping.load_all(manager))  
  
    _register_constraints(env, constraint_mapping.load_all(manager))

在_load_global_resources方法里面，首先是往env里面注册constraints和stack_lifecycle_plugins，

注册的原理很简单，也是用stevedore进行插件的加载，需要加载的项在api-paste.init中的entry_points里面可以看到。

然后通过下面的方法，将这些插件注册到前面定义的global_env对象中，接下面就是resources的加载的，

这里定义了2个插件相关的类，PluginManager以及PluginMapping，首先看下PluginManager

heat/engine/plugin_manager.py
class PluginManager(object):  
    '''''A class for managing plugin modules.'''  
  
    def __init__(self, *extra_packages):  
        '''''Initialise the Heat Engine plugin package, and any others. 
 
        The heat.engine.plugins package is always created, if it does not 
        exist, from the plugin directories specified in the config file, and 
        searched for modules. In addition, any extra packages specified are 
        also searched for modules. e.g. 
 
        >>> PluginManager('heat.engine.resources') 
 
        will load all modules in the heat.engine.resources package as well as 
        any user-supplied plugin modules. 
 
        '''  
        def packages():  
            for package_name in extra_packages:  
                yield sys.modules[package_name]  
  
            cfg.CONF.import_opt('plugin_dirs', 'heat.common.config')  
            yield plugin_loader.create_subpackage(cfg.CONF.plugin_dirs,  
                                                  'heat.engine')  
  
        def modules():  
            pkg_modules = itertools.imap(plugin_loader.load_modules,  
                                         packages())  
            return itertools.chain.from_iterable(pkg_modules)  
  
        self.modules = list(modules())  

其属性modules主要是列出了资源所在的各个模块，这里默认加载的'heat.engine.resources'包下面的各个模块，

此外，heat还允许我们自己配置加载资源插件的路径，但是默认来说，heat自身只会加载'heat.engine.resources'下面的模块，

所以我们扩展资源插件的时候，可以选择放在这个目录heat/engine/resources/。

接下来定义了2个PluginMapping，分别针对resources和constraints，注意到这里传入的参数，resources传入的是

available_resource和resource，后面我们会看到为什么要传这个.

resource_mapping= plugin_manager.PluginMapping(['available_resource',  
                                                    'resource']) 
constraint_mapping = plugin_manager.PluginMapping('constraint')

接着会执行下面这段代码

_register_resources(env, resource_mapping.load_all(manager))  
def _register_resources(env, type_pairs):  
    for res_name, res_class in type_pairs:  
        env.register_class(res_name, res_class) 

那type_pairs里面到底是啥呢，我们看下相关的方法，首先是resource_mapping.load_all(manager)的返回，

可以看到这里返回了个迭代器,迭代器中的内容又是一个迭代器，每个迭代器的内容又来源于itertools.imap(function,self.modules)，

也就是将load_from_module方法作用于之前加载起来的在resources下面的各个资源模块。

 def load_all(self, plugin_manager):  
        '''''Iterate over the mappings from all modules in the plugin manager. 
 
        Mappings are returned as a list of (key, value) tuples. 
        '''  
        mod_dicts = plugin_manager.map_to_modules(self.load_from_module)  
        return itertools.chain.from_iterable(six.iteritems(d) for d  
def map_to_modules(self, function):  
        '''''Iterate over the results of calling a function on every module.'''  
        return itertools.imap(function, self.modules) 

我们来看下PluginMapping中的load_from_module这个方法，可以看到，之前传的available_resource和resource参数就起作用了，

该方法会从这个模块尝试去取available_resource_mapping和resource_mapping，如果available_resource_mapping或resource_mapping是函数，就会取resouce_mapping方法的内容

heat/engine/plugin_manager.py

def load_from_module(self, module):  
        '''''Return the mapping specified in the given module. 
 
        If no such mapping is specified, an empty dictionary is returned. 
        '''  
        for mapping_name in self.names:  
            mapping_func = getattr(module, mapping_name, None)  
            if callable(mapping_func):  
                fmt_data = {'mapping_name': mapping_name, 'module': module}  
                try:  
                    mapping_dict = mapping_func(*self.args, **self.kwargs)  
     (部分代码有删减）

我们以heat自带的autoscaling模块为例，它的返回如下

def resource_mapping():  
    return {  
        'AWS::AutoScaling::AutoScalingGroup': AutoScalingGroup,  
        'OS::Heat::InstanceGroup': InstanceGroup,  
        'OS::Heat::AutoScalingGroup': AutoScalingResourceGroup,  
    }  

所以接着上面的代码看，这里的res_name和res_class就是resource_mapping或者available_mapping返回的字

典的内容，然后往之前的env对象中注册该资源.

def _register_resources(env, type_pairs):  
    for res_name, res_class in type_pairs:  
        env.register_class(res_name, res_class)  
def register_class(self, resource_type, resource_class):  
        ri = ResourceInfo(self, [resource_type], resource_class)  
        self._register_info([resource_type], ri)

这里首先根据我们传入的资源创建一个合适的资源类，然后注册到env中，直至把module中的资源加载完成。

加载constraints的过程也和resource类似，这里不继续展开。

当env加载完各种插件后，回到最之前的代码会把这个global_env对象赋值给全局变量_environment，

def initialise():  
    global _environment  
    if _environment is not None:  
        return  
  
    clients.initialise()  
  
    global_env = environment.Environment({}, user_env=False)  
    _load_global_environment(global_env)  
    _environment = global_env 

添加自定义的资源到heat中

现在我们在resource目录下自定义一个自己的插件，按照上面的分析，可以这么写，让heat能够正确加载我们的插件

from heat.engine import resource  
  
  
class MyResource(resource.Resource):  	
        def handle_create(self):
		pass
	def handle_update(self):
		pass
	def handle_delete(self):
		pass
      
  
def available_resource_mapping():  
    return {'OS::Openstack::MyRSC': MyResource}  

#我们在加载资源插件之后加一句打印代码
def _load_global_resources(env):  
    _register_constraints(env, _get_mapping('heat.constraints'))  
    _register_stack_lifecycle_plugins(  
        env,  
        _get_mapping('heat.stack_lifecycle_plugins'))  
  
    manager = plugin_manager.PluginManager(__name__)  
    # Sometimes resources should not be available for registration in Heat due  
    # to unsatisfied dependencies. We look first for the function  
    # 'available_resource_mapping', which should return the filtered resources.  
    # If it is not found, we look for the legacy 'resource_mapping'.  
    resource_mapping = plugin_manager.PluginMapping(['available_resource',  
                                                     'resource'])  
    constraint_mapping = plugin_manager.PluginMapping('constraint')  
  
    _register_resources(env, resource_mapping.load_all(manager))  
  
    _register_constraints(env, constraint_mapping.load_all(manager))  
    print 'OS::Openstack::MyRSC' in env.registry._registry  

可以发现打印为True，代表我们已经成功加载我们自定义的插件了.