• cinder侧挂载卷流程分析


    cinder侧挂载卷流程分析,存储类型以lvm+iscsi的方式为分析基础
    cinder侧主要调用了三个接口
    1)reserve_volume: 把volume的状态改为attaching,阻止其它节点执行挂载操作。
    2)initialize_connection: 这个方法负责构建和返回nova调用者需要的所有信息。返回的信息中包括CHAP credential, target-iqn 和lun 信息。
    3)attach_volume: 把volume状态改为in-use,挂载成功,并创建对应的attach记录。

    1、nova侧调用cinder的reserve_volume方法

    nova/volume/cinder.py
        @translate_volume_exception
        def reserve_volume(self, context, volume_id):
            cinderclient(context).volumes.reserve(volume_id)

    1)cinderclient端接受到nova发送的reserve操作的http请求,其入口处理函数为

    cinder/api/contrib/volume_actions.py:VolumeActionsController
        @wsgi.action('os-reserve')
        def _reserve(self, req, id, body):
            """Mark volume as reserved."""
            context = req.environ['cinder.context']
            # Not found exception will be handled at the wsgi level
            volume = self.volume_api.get(context, id)
    
            self.volume_api.reserve_volume(context, volume)
            return webob.Response(status_int=http_client.ACCEPTED)

    该函数的主要作用是通过volume 的uuid,获取volume实例信息,并调用volume目录下的api模块

    2)进一步调用cinder volume的api模块的reserve_volume函数,进行数据库的操作,更新卷的状态为attaching
    该函数的主要作用是检查指定的卷是否为available,如果卷的状态是available,更新cinder数据库,把卷的状态标记为attaching来预留这个卷,防止其他api在别的地方使用这个卷
    对于支持多路挂载的卷,有效状态包括in-use

    def reserve_volume(self, context, volume):
        expected = {'multiattach': volume.multiattach,
                    'status': (('available', 'in-use') if volume.multiattach
                               else 'available')}
    
        result = volume.conditional_update({'status': 'attaching'}, expected)
    
        if not result:
            expected_status = utils.build_or_str(expected['status'])
            msg = _('Volume status must be %(expected)s to reserve, but the '
                    'status is %(current)s.') % {'expected': expected_status,
                                                 'current': volume.status}
            LOG.error(msg)
            raise exception.InvalidVolume(reason=msg)
    
        LOG.info(_LI("Reserve volume completed successfully."),
                 resource=volume)

    2、nova侧向cinder发送initialize_connection请求,请求获取卷的所有连接信息

    nova/virt/block_device.py:DriverVolumeBlockDevice
        def attach(self, context, instance, volume_api, virt_driver,
                   do_check_attach=True, do_driver_attach=False, **kwargs):
            volume = volume_api.get(context, self.volume_id)
            if do_check_attach:
                volume_api.check_attach(context, volume, instance=instance)
    
            volume_id = volume['id']
            context = context.elevated()
    
            connector = virt_driver.get_volume_connector(instance)
            connection_info = volume_api.initialize_connection(context,
                                                               volume_id,
                                                               connector)
            if 'serial' not in connection_info:
                connection_info['serial'] = self.volume_id
            self._preserve_multipath_id(connection_info)
            ........
        

    1)cinderclient接受nova发送过来的os-initialize_connection请求

    @wsgi.action('os-initialize_connection')
    def _initialize_connection(self, req, id, body):
        """Initialize volume attachment."""
        context = req.environ['cinder.context']
        # Not found exception will be handled at the wsgi level
        volume = self.volume_api.get(context, id)
        try:
            connector = body['os-initialize_connection']['connector']
        except KeyError:
            raise webob.exc.HTTPBadRequest(
                explanation=_("Must specify 'connector'"))
        try:
            info = self.volume_api.initialize_connection(context,volume,connector)    
            ....

    2)进一步调用volume目录下的api模块的initialize_connection函数,对该请求进行处理

    cinder/volume/api.py:API类
        @wrap_check_policy
        def initialize_connection(self, context, volume, connector):
            if volume.status == 'maintenance':
                LOG.info(_LI('Unable to initialize the connection for '
                             'volume, because it is in '
                             'maintenance.'), resource=volume)
                msg = _("The volume connection cannot be initialized in "
                        "maintenance mode.")
                raise exception.InvalidVolume(reason=msg)
            init_results = self.volume_rpcapi.initialize_connection(context,
                                                                    volume,
                                                                    connector)
            LOG.info(_LI("Initialize volume connection completed successfully."),
                     resource=volume)
            return init_results

    3)cinder api进一步发送RPC请求给volume所在的cinder-volume服务节点,最终在cinder-volume节点,
    由cinder/volume/manager.py:VolumeManager的initialize_connection处理,该函数的处理,主要包括如下内容

       def initialize_connection(self, context, volume, connector):
             ....
               utils.require_driver_initialized(self.driver)
                step 1: self.driver.validate_connector(connector)
                step 2: model_update = self.driver.create_export(context.elevated(),volume, connector)
                step 3: volume.update(model_update)
                setp 4: conn_info = self.driver.initialize_connection(volume, connector)
            return conn_info

    step 1:
    对于LVM + iSCSI方式,validate_connector就是检查有没有initiator字段,即nova-compute节点的initiator信息
    代码跳转过程如下:drivers/lvm.py -> targets/lio.py -> targets/iscsi.py。

    cinder/volume/targets/iscsi.py:ISCSITarget
    def validate_connector(self, connector):
        # NOTE(jdg): api passes in connector which is initiator info
        if 'initiator' not in connector:
            err_msg = (_LE('The volume driver requires the iSCSI initiator '
                           'name in the connector.'))
            LOG.error(err_msg)
            raise exception.InvalidConnectorException(missing='initiator')
        return True

    step 2 :调用cinder-rtstool工具创建target,并把卷volume添加到target中创建出lun,认证信息。

    def create_export(self, context, volume, connector, vg=None):
        if vg is None:
            vg = self.configuration.volume_group
        volume_path = "/dev/%s/%s" % (vg, volume['name'])
        export_info = self.target_driver.create_export(
            context,
            volume,
            volume_path)
        return {'provider_location': export_info['location'],
                'provider_auth': export_info['auth'], }

    最终调用的是cinder/volume/targets/iscsi.py:ISCSITarget类

        def create_export(self, context, volume, volume_path):
            """Creates an export for a logical volume."""
            # 'iscsi_name': 'iqn.2010-10.org.openstack:volume-00000001'
            iscsi_name = "%s%s" % (self.configuration.iscsi_target_prefix,------设置iscsi name,形式为iqn.2010-10.org.openstack:volume-uuid
                                   volume['name'])
            iscsi_target, lun = self._get_target_and_lun(context, volume)---返回target,和lun的编号,值为(0,0)
    
            # Verify we haven't setup a CHAP creds file already
            # if DNE no big deal, we'll just create it
            chap_auth = self._get_target_chap_auth(context, volume)------从数据库volumes表中,读取该卷的provider_auth字段,获取认证信息,若没有,则创建
            if not chap_auth:
                chap_auth = (vutils.generate_username(),-----创建auth认证信息
                             vutils.generate_password())
    
            # Get portals ips and port
            portals_config = self._get_portals_config()-------获取portals配置,该函数返回的字典格式如下 {'portals_ips': portals_ips,'portals_port': self.configuration.iscsi_port}
    
            # NOTE(jdg): For TgtAdm case iscsi_name is the ONLY param we need
            # should clean this all up at some point in the future
            tid = self.create_iscsi_target(iscsi_name,-----------------创建target
                                           iscsi_target,
                                           lun,
                                           volume_path,
                                           chap_auth,
                                           **portals_config)
            data = {}
            data['location'] = self._iscsi_location(
                self.configuration.iscsi_ip_address, tid, iscsi_name, lun,
                self.configuration.iscsi_secondary_ip_addresses)
            LOG.debug('Set provider_location to: %s', data['location'])
            data['auth'] = self._iscsi_authentication(
                'CHAP', *chap_auth)
            return data

    创建target操作,调用的是cinder/volume/targets/lio.py中的create_iscsi_target方法
    这个函数下发的参数为
    name:iqn.2010-10.org.openstack:volume-uuid
    tid:0
    lun:0
    path:卷的路径
    chap_auth:{username,passowrd}
    kwargs:{'portals_ips': portals_ips,存储服务器的ip
    'portals_port': self.configuration.iscsi_port,一般是3260
    }

    def create_iscsi_target(self, name, tid, lun, path,chap_auth=None, **kwargs):
        # tid and lun are not used
        vol_id = name.split(':')[1]
        LOG.info(_LI('Creating iscsi_target for volume: %s'), vol_id)
        chap_auth_userid = ""
        chap_auth_password = ""
        if chap_auth is not None:
            (chap_auth_userid, chap_auth_password) = chap_auth
    
        optional_args = []
        if 'portals_port' in kwargs:
            optional_args.append('-p%s' % kwargs['portals_port'])
    
        if 'portals_ips' in kwargs:
            optional_args.append('-a' + ','.join(kwargs['portals_ips']))
    
        try:
            command_args = ['cinder-rtstool',
                            'create',
                            path,
                            name,
                            chap_auth_userid,
                            chap_auth_password,
                            self.iscsi_protocol == 'iser'] + optional_args
            self._execute(*command_args, run_as_root=True)
        except putils.ProcessExecutionError:
            LOG.exception(_LE("Failed to create iscsi target for volume "
                              "id:%s."), vol_id)
    
            raise exception.ISCSITargetCreateFailed(volume_id=vol_id)
    
        iqn = '%s%s' % (self.iscsi_target_prefix, vol_id)
        tid = self._get_target(iqn)
        if tid is None:
            LOG.error(_LE("Failed to create iscsi target for volume "
                          "id:%s."), vol_id)
            raise exception.NotFound()
    
        # We make changes persistent
        self._persist_configuration(vol_id)
        return tid
    
    def _iscsi_location(self, ip, target, iqn, lun=None, ip_secondary=None):
        ip_secondary = ip_secondary or []
        port = self.configuration.iscsi_port
        portals = map(lambda x: "%s:%s" % (x, port), [ip] + ip_secondary)
        return ("%(portals)s,%(target)s %(iqn)s %(lun)s"
                % ({'portals': ";".join(portals),
                    'target': target, 'iqn': iqn, 'lun': lun}))

    step 3:创建完target以后,更新cinder数据库volumes表中,该volume的provider_location,provider_auth两个字段的值

    step 4:调用cinder-rtstool的add-initiator子命令,把计算节点的initiator增加到刚刚创建的target acls中,并把所有的信息拼装返回给nova使用。

    cinder/volume/targets/lio.py:
       def initialize_connection(self, volume, connector):
            volume_iqn = volume['provider_location'].split(' ')[1]
            (auth_method, auth_user, auth_pass) = 
                volume['provider_auth'].split(' ', 3)
            # Add initiator iqns to target ACL
            try:
                self._execute('cinder-rtstool', 'add-initiator',
                              volume_iqn,
                              auth_user,
                              auth_pass,
                              connector['initiator'],
                              run_as_root=True)
            ......
            return super(LioAdm, self).initialize_connection(volume, connector)

    3、nova给cinderclient发送attach_volume命令,更改cinder数据库中,volume状态

    nova/virt/block_device.py:API
        @translate_volume_exception
        def attach(self, context, volume_id, instance_uuid, mountpoint, mode='rw'):
            cinderclient(context).volumes.attach(volume_id, instance_uuid,
                                                 mountpoint, mode=mode)

    1)cinder侧接受nova更新cinder数据库的入口函数

    cinder/api/contrib/volume_actions.py
        @wsgi.action('os-attach')
        def _attach(self, req, id, body):
                .....
                self.volume_api.attach(context, volume,instance_uuid, host_name, mountpoint, mode)
                ....

    2)最后cinder-api通过RPC请求到cinder-volume节点,更新数据库,把volume状态改为in-use,并创建对应的attach记录。

    cinder/volume/manager.py:VolumeManager
        def attach_volume(self, context, volume_id, instance_uuid, host_name,
                          mountpoint, mode, volume=None):
            """Updates db to show volume is attached.""
       ......
         attachment = volume.begin_attach(mode
        ......
  • 相关阅读:
    Treap仿set 模板
    线段树(区间更改,区间查最值)模板
    UVA 12345 Dynamic len(带修莫队)
    服务器配置环境以及部署项目流程
    使用SSH的scp命令行传输文件到远程服务器
    服务器部署javaweb项目
    在ubuntu服务器上安装mysql并配置外网访问
    在ubuntu服务器上配置tomcat
    在ubuntu服务器上配置jdk
    linux命令--解压与压缩
  • 原文地址:https://www.cnblogs.com/potato-chip/p/10794688.html
Copyright © 2020-2023  润新知