cinder侧卸载卷流程分析

cinder侧卸载卷分析，存储类型以lvm+iscsi的方式为分析基础
在虚机卸载卷的过程中，主要涉及如下三个函数
1)cinder.volume.api.begin_detaching 把volume的状态改为detaching，阻止其它节点执行挂载操作
2)cinder.volume.api.terminate_connection 进行target,lun等信息的清理
3)cinder.volume.api.detach 更新cinder数据库，设置卷的状态为available

1、nova侧调用cinderclient的begin_detaching方法，

nova/compute/api.py:API
    def _check_and_begin_detach(self, context, volume, instance):
        self.volume_api.check_detach(context, volume, instance=instance)
        self.volume_api.begin_detaching(context, volume['id'])

1)cinderclient端接受到nova发送的begin_detaching操作的http请求，其入口处理函数为

cinder/api/contrib/volume_actions.py:VolumeActionsController
    def _begin_detaching(self, req, id, body):
        """Update volume status to 'detaching'."""
        context = req.environ['cinder.context']
        # Not found exception will be handled at the wsgi level
        volume = self.volume_api.get(context, id)

        self.volume_api.begin_detaching(context, volume)
        return webob.Response(status_int=http_client.ACCEPTED)    
该函数的主要作用是通过volume 的uuid，获取volume实例信息，并调用volume目录下的api模块

2)进一步调用cinder volume的api模块的begin_detaching函数，进行数据库的操作，更新卷的状态为detaching,防止其他api对这个卷操作

    @wrap_check_policy
    def begin_detaching(self, context, volume):
        # If we are in the middle of a volume migration, we don't want the
        # user to see that the volume is 'detaching'. Having
        # 'migration_status' set will have the same effect internally.
        expected = {'status': 'in-use',
                    'attach_status': fields.VolumeAttachStatus.ATTACHED,
                    'migration_status': self.AVAILABLE_MIGRATION_STATUS}

        result = volume.conditional_update({'status': 'detaching'}, expected)

        if not (result or self._is_volume_migrating(volume)):
            msg = _("Unable to detach volume. Volume status must be 'in-use' "
                    "and attach_status must be 'attached' to detach.")
            LOG.error(msg)
            raise exception.InvalidVolume(reason=msg)

        LOG.info(_LI("Begin detaching volume completed successfully."),
                 resource=volume)

2、nova侧向cinder发送terminate_connection请求，请求删除卷的连接信息

def _detach_volume(self, context, volume_id, instance, destroy_bdm=True,attachment_id=None):
     ......
     self.volume_api.terminate_connection(context, volume_id, connector)
     ......
     self.volume_api.detach(context.elevated(), volume_id, instance.uuid,attachment_id)
     

1)cinderclient接受nova发送过来的os-terminate_connection请求

   @wsgi.action('os-terminate_connection')
    def _terminate_connection(self, req, id, body):
        """Terminate volume attachment."""
        context = req.environ['cinder.context']
        # Not found exception will be handled at the wsgi level
        volume = self.volume_api.get(context, id)
        try:
            connector = body['os-terminate_connection']['connector']
        except KeyError:
            raise webob.exc.HTTPBadRequest(
                explanation=_("Must specify 'connector'"))
        try:
            self.volume_api.terminate_connection(context, volume, connector)
        except exception.VolumeBackendAPIException:
            msg = _("Unable to terminate volume connection from backend.")
            raise webob.exc.HTTPInternalServerError(explanation=msg)
        return webob.Response(status_int=http_client.ACCEPTED)
        

2)进一步调用volume目录下api模块的 terminate_connection 函数,对该请求进行处理

cinder/volume/api.py:API类        
    @wrap_check_policy
    def terminate_connection(self, context, volume, connector, force=False):
       step 1: self.volume_rpcapi.terminate_connection(context,volume,connector,force)
       step 2 :self.unreserve_volume(context, volume)

step 1:cinder api进一步发送RPC请求给volume所在的cinder-volume服务节点，最终在cinder-volume节点，
由cinder/volume/manager.py:VolumeManager的terminate_connection处理，该函数的处理，主要包括如下内容

   def terminate_connection(self, context, volume_id, connector, force=False):
        utils.require_driver_initialized(self.driver)----获取对应的驱动信息
        volume_ref = self.db.volume_get(context, volume_id)-----从数据库中获取卷的信息
        try:
            step 1.1 self.driver.terminate_connection(volume_ref, connector, force=force)-----调用对应的驱动的terminate_connection函数

step 1.1 :
使用lvm+lio的方式，代码跳转过程如下：drivers/lvm.py -> targets/lio.py，从target 的acl中删除initiator，从而有效的在 target侧关闭iscis session连接

    def terminate_connection(self, volume, connector, **kwargs):
        volume_iqn = volume['provider_location'].split(' ')[1]
        # Delete initiator iqns from target ACL
        try:
            self._execute('cinder-rtstool', 'delete-initiator',
                          volume_iqn,
                          connector['initiator'],
                          run_as_root=True)
        except putils.ProcessExecutionError:
            LOG.exception(
                _LE("Failed to delete initiator iqn %s from target."),
                connector['initiator'])
            raise exception.ISCSITargetDetachFailed(volume_id=volume['id'])

        # We make changes persistent
        self._persist_configuration(volume['id'])

3、nova给cinderclient发送os-detach命令，更改cinder数据库

1)cinder侧接受nova更新cinder数据库的入口函数

cinder/api/contrib/volume_actions.py
    @wsgi.action('os-detach')
    def _detach(self, req, id, body):
        volume = self.volume_api.get(context, id)
        attachment_id = None
        if body['os-detach']:
            attachment_id = body['os-detach'].get('attachment_id', None)
        try:
            self.volume_api.detach(context, volume, attachment_id)

2)最后cinder-api通过RPC请求到cinder-volume节点，调用remove_export,移除target信息，更新数据库，把volume状态改为available，attach_status状态为detached

cindervolumeapi.py
    @wrap_check_policy
    def detach(self, context, volume, attachment_id):
        if volume['status'] == 'maintenance':
            LOG.info(_LI('Unable to detach volume, '
                         'because it is in maintenance.'), resource=volume)
            msg = _("The volume cannot be detached in maintenance mode.")
            raise exception.InvalidVolume(reason=msg)
        detach_results = self.volume_rpcapi.detach_volume(context, volume,
                                                          attachment_id)
        LOG.info(_LI("Detach volume completed successfully."),
                 resource=volume)
        return detach_results

最终调用cinder-volume服务的manage.py文件中的detach_volume接口

    @coordination.synchronized('{volume_id}-{f_name}')
    def detach_volume(self, context, volume_id, attachment_id=None,
                      volume=None):
        """Updates db to show volume is detached."""
        # TODO(vish): refactor this into a more general "unreserve"
        # FIXME(lixiaoy1): Remove this in v4.0 of RPC API.
        if volume is None:
            # For older clients, mimic the old behavior and look up the volume
            # by its volume_id.
            volume = objects.Volume.get_by_id(context, volume_id)

        if attachment_id:
            try:
                attachment = objects.VolumeAttachment.get_by_id(context,
                                                                attachment_id)
            except exception.VolumeAttachmentNotFound:
                LOG.info(_LI("Volume detach called, but volume not attached."),
                         resource=volume)
                # We need to make sure the volume status is set to the correct
                # status.  It could be in detaching status now, and we don't
                # want to leave it there.
                volume.finish_detach(attachment_id)
                return
        else:
            # We can try and degrade gracefully here by trying to detach
            # a volume without the attachment_id here if the volume only has
            # one attachment.  This is for backwards compatibility.
            attachments = volume.volume_attachment
            if len(attachments) > 1:
                # There are more than 1 attachments for this volume
                # we have to have an attachment id.
                msg = _("Detach volume failed: More than one attachment, "
                        "but no attachment_id provided.")
                LOG.error(msg, resource=volume)
                raise exception.InvalidVolume(reason=msg)
            elif len(attachments) == 1:
                attachment = attachments[0]
            else:
                # there aren't any attachments for this volume.
                # so set the status to available and move on.
                LOG.info(_LI("Volume detach called, but volume not attached."),
                         resource=volume)
                volume.status = 'available'
                volume.attach_status = fields.VolumeAttachStatus.DETACHED
                volume.save()
                return

        self._notify_about_volume_usage(context, volume, "detach.start")
        try:
            # NOTE(flaper87): Verify the driver is enabled
            # before going forward. The exception will be caught
            # and the volume status updated.
            utils.require_driver_initialized(self.driver)

            LOG.info(_LI('Detaching volume %(volume_id)s from instance '
                         '%(instance)s.'),
                     {'volume_id': volume_id,
                      'instance': attachment.get('instance_uuid')},
                     resource=volume)
            self.driver.detach_volume(context, volume, attachment)
        except Exception as ex:
            with excutils.save_and_reraise_exception():
                self.db.volume_attachment_update(
                    context, attachment.get('id'), {
                        'attach_status':
                            fields.VolumeAttachStatus.ERROR_DETACHING})

            self.db.volume_metadata_update(context,
                                           volume_id,
                                           {'error': six.text_type(ex)},
                                           False)

        # NOTE(jdg): We used to do an ensure export here to
        # catch upgrades while volumes were attached (E->F)
        # this was necessary to convert in-use volumes from
        # int ID's to UUID's.  Don't need this any longer

        # We're going to remove the export here
        # (delete the iscsi target)
        try:
            utils.require_driver_initialized(self.driver)
            self.driver.remove_export(context.elevated(), volume)
        except exception.DriverNotInitialized:
            with excutils.save_and_reraise_exception():
                LOG.exception(_LE("Detach volume failed, due to "
                                  "uninitialized driver."),
                              resource=volume)
        except Exception as ex:
            LOG.exception(_LE("Detach volume failed, due to "
                              "remove-export failure."),
                          resource=volume)
            raise exception.RemoveExportException(volume=volume_id,
                                                  reason=six.text_type(ex))

        volume.finish_detach(attachment.id)
        self._notify_about_volume_usage(context, volume, "detach.end")
        LOG.info(_LI("Detach volume completed successfully."), resource=volume)