OpenStack 提供 Block Storage Service 的是 Cinder，其具体功能是：
-
提供 REST API 使用户能够查询和管理 volume、volume snapshot 以及 volume type
-
提供 scheduler 调度 volume 创建请求，合理优化存储资源的分配
-
通过 driver 架构支持多种 back-end（后端）存储方式，包括 LVM，NFS，Ceph 和其他诸如 EMC、IBM 等商业存储产品和方案
Cinder 架构
下图是 cinder 的逻辑架构图
Cinder 包含如下几个组件：
cinder-api
接收 API 请求，调用 cinder-volume 执行操作。
cinder-volume
管理 volume 的服务，与 volume provider 协调工作，管理 volume 的生命周期。运行 cinder-volume 服务的节点被称作为存储节点。
cinder-scheduler
scheduler 通过调度算法选择最合适的存储节点创建 volume。
volume provider
数据的存储设备，为 volume 提供物理存储空间。
cinder-volume 支持多种 volume provider，每种 volume provider 通过自己的 driver 与cinder-volume 协调工作。
Message Queue
Cinder 各个子服务通过消息队列实现进程间通信和相互协作。因为有了消息队列，子服务之间实现了解耦，这种松散的结构也是分布式系统的重要特征。
Database Cinder 有一些数据需要存放到数据库中，一般使用 MySQL。数据库是安装在控制节点上的，比如在我们的实验环境中，可以访问名称为“cinder”的数据库。
在控制节点可以看到，部署了cinder-api，cinder-schedule，cinder-volume,控制节点即是存储节点，又是控制节点。当然也可以用单独的机器来部署cinder-volume
[root@cc07 ~]# ps -e|grep cinder 7007 ? 04:39:10 cinder-volume 7193 ? 10:36:20 cinder-volume 13007 pts/6 04:27:02 cinder-api 13385 pts/14 11:13:49 cinder-schedule 13794 ? 04:41:05 cinder-volume 14423 pts/28 00:24:10 cinder-backup 19069 pts/6 10:17:50 cinder-api 19070 pts/6 10:29:14 cinder-api 19071 pts/6 10:25:13 cinder-api 19072 pts/6 10:22:08 cinder-api 19073 pts/6 10:31:14 cinder-api 19074 pts/6 10:17:53 cinder-api 19075 pts/6 10:29:18 cinder-api 19076 pts/6 10:21:52 cinder-api 19262 ? 10:58:48 cinder-volume 19267 ? 10:57:51 cinder-volume 19272 ? 10:55:53 cinder-volume
存储节点支持多种 volume provider，包括 LVM, NFS, Ceph, GlusterFS，以及 EMC, IBM 等商业存储系统。 cinder-volume 为这些 volume provider 定义了统一的 driver 接口，volume provider 只需要实现这些接口，就可以 driver 的形式即插即用到 OpenStack 中。下面是 cinder driver 的架构示意图：
在 cinder-volume 的配置文件 /etc/cinder/cinder.conf 中 volume_driver 配置项设置该存储节点使用哪种 volume provider 的 driver，下面的示例表示使用的是 LVM和ceph
[DEFAULT] enabled_backends=ceph,ceph-ssd,cephSsd debug = True log_dir = /var/log/kolla/cinder use_forwarded_for = true use_stderr = False enable_v1_api = false volume_name_template = volume-%s glance_api_servers = http://172.16.12.7:9292 glance_api_version = 2 os_region_name = RegionOne osapi_volume_listen = 172.16.12.7 osapi_volume_listen_port = 8776 api_paste_config = /etc/cinder/api-paste.ini nova_catalog_info = compute:nova:internalURL auth_strategy = keystone [database] connection = mysql+pymysql://cinder:cinder@172.16.12.7/cinder max_retries = -1 [keystone_authtoken] auth_uri = http://172.16.12.7:5000 auth_url = http://172.16.12.7:35357 auth_type = password project_domain_id = default user_domain_id = default project_name = service username = cinder password = cinder memcache_security_strategy = ENCRYPT memcache_secret_key = dKcXVdSMqdHcQUgjE0ZiB2L9VeGyhB49cSXsA8Ib memcached_servers = 172.16.12.7:11211 [oslo_concurrency] lock_path = /var/lib/cinder/tmp [oslo_messaging_rabbit] rabbit_userid = openstack rabbit_password = huacloudhuacloud rabbit_ha_queues = true rabbit_hosts = 172.16.12.7:5672 [lvm] volume_group = cinder-volumes volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver volume_backend_name = iscsi iscsi_helper = lioadm iscsi_protocol = iscsi [ceph] volume_driver=cinder.volume.drivers.rbd.RBDDriver rbd_pool=volumes rbd_ceph_conf=/etc/ceph/ceph.conf rbd_flatten_volume_from_snapshot=false rbd_max_clone_depth=5 glance_api_version=1 volume_backend_name=ceph [ceph-ssd] volume_driver=cinder.volume.drivers.rbd.RBDDriver rbd_pool=volumes rbd_ceph_conf=/etc/ceph/ceph.conf rbd_flatten_volume_from_snapshot=false rbd_max_clone_depth=5 glance_api_version=1 volume_backend_name=ceph-ssd [cephSsd] volume_driver=cinder.volume.drivers.rbd.RBDDriver rbd_pool=volumes rbd_ceph_conf=/etc/ceph/ceph.conf rbd_flatten_volume_from_snapshot=false rbd_max_clone_depth=5 glance_api_version=1 volume_backend_name=cephSsd
上面的lvm中volume_group指的是LVM中用vgcreate创建出来的卷组vg
基本命令
Volume Type
Cinder 允许用户创建 Volume 时通过 Volume Type 指定需要的 Capabilities，通过 Volume Type 的 Extra Specs 定义 Capabilities，cinder-volume 会在自己的配置文件 /etc/cinder/cinder.conf 中设置“volume_backend_name”这个参数，其作用是为存储节点的 Volume Provider 命名。这样，CapabilitiesFilter 就可以通过 Volume Type 的“volume_backend_name”筛选出指定的 Volume Provider。
主要命令
extra-specs-list Lists current volume types and extra specs. qos-associate Associates qos specs with specified volume type. qos-disassociate Disassociates qos specs from specified volume type. retype Changes the volume type for a volume. type-access-add Adds volume type access for the given project. type-access-list Print access information about the given volume type. type-access-remove Removes volume type access for the given project. type-create Creates a volume type. type-default List the default volume type. type-delete Deletes a volume type. type-key Sets or unsets extra_spec for a volume type. type-list Lists available 'volume types'. (Admin only will see private types) type-show Show volume type details. type-update Updates volume type name, description, and/or
△显示存储卷类型列表
(nova-api)[root@cc07 /]# cinder extra-specs-list +--------------------------------------+----------+-------------------------------------+ | ID | Name | extra_specs | +--------------------------------------+----------+-------------------------------------+ | 0a220534-3404-40ff-b0a0-a449429e13c1 | ceph | {'volume_backend_name': 'ceph'} | | 1bcf273a-79de-4847-9dce-57628b2cd185 | vmdk | {'volume_backend_name': 'vmdk'} | | 747632fd-e8f2-403a-978d-892002498497 | ceph_new | {'volume_backend_name': 'ceph_new'} | | 88d1db93-1104-489c-ac9c-389a4e4736a1 | ceph-ssd | {'volume_backend_name': 'ceph-ssd'} | | 9b59e61e-b588-43fc-9c47-16337b595e74 | iscsi | {'volume_backend_name': 'iscsi'} | | b1c40afc-a8e3-4b78-b534-5d73a8ba98a8 | cephSsd | {'volume_backend_name': 'cephSsd'} | +--------------------------------------+----------+-------------------------------------+
△创建存储卷类型
(nova-api)[root@cc07 /]# cinder type-create --is-public true hzbtest +--------------------------------------+---------+-------------+-----------+ | ID | Name | Description | Is_Public | +--------------------------------------+---------+-------------+-----------+ | 6246eb36-4268-48d5-bbac-8da96cd3ea59 | hzbtest | - | True | +--------------------------------------+---------+-------------+-----------+
(nova-api)[root@cc07 /]# cinder extra-specs-list +--------------------------------------+----------+-------------------------------------+ | ID | Name | extra_specs | +--------------------------------------+----------+-------------------------------------+ | 0a220534-3404-40ff-b0a0-a449429e13c1 | ceph | {'volume_backend_name': 'ceph'} | | 1bcf273a-79de-4847-9dce-57628b2cd185 | vmdk | {'volume_backend_name': 'vmdk'} | | 6246eb36-4268-48d5-bbac-8da96cd3ea59 | hzbtest | {} | | 747632fd-e8f2-403a-978d-892002498497 | ceph_new | {'volume_backend_name': 'ceph_new'} | | 88d1db93-1104-489c-ac9c-389a4e4736a1 | ceph-ssd | {'volume_backend_name': 'ceph-ssd'} | | 9b59e61e-b588-43fc-9c47-16337b595e74 | iscsi | {'volume_backend_name': 'iscsi'} | | b1c40afc-a8e3-4b78-b534-5d73a8ba98a8 | cephSsd | {'volume_backend_name': 'cephSsd'} | +--------------------------------------+----------+-------------------------------------+
△type-key
新创建的volumeType并没有指定对接的是哪一个volume_backend。这时我们需要设置一个
usage: cinder type-key <vtype> <action> <key=value> [<key=value> ...] Sets or unsets extra_spec for a volume type. Positional arguments: <vtype> Name or ID of volume type. <action> The action. Valid values are "set" or "unset." <key=value> The extra specs key and value pair to set or unset. For unset, specify only the key.
(nova-api)[root@cc07 /]# cinder type-key hzbtest set volume_backend_name=ceph (nova-api)[root@cc07 /]# cinder extra-specs-list +--------------------------------------+----------+-------------------------------------+ | ID | Name | extra_specs | +--------------------------------------+----------+-------------------------------------+ | 0a220534-3404-40ff-b0a0-a449429e13c1 | ceph | {'volume_backend_name': 'ceph'} | | 1bcf273a-79de-4847-9dce-57628b2cd185 | vmdk | {'volume_backend_name': 'vmdk'} | | 6246eb36-4268-48d5-bbac-8da96cd3ea59 | hzbtest | {'volume_backend_name': 'ceph'} | | 747632fd-e8f2-403a-978d-892002498497 | ceph_new | {'volume_backend_name': 'ceph_new'} | | 88d1db93-1104-489c-ac9c-389a4e4736a1 | ceph-ssd | {'volume_backend_name': 'ceph-ssd'} | | 9b59e61e-b588-43fc-9c47-16337b595e74 | iscsi | {'volume_backend_name': 'iscsi'} | | b1c40afc-a8e3-4b78-b534-5d73a8ba98a8 | cephSsd | {'volume_backend_name': 'cephSsd'} | +--------------------------------------+----------+-------------------------------------+
取消设置
nova-api)[root@cc07 /]# cinder type-key hzbtest unset volume_backend_name=ceph
△删除volumeType
cinder type-delete 6246eb36-4268-48d5-bbac-8da96cd3ea59
Volume
△显示存储卷列表（和nova volume-list命令功能相同）
cinder list
△创建存储卷
cinder create --name VOLNAME SIZE（SIZE的单位为GB）
(nova-api)[root@cc07 /]# cinder create --name hzb-1g --volume-type ceph 1 +------------------------------+--------------------------------------+ | Property | Value | +------------------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2018-10-29T02:14:28.000000 | | description | None | | encrypted | False | | id | 48e23242-6ce6-486f-8841-417693b5f37c | | metadata | {} | | multiattach | False | | name | hzb-1g | | os-vol-tenant-attr:tenant_id | 26a8e1feb372493d831c19d4d9e28d73 | | replication_status | disabled | | size | 1 | | snapshot_id | None | | source_volid | None | | status | creating | | updated_at | None | | user_id | d5fa609f7ffd4454b45b555940f41594 | | volume_type | ceph | +------------------------------+--------------------------------------+ (nova-api)[root@cc07 /]# cinder list +--------------------------------------+-----------+--------+------+-------------+----------+-------------+ | ID | Status | Name | Size | Volume Type | Bootable | Attached to | +--------------------------------------+-----------+--------+------+-------------+----------+-------------+ | 48e23242-6ce6-486f-8841-417693b5f37c | available | hzb-1g | 1 | ceph | false | | +--------------------------------------+-----------+--------+------+-------------+----------+-------------+
△删除存储卷
cinder delete VOLNAME-OR-ID
cinder force-delete VOLNAME-OR-ID（强制删除）
△重命名存储卷
cinder rename VOLNAME-OR-ID NEW-VOLNAME
△显示存储卷信息
cinder show VOLNAME-OR-ID
△存储卷元数据
cinder metadata VOLNAME-OR-ID set KEY=VALUE
cinder metadata VOLNAME-OR-ID unset KEY
△创建存储卷备份
cinder backup-create --display-name BACKUP-VOLNAME VOLNAME-OR-ID
△删除存储卷备份
cinder backup-delete VOLNAME-OR-ID
△显示存储卷备份信息
cinder backup-show VOLNAME-OR-ID
△显示存储卷备份列表
cinder backup-list
△创建存储卷快照
cinder snapshot-create --display-name SNAPSHOT-VOLNAME VOLNAME-OR-ID
△删除存储卷快照
cinder snapshot-delete VOLNAME-OR-ID
△重命名存储卷快照
cinder snapshot-rename VOLNAME-OR-ID NEW-VOLNAME
△显示存储卷快照信息
cinder snapshot-show VOLNAME-OR-ID
△显示存储卷快照列表
cinder snapshot-list
△上传存储卷作为镜像
cinder upload-to-image VOLNAME-OR-ID IMAGE-NAME-OR-ID
△重置存储卷的status
假如有一个volume，状态 为in-use，但是确没有具体的虚拟机在使用，这是一种僵尸卷，无法删除，这时候需要重置状态为available才能删除
cinder reset-state ad7876d3-5a2a-46ee-8127-df8f60f0f571