简介
本次部署nova组件，分别部署nova和nova-compute，nova部署到controller节点，nova-compute部署到另一台设备，这样部署说明nova-compute是可以部署多个进行横向扩展的。下边开始部署nova组件，controller节点IP 192.168.46.130，nova-compute节点IP 192.168.46.133
部署controller nova
创建数据库
与其他组件的部署一样，首先需要创建nova需要的数据库，操作如下：
$ mysql -u root -p
创建nova_api、nova、nova_cell0数据库
MariaDB [(none)]> CREATE DATABASE nova_api; MariaDB [(none)]> CREATE DATABASE nova; MariaDB [(none)]> CREATE DATABASE nova_cell0;
授权数据库
MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova_api.* TO 'nova'@'localhost' IDENTIFIED BY 'nova'; MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova_api.* TO 'nova'@'%' IDENTIFIED BY 'nova'; MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'localhost' IDENTIFIED BY 'nova'; MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'%' IDENTIFIED BY 'nova'; MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova_cell0.* TO 'nova'@'localhost' IDENTIFIED BY 'nova'; MariaDB [(none)]> GRANT ALL PRIVILEGES ON nova_cell0.* TO 'nova'@'%' IDENTIFIED BY 'nova';
组件部署
首先设置所需要的环境变量，这个在前边组件的部署中有过说明：
[root@localhost ~]# source admin-openstack.sh
1、创建nova user
$ openstack user create --domain default --password-prompt nova
User Password:
Repeat User Password:
+---------------------+----------------------------------+
| Field | Value |
+---------------------+----------------------------------+
| domain_id | default |
| enabled | True |
| id | 8a7dbf5279404537b1c7b86c033620fe |
| name | nova |
| options | {} |
| password_expires_at | None |
+---------------------+----------------------------------+
2、为nova用户设置admin角色和项目
$ openstack role add --project service --user nova admin
3、注册服务
$ openstack service create --name nova --description "OpenStack Compute" compute +-------------+----------------------------------+ | Field | Value | +-------------+----------------------------------+ | description | OpenStack Compute | | enabled | True | | id | 060d59eac51b4594815603d75a00aba2 | | name | nova | | type | compute | +-------------+----------------------------------+
4、注册endpoints
$ openstack endpoint create --region RegionOne compute public http://192.168.46.130:8774/v2.1 +--------------+-------------------------------------------+ | Field | Value | +--------------+-------------------------------------------+ | enabled | True | | id | 3c1caa473bfe4390a11e7177894bcc7b | | interface | public | | region | RegionOne | | region_id | RegionOne | | service_id | 060d59eac51b4594815603d75a00aba2 | | service_name | nova | | service_type | compute | | url | http://192.168.46.130:8774/v2.1 | +--------------+-------------------------------------------+ $ openstack endpoint create --region RegionOne compute internal http://192.168.46.130:8774/v2.1 +--------------+-------------------------------------------+ | Field | Value | +--------------+-------------------------------------------+ | enabled | True | | id | e3c918de680746a586eac1f2d9bc10ab | | interface | internal | | region | RegionOne | | region_id | RegionOne | | service_id | 060d59eac51b4594815603d75a00aba2 | | service_name | nova | | service_type | compute | | url | http://192.168.46.130:8774/v2.1 | +--------------+-------------------------------------------+ $ openstack endpoint create --region RegionOne compute admin http://192.168.46.130:8774/v2.1 +--------------+-------------------------------------------+ | Field | Value | +--------------+-------------------------------------------+ | enabled | True | | id | 38f7af91666a47cfb97b4dc790b94424 | | interface | admin | | region | RegionOne | | region_id | RegionOne | | service_id | 060d59eac51b4594815603d75a00aba2 | | service_name | nova | | service_type | compute | | url | http://192.168.46.130:8774/v2.1 | +--------------+-------------------------------------------+
5、创建Placement service
$ openstack user create --domain default --password-prompt placement
User Password:
Repeat User Password:
+---------------------+----------------------------------+
| Field | Value |
+---------------------+----------------------------------+
| domain_id | default |
| enabled | True |
| id | fa742015a6494a949f67629884fc7ec8 |
| name | placement |
| options | {} |
| password_expires_at | None |
+---------------------+----------------------------------+
6、为Placement service添加项目和角色
$ openstack role add --project service --user placement admin
7、注册Placement服务
$ openstack service create --name placement --description "Placement API" placement +-------------+----------------------------------+ | Field | Value | +-------------+----------------------------------+ | description | Placement API | | enabled | True | | id | 2d1a27022e6e4185b86adac4444c495f | | name | placement | | type | placement | +-------------+----------------------------------+
8、为Placement服务注册endpoints
$ openstack endpoint create --region RegionOne placement public http://192.168.46.130:8778 +--------------+----------------------------------+ | Field | Value | +--------------+----------------------------------+ | enabled | True | | id | 2b1b2637908b4137a9c2e0470487cbc0 | | interface | public | | region | RegionOne | | region_id | RegionOne | | service_id | 2d1a27022e6e4185b86adac4444c495f | | service_name | placement | | service_type | placement | | url | http://192.168.46.130:8778 | +--------------+----------------------------------+ $ openstack endpoint create --region RegionOne placement internal http://192.168.46.130:8778 +--------------+----------------------------------+ | Field | Value | +--------------+----------------------------------+ | enabled | True | | id | 02bcda9a150a4bd7993ff4879df971ab | | interface | internal | | region | RegionOne | | region_id | RegionOne | | service_id | 2d1a27022e6e4185b86adac4444c495f | | service_name | placement | | service_type | placement | | url | http://192.168.46.130:8778 | +--------------+----------------------------------+ $ openstack endpoint create --region RegionOne placement admin http://192.168.46.130:8778 +--------------+----------------------------------+ | Field | Value | +--------------+----------------------------------+ | enabled | True | | id | 3d71177b9e0f406f98cbff198d74b182 | | interface | admin | | region | RegionOne | | region_id | RegionOne | | service_id | 2d1a27022e6e4185b86adac4444c495f | | service_name | placement | | service_type | placement | | url | http://192.168.46.130:8778 | +--------------+----------------------------------+
配置组件
- 安装组件包
# yum install openstack-nova-api openstack-nova-conductor openstack-nova-console openstack-nova-novncproxy openstack-nova-scheduler openstack-nova-placement-api
- 修改/etc/nova/nova.conf配置文件
1、设置对外的API
[DEFAULT] # ... enabled_apis = osapi_compute,metadata
2、配置数据库
[api_database] # ... connection = mysql+pymysql://nova:nova@192.168.46.130/nova_api [database] # ... connection = mysql+pymysql://nova:nova@192.168.46.130/nova
3、配置RabbitMQ
[DEFAULT] # ... transport_url = rabbit://openstack:openstack@192.168.46.130
4、打开keystone认证和配置认证token
[api] # ... auth_strategy = keystone [keystone_authtoken] auth_uri = http://192.168.46.130:5000 auth_url = http://192.168.46.130:35357 memcached_servers = 192.168.46.130:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = nova password = nova
5、关闭防火墙配置
[DEFAULT] # ... use_neutron = True firewall_driver = nova.virt.firewall.NoopFirewallDriver
6、vnc配置
[vnc] enabled = true # ... vncserver_listen = 192.168.46.130 vncserver_proxyclient_address = 192.168.46.130
7、配置本地镜像服务API
[glance] # ... api_servers= http://192.168.46.130:9292
8、锁文件路径
[oslo_concurrency] # ... lock_path = /var/lib/nova/tmp
9、配置Placement API
[placement] os_region_name = RegionOne project_domain_name = Default project_name = service auth_type = password user_domain_name = Default auth_url = http://192.168.46.130:35357/v3 username = placement password = placement
10、修改当前版本bug，编辑/etc/httpd/conf.d/00-nova-placement-api.conf文件，增加以下代码
<Directory /usr/bin>
<IfVersion >= 2.4>
Require all granted
</IfVersion>
<IfVersion < 2.4>
Order allow,deny
Allow from all
</IfVersion>
</Directory>
11、启动httpd服务
# systemctl restart httpd
最后的配置
1、同步cell0数据库
# su -s /bin/sh -c "nova-manage cell_v2 map_cell0" nova
2、创建cell1
# su -s /bin/sh -c "nova-manage cell_v2 create_cell --name=cell1 --verbose" nova 109e1d4b-536a-40d0-83c6-5f121b82b650
3、同步nova数据库
# su -s /bin/sh -c "nova-manage db sync" nova
4、验证cell0和cell1
# nova-manage cell_v2 list_cells +-------+--------------------------------------+ | Name | UUID | +-------+--------------------------------------+ | cell1 | 109e1d4b-536a-40d0-83c6-5f121b82b650 | | cell0 | 00000000-0000-0000-0000-000000000000 | +-------+--------------------------------------+
配置服务自动启动
# systemctl enable openstack-nova-api.service openstack-nova-consoleauth.service openstack-nova-scheduler.service openstack-nova-conductor.service openstack-nova-novncproxy.service # systemctl start openstack-nova-api.service openstack-nova-consoleauth.service openstack-nova-scheduler.service openstack-nova-conductor.service openstack-nova-novncproxy.service
部署compute node
1、安装包
# yum install openstack-nova-compute
2、编辑/etc/nova/nova.conf配置文件
- 打开API
[DEFAULT] # ... enabled_apis = osapi_compute,metadata
- 配置RabbitMQ
[DEFAULT] # ... transport_url= rabbit://openstack:openstack@192.168.46.130
- 配置keystone认证
[api] # ... auth_strategy = keystone [keystone_authtoken] auth_uri = http://192.168.46.130:5000 auth_url = http://192.168.46.130:35357 memcached_servers = 192.168.46.130:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = nova password = nova
- 关闭防火墙驱动
[DEFAULT] # ... use_neutron = True firewall_driver = nova.virt.firewall.NoopFirewallDriver
- vnc配置
[vnc] # ... enabled = True vncserver_listen= 192.168.46.133 vncserver_proxyclient_address= 192.168.46.133 novncproxy_base_url=http://192.168.46.130:6080/vnc_auto.html
- 配置Glance服务API
[glance] # ... api_servers= http://192.168.46.130:9292
- 锁文件路径
[oslo_concurrency] # ... lock_path = /var/lib/nova/tmp
- 配置Placement API
[placement] os_region_name = RegionOne project_domain_name = Default project_name = service auth_type = password user_domain_name = Default auth_url = http://192.168.46.130:35357/v3 username = placement password = placement
- 配置libvirt，在生产环境中一般使用kvm
[libvirt] # ... virt_type = qemu
- 启动服务
# systemctl enable libvirtd.service openstack-nova-compute.service //设置自动启动 # systemctl start libvirtd.service openstack-nova-compute.service //启动服务
问题记录
在以上执行启动服务时，命令行长时间卡住，一般说明连接RabbitMQ失败，可以在/var/log/nova/nova-compute.log文件中查看详细报错，报错如下：
2019-05-11 12:02:57.432 5308 ERROR oslo.messaging._drivers.impl_rabbit [req-503bd955-7075-44a8-b553-f70b9437ff3b - - - - -] [84517112-2f15-4dfa-b9f0-e195a80a67c2] AMQP server on 192.168.46.130:5672 is unreachable: [Errno 113] EHOSTUNREACH. Trying again in 32 seconds.: error: [Errno 113] EHOSTUNREACH 2019-05-11 12:03:29.606 5308 ERROR oslo.messaging._drivers.impl_rabbit [req-503bd955-7075-44a8-b553-f70b9437ff3b - - - - -] [84517112-2f15-4dfa-b9f0-e195a80a67c2] AMQP server on 192.168.46.130:5672 is unreachable: [Errno 113] EHOSTUNREACH. Trying again in 32 seconds.: error: [Errno 113] EHOSTUNREACH 2019-05-11 12:04:01.792 5308 ERROR oslo.messaging._drivers.impl_rabbit [req-503bd955-7075-44a8-b553-f70b9437ff3b - - - - -] [84517112-2f15-4dfa-b9f0-e195a80a67c2] AMQP server on 192.168.46.130:5672 is unreachable: [Errno 113] EHOSTUNREACH. Trying again in 32 seconds.: error: [Errno 113] EHOSTUNREACH
以上错误说明nova-compute节点连接controller节点的RabbitMQ失败了，一般原因是controller上配置了iptables，需要设置iptables把RabbitMQ的5672端口打开。设置如下：
# iptables -I INPUT -p tcp --dport 5672 -j ACCEPT #添加规则 # service iptables save #保存设置 # service iptables restart # 重启iptables，生效规则
添加compute node到cell数据库
- 在controller node上查看，确定在数据库中有nova-compute
[root@localhost ~]# source admin-openstack.sh [root@localhost ~]# openstack compute service list +----+------------------+-----------------------+----------+---------+-------+----------------------------+ | ID | Binary | Host | Zone | Status | State | Updated At | +----+------------------+-----------------------+----------+---------+-------+----------------------------+ | 1 | nova-consoleauth | localhost.localdomain | internal | enabled | up | 2019-05-12T10:36:09.000000 | | 2 | nova-conductor | localhost.localdomain | internal | enabled | up | 2019-05-12T10:36:09.000000 | | 3 | nova-scheduler | localhost.localdomain | internal | enabled | up | 2019-05-12T10:36:10.000000 | | 8 | nova-compute | node2 | nova | enabled | up | 2019-05-12T10:36:07.000000 | +----+------------------+-----------------------+----------+---------+-------+----------------------------+
- 发现compute node，每次添加新的compute nodes时，都需要在controller node上执行nova-manage cell_v2 discover_hosts注册新的compute nodes
# su -s /bin/sh -c "nova-manage cell_v2 discover_hosts --verbose" nova Found 2 cell mappings. Skipping cell0 since it does not contain hosts. Getting compute nodes from cell 'cell1': ad5a5985-a719-4567-98d8-8d148aaae4bc Found 1 computes in cell: ad5a5985-a719-4567-98d8-8d148aaae4bc Checking host mapping for compute host 'compute': fe58ddc1-1d65-4f87-9456-bc040dc106b3 Creating host mapping for compute host 'compute': fe58ddc1-1d65-4f87-9456-bc040dc106b3
通过以上的组件部署，nova-compute可以有多个，并且当配置完一个nova-compute时，其他的与此类似。