openstack安装配置—— file share node 配置
实际生产中，很多时候需要数据源的共享来实现多节点的实时数据保持一致，openstack官方提供了manila服务模块实现了云盘共享，manila服务也是需要manila服务端和存储节点共同组成的，本实验中为了节约虚机节点，就把manila服务端安装在了controller节点上，manila数据存储节点和cinder存储节点合并使用一个虚机节点，各自使用了一块独立硬盘。manila服务端配置
准备数据库
# mysql
Welcome to the MariaDB monitor.Commands end with ; or \g.
Your MariaDB connection id is 5826
Server version: 10.1.20-MariaDB MariaDB Server
Copyright (c) 2000, 2016, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> CREATE DATABASE manila;
Query OK, 1 row affected (0.00 sec)
MariaDB [(none)]> GRANT ALL PRIVILEGES ON manila.* TO 'manila'@'localhost' \
-> IDENTIFIED BY 'MANILA_DBPASS';
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> GRANT ALL PRIVILEGES ON manila.* TO 'manila'@'%' \
-> IDENTIFIED BY 'MANILA_DBPASS';
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> quit
Bye
#
创建 manila 用户并添加管理员权限
# . admin-openrc
# openstack user create --domain default --password-prompt manila
User Password:
Repeat User Password:
+-----------+----------------------------------+
| Field | Value |
+-----------+----------------------------------+
| domain_id | 3ad6ac5f704c494e9f16b9e04ef745fe |
| enabled | True |
| id | 4313269c426e4ad882488fc8fe738bd4 |
| name | manila |
+-----------+----------------------------------+
# openstack role add --project service --user manila admin
#
创建 manila 和 manilav2 服务
# openstack service create --name manila \
> --description "OpenStack Shared File Systems" share
+-------------+----------------------------------+
| Field | Value |
+-------------+----------------------------------+
| description | OpenStack Shared File Systems |
| enabled | True |
| id | 8989f2832a7d430ea292fc4574843576 |
| name | manila |
| type | share |
+-------------+----------------------------------+
# openstack service create --name manilav2 \
> --description "OpenStack Shared File Systems" sharev2
+-------------+----------------------------------+
| Field | Value |
+-------------+----------------------------------+
| description | OpenStack Shared File Systems |
| enabled | True |
| id | a92939e66ab1464d8010e316b3c23a11 |
| name | manilav2 |
| type | sharev2 |
+-------------+----------------------------------+
#
创建文件分享服务的API endpoint
# openstack endpoint create --region RegionOne \
> share public http://controller:8786/v1/%\(tenant_id\)s
+--------------+-----------------------------------------+
| Field | Value |
+--------------+-----------------------------------------+
| enabled | True |
| id | de8406193ea5405bbc53e336c560df31 |
| interface | public |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 8989f2832a7d430ea292fc4574843576 |
| service_name | manila |
| service_type | share |
| url | http://controller:8786/v1/%(tenant_id)s |
+--------------+-----------------------------------------+
# openstack endpoint create --region RegionOne \
> share internal http://controller:8786/v1/%\(tenant_id\)s
+--------------+-----------------------------------------+
| Field | Value |
+--------------+-----------------------------------------+
| enabled | True |
| id | 4582dfee0e1b4fc29dd213af00b6a15f |
| interface | internal |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 8989f2832a7d430ea292fc4574843576 |
| service_name | manila |
| service_type | share |
| url | http://controller:8786/v1/%(tenant_id)s |
+--------------+-----------------------------------------+
# openstack endpoint create --region RegionOne \
> share admin http://controller:8786/v1/%\(tenant_id\)s
+--------------+-----------------------------------------+
| Field | Value |
+--------------+-----------------------------------------+
| enabled | True |
| id | f1aa8cf3ae47481cb8dafe287bd86fb4 |
| interface | admin |
| region | RegionOne |
| region_id | RegionOne |
| service_id | 8989f2832a7d430ea292fc4574843576 |
| service_name | manila |
| service_type | share |
| url | http://controller:8786/v1/%(tenant_id)s |
+--------------+-----------------------------------------+
#
# openstack endpoint create --region RegionOne \
> sharev2 public http://controller:8786/v2/%\(tenant_id\)s
+--------------+-----------------------------------------+
| Field | Value |
+--------------+-----------------------------------------+
| enabled | True |
| id | 27f84d115d7b4807a81385db5a89df23 |
| interface | public |
| region | RegionOne |
| region_id | RegionOne |
| service_id | a92939e66ab1464d8010e316b3c23a11 |
| service_name | manilav2 |
| service_type | sharev2 |
| url | http://controller:8786/v2/%(tenant_id)s |
+--------------+-----------------------------------------+
# openstack endpoint create --region RegionOne \
> sharev2 internal http://controller:8786/v2/%\(tenant_id\)s
+--------------+-----------------------------------------+
| Field | Value |
+--------------+-----------------------------------------+
| enabled | True |
| id | d6733127a36a4fbfb67682eba0117fca |
| interface | internal |
| region | RegionOne |
| region_id | RegionOne |
| service_id | a92939e66ab1464d8010e316b3c23a11 |
| service_name | manilav2 |
| service_type | sharev2 |
| url | http://controller:8786/v2/%(tenant_id)s |
+--------------+-----------------------------------------+
# openstack endpoint create --region RegionOne \
> sharev2 admin http://controller:8786/v2/%\(tenant_id\)s
+--------------+-----------------------------------------+
| Field | Value |
+--------------+-----------------------------------------+
| enabled | True |
| id | f8f16f76dc444c55b5437ec337b7b888 |
| interface | admin |
| region | RegionOne |
| region_id | RegionOne |
| service_id | a92939e66ab1464d8010e316b3c23a11 |
| service_name | manilav2 |
| service_type | sharev2 |
| url | http://controller:8786/v2/%(tenant_id)s |
+--------------+-----------------------------------------+
#
安装并编辑配置文件
# yum install -y openstack-manila python-manilaclient
# cp /etc/manila/manila.conf{,.bak}
# vim /etc/manila/manila.conf
# grep -v ^# /etc/manila/manila.conf | tr -s [[:space:]]
rpc_backend = rabbit
default_share_type = default_share_type
rootwrap_config = /etc/manila/rootwrap.conf
auth_strategy = keystone
my_ip = 192.168.10.10
connection = mysql+pymysql://manila:MANILA_DBPASS@controller/manila
memcached_servers = controller:11211
auth_uri = http://controller:5000
auth_url = http://controller:35357
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = manila
password = MANILA_PASS
lock_path = /var/lib/manila/tmp
rabbit_host = controller
rabbit_userid = openstack
rabbit_password = RABBIT_PASS
#
导入manila数据库
# su -s /bin/sh -c "manila-manage db sync" manila
2017-07-28 17:53:02.668 20691 INFO alembic.runtime.migration [-] Context impl MySQLImpl.
2017-07-28 17:53:02.668 20691 INFO alembic.runtime.migration [-] Will assume non-transactional DDL.
2017-07-28 17:53:02.980 20691 INFO alembic.runtime.migration [-] Running upgrade-> 162a3e673105, manila_init
2017-07-28 17:53:11.109 20691 INFO alembic.runtime.migration [-] Running upgrade 162a3e673105 -> 211836bf835c, add access level
2017-07-28 17:53:11.601 20691 INFO alembic.runtime.migration [-] Running upgrade 211836bf835c -> 38e632621e5a, change volume_type to share_type
2017-07-28 17:53:11.602 20691 INFO 38e632621e5a_change_volume_type_to_share_type_py [-] Renaming column name shares.volume_type_id to shares.share_type.id
2017-07-28 17:53:11.636 20691 INFO 38e632621e5a_change_volume_type_to_share_type_py [-] Renaming volume_types table to share_types
2017-07-28 17:53:12.111 20691 INFO 38e632621e5a_change_volume_type_to_share_type_py [-] Creating share_type_extra_specs table
2017-07-28 17:53:12.574 20691 INFO 38e632621e5a_change_volume_type_to_share_type_py [-] Migrating volume_type_extra_specs to share_type_extra_specs
2017-07-28 17:53:12.578 20691 INFO 38e632621e5a_change_volume_type_to_share_type_py [-] Dropping volume_type_extra_specs table
2017-07-28 17:53:12.769 20691 INFO alembic.runtime.migration [-] Running upgrade 38e632621e5a -> 17115072e1c3, add_nova_net_id_column_to_share_networks
2017-07-28 17:53:13.310 20691 INFO alembic.runtime.migration [-] Running upgrade 17115072e1c3 -> 4ee2cf4be19a, Remove share_snapshots.export_location
2017-07-28 17:53:14.001 20691 INFO alembic.runtime.migration [-] Running upgrade 4ee2cf4be19a -> 59eb64046740, Add required extra spec
/usr/lib64/python2.7/site-packages/sqlalchemy/sql/default_comparator.py:153: SAWarning: The IN-predicate on "share_types.id" was invoked with an empty sequence. This results in a contradiction, which nonetheless can be expensive to evaluate.Consider alternative strategies for improved performance.
'strategies for improved performance.' % expr)
2017-07-28 17:53:14.076 20691 INFO alembic.runtime.migration [-] Running upgrade 59eb64046740 -> ef0c02b4366, Add_share_type_projects
2017-07-28 17:53:15.051 20691 INFO alembic.runtime.migration [-] Running upgrade ef0c02b4366 -> 30cb96d995fa, add public column for share
2017-07-28 17:53:15.643 20691 INFO alembic.runtime.migration [-] Running upgrade 30cb96d995fa -> 56cdbe267881, Add share_export_locations table
2017-07-28 17:53:16.636 20691 INFO alembic.runtime.migration [-] Running upgrade 56cdbe267881 -> 3a482171410f, add_driver_private_data_table
2017-07-28 17:53:16.927 20691 INFO alembic.runtime.migration [-] Running upgrade 3a482171410f -> 533646c7af38, Remove unused attr status
2017-07-28 17:53:17.801 20691 INFO alembic.runtime.migration [-] Running upgrade 533646c7af38 -> 3db9992c30f3, Transform statuses to lowercase
2017-07-28 17:53:17.910 20691 INFO alembic.runtime.migration [-] Running upgrade 3db9992c30f3 -> 5077ffcc5f1c, add_share_instances
2017-07-28 17:53:26.745 20691 INFO alembic.runtime.migration [-] Running upgrade 5077ffcc5f1c -> 579c267fbb4d, add_share_instances_access_map
2017-07-28 17:53:27.653 20691 INFO alembic.runtime.migration [-] Running upgrade 579c267fbb4d -> 1f0bd302c1a6, add_availability_zones_table
2017-07-28 17:53:32.028 20691 INFO alembic.runtime.migration [-] Running upgrade 1f0bd302c1a6 -> 55761e5f59c5, Add 'snapshot_support' extra spec to share types
2017-07-28 17:53:32.628 20691 INFO alembic.runtime.migration [-] Running upgrade 55761e5f59c5 -> 3651e16d7c43, Create Consistency Groups Tables and Columns
2017-07-28 17:53:37.354 20691 INFO alembic.runtime.migration [-] Running upgrade 3651e16d7c43 -> 323840a08dc4, Add shares.task_state
2017-07-28 17:53:38.012 20691 INFO alembic.runtime.migration [-] Running upgrade 323840a08dc4 -> dda6de06349, Add DB support for share instance export locations metadata.
2017-07-28 17:53:40.096 20691 INFO alembic.runtime.migration [-] Running upgrade dda6de06349 -> 344c1ac4747f, Remove access rules status and add access_rule_status to share_instance
model
2017-07-28 17:53:41.405 20691 INFO alembic.runtime.migration [-] Running upgrade 344c1ac4747f -> 293fac1130ca, Add replication attributes to Share and ShareInstance models.
2017-07-28 17:53:42.613 20691 INFO alembic.runtime.migration [-] Running upgrade 293fac1130ca -> 5155c7077f99, Add more network info attributes to 'network_allocations' table.
2017-07-28 17:53:45.223 20691 INFO alembic.runtime.migration [-] Running upgrade 5155c7077f99 -> eb6d5544cbbd, add provider_location to share_snapshot_instances
# mysql -e "show tables from manila"
+--------------------------------------------+
| Tables_in_manila |
+--------------------------------------------+
| alembic_version |
| availability_zones |
| cgsnapshot_members |
| cgsnapshots |
| consistency_group_share_type_mappings |
| consistency_groups |
| drivers_private_data |
| network_allocations |
| project_user_quotas |
| quota_classes |
| quota_usages |
| quotas |
| reservations |
| security_services |
| services |
| share_access_map |
| share_instance_access_map |
| share_instance_export_locations |
| share_instance_export_locations_metadata |
| share_instances |
| share_metadata |
| share_network_security_service_association |
| share_networks |
| share_server_backend_details |
| share_servers |
| share_snapshot_instances |
| share_snapshots |
| share_type_extra_specs |
| share_type_projects |
| share_types |
| shares |
+--------------------------------------------+
#
启用并启动manila服务
# systemctl enable openstack-manila-api.service openstack-manila-scheduler.service
Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-manila-api.service to /usr/lib/systemd/system/openstack-manila-api.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-manila-scheduler.service to /usr/lib/systemd/system/openstack-manila-scheduler.service.
# systemctl start openstack-manila-api.service openstack-manila-scheduler.service
# ss -tnl//manila服务启动成功后会新增8786端口
State Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0 128 *:8776 *:*
LISTEN 0 128 *:25672 *:*
LISTEN 0 128 192.168.10.10:3306 *:*
LISTEN 0 128 127.0.0.1:11211 *:*
LISTEN 0 128 *:9292 *:*
LISTEN 0 128 *:4369 *:*
LISTEN 0 128 *:9696 *:*
LISTEN 0 100 *:6080 *:*
LISTEN 0 128 *:8774 *:*
LISTEN 0 128 *:22022 *:*
LISTEN 0 128 *:8775 *:*
LISTEN 0 128 *:9191 *:*
LISTEN 0 128 :::5000 :::*
LISTEN 0 128 :::5672 :::*
LISTEN 0 128 ::1:11211 :::*
LISTEN 0 128 :::80 :::*
LISTEN 0 128 :::8786 :::*
LISTEN 0 128 :::35357 :::*
LISTEN 0 128 :::22022 :::*
#
manila存储节点配置
关闭block节点回到kvm宿主机进行操作
# qemu-img create -f qcow2 /kvm/images/share_lvm.qcow2 100G
Formatting '/kvm/images/share_lvm.qcow2', fmt=qcow2 size=107374182400 encryption=off cluster_size=65536 lazy_refcounts=off
# virsh edit block
添加如下内容
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/kvm/images/share_lvm.qcow2'/>
<target dev='vdc' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x11' function='0x0'/>
</disk>
block节点操作
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sr0 11:0 11024M0 rom
vda 252:0 0 400G0 disk
├─vda1 252:1 0 500M0 part /boot
└─vda2 252:2 0 399.5G0 part
├─centos-root 253:0 0 50G0 lvm/
├─centos-swap 253:1 0 3.9G0 lvm
└─centos-data 253:2 0 345.6G0 lvm/data
vdb 252:16 0 100G0 disk
├─cinder--volumes-volume--826e2ea7--4557--4dd5--b540--f213fbedcda4 253:3 0 5G0 lvm
└─cinder--volumes-volume--4feea69b--4e1f--4a06--a320--602842508d20 253:4 0 2G0 lvm
vdc 252:32 0 100G0 disk
#
安装并编辑配置文件
# yum install -y openstack-manila-share python2-PyMySQL
# cp /etc/manila/manila.conf{,.bak}
# vim /etc/manila/manila.conf
# grep -v ^# /etc/manila/manila.conf | tr -s [[:space:]]
rpc_backend = rabbit
default_share_type = default_share_type
rootwrap_config = /etc/manila/rootwrap.conf
auth_strategy = keystone
my_ip = 192.168.10.20
connection = mysql://manila:MANILA_DBPASS@controller/manila
memcached_servers = controller:11211
auth_uri = http://controller:5000
auth_url = http://controller:35357
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = manila
password = MANILA_PASS
lock_path = /var/lib/manila/tmp
rabbit_host = controller
rabbit_userid = openstack
rabbit_password = RABBIT_PASS
#
选择没有驱动的模式进行安装
# yum install -y lvm2 nfs-utils nfs4-acl-tools portmap
# systemctl enable lvm2-lvmetad.service
Created symlink from /etc/systemd/system/sysinit.target.wants/lvm2-lvmetad.service to /usr/lib/systemd/system/lvm2-lvmetad.service.
# systemctl start lvm2-lvmetad.service
# vim /etc/lvm/lvm.conf
# grep "^filter =" /etc/lvm/lvm.conf
filter = [ "a/vda/", "a/vdb/", "a/vdc/", "r/.*/"]
# pvcreate /dev/vdc
Physical volume "/dev/vdc" successfully created.
# vgcreate manila-volumes /dev/vdc
Volume group "manila-volumes" successfully created
# vim /etc/manila/manila.conf
添加如下内容
...
enabled_share_backends = lvm
enabled_share_protocols = NFS,CIFS
share_backend_name = LVM
share_driver = manila.share.drivers.lvm.LVMShareDriver
driver_handles_share_servers = False
lvm_share_volume_group = manila-volumes
lvm_share_export_ip = 192.168.10.20
启用并启动服务
# systemctl enable openstack-manila-share.service
Created symlink from /etc/systemd/system/multi-user.target.wants/openstack-manila-share.service to /usr/lib/systemd/system/openstack-manila-share.service.
# systemctl start openstack-manila-share.service
回到控制节点验证操作
# . admin-openrc
# manila service-list
+----+------------------+------------+------+---------+-------+----------------------------+
| Id | Binary | Host | Zone | Status| State | Updated_at |
+----+------------------+------------+------+---------+-------+----------------------------+
| 1| manila-scheduler | controller | nova | enabled | up | 2017-07-28T12:07:03.000000 |
| 2| manila-share | block@lvm| nova | enabled | up | 2017-07-28T12:07:02.000000 |
+----+------------------+------------+------+---------+-------+----------------------------+
#
下载共享服务器的源镜像
# wget http://tarballs.openstack.org/manila-image-elements/images/manila-service-image-master.qcow2
上传镜像到镜像服务里
# . admin-openrc
# openstack image create "manila-service-image" \
> --file manila-service-image-master.qcow2 \
> --disk-format qcow2 \
> --container-format bare \
> --public
+------------------+------------------------------------------------------+
| Field | Value |
+------------------+------------------------------------------------------+
| checksum | 2d26bfd3712f077e04f6cb91bf0e0fe6 |
| container_format | bare |
| created_at | 2017-07-29T05:04:39Z |
| disk_format | qcow2 |
| file | /v2/images/75301b1d-e22a-45b9-8372-72563ee29bf9/file |
| id | 75301b1d-e22a-45b9-8372-72563ee29bf9 |
| min_disk | 0 |
| min_ram | 0 |
| name | manila-service-image |
| owner | 9b07e2a368214247bb3051e806f94f9b |
| protected | False |
| schema | /v2/schemas/image |
| size | 337876992 |
| status | active |
| tags | |
| updated_at | 2017-07-29T05:04:44Z |
| virtual_size | None |
| visibility | public |
+------------------+------------------------------------------------------+
# openstack image list
+--------------------------------------+----------------------+--------+
| ID | Name | Status |
+--------------------------------------+----------------------+--------+
| 75301b1d-e22a-45b9-8372-72563ee29bf9 | manila-service-image | active |
| 9b0a7de0-6ff5-488b-9067-813e8a88de98 | cirros | active |
+--------------------------------------+----------------------+--------+
#
为支持共享文件系统的实例创建一个模版
# openstack flavor create manila-service-flavor --id 100 --ram 256 --disk 0 --vcpus 1
+----------------------------+-----------------------+
| Field | Value |
+----------------------------+-----------------------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral| 0 |
| disk | 0 |
| id | 100 |
| name | manila-service-flavor |
| os-flavor-access:is_public | True |
| ram | 256 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 1 |
+----------------------------+-----------------------+
# openstack flavor list
+-----+-----------------------+-------+------+-----------+-------+-----------+
| ID| Name | RAM | Disk | Ephemeral | VCPUs | Is Public |
+-----+-----------------------+-------+------+-----------+-------+-----------+
| 0 | m1.nano | 64 | 1 | 0 | 1 | True |
| 1 | m1.tiny | 512 | 1 | 0 | 1 | True |
| 100 | manila-service-flavor | 256 | 0 | 0 | 1 | True |
| 2 | m1.small |2048 | 20 | 0 | 1 | True |
| 3 | m1.medium |4096 | 40 | 0 | 2 | True |
| 4 | m1.large |8192 | 80 | 0 | 4 | True |
| 5 | m1.xlarge | 16384 |160 | 0 | 8 | True |
+-----+-----------------------+-------+------+-----------+-------+-----------+
#
无驱动模式创建共享
创建一个禁用DHSS的共享类型
# . admin-openrc
# manila type-create default_share_type False
+----------------------+--------------------------------------+
| Property | Value |
+----------------------+--------------------------------------+
| required_extra_specs | driver_handles_share_servers : False |
| Name | default_share_type |
| Visibility | public |
| is_default | - |
| ID | b07d7dc2-7259-434b-ad8e-7d95eaebffaf |
| optional_extra_specs | snapshot_support : True |
+----------------------+--------------------------------------+
# . demo-openrc
# manila create NFS 1 --name share1
+-----------------------------+--------------------------------------+
| Property | Value |
+-----------------------------+--------------------------------------+
| status | creating |
| share_type_name | default_share_type |
| description | None |
| availability_zone | None |
| share_network_id | None |
| host | |
| access_rules_status | active |
| snapshot_id | None |
| is_public | False |
| task_state | None |
| snapshot_support | True |
| id | 4e2a2bfe-9545-4c07-8706-1320c0a06821 |
| size | 1 |
| name | share1 |
| share_type | b07d7dc2-7259-434b-ad8e-7d95eaebffaf |
| has_replicas | False |
| replication_type | None |
| created_at | 2017-07-29T15:04:13.000000 |
| share_proto | NFS |
| consistency_group_id | None |
| source_cgsnapshot_member_id | None |
| project_id | 0200f6457da84abd9055a5c192386747 |
| metadata | {} |
+-----------------------------+--------------------------------------+
# manila list
+--------------------------------------+--------+------+-------------+--------+-----------+--------------------+---------------------------+-------------------+
| ID | Name | Size | Share Proto | Status | Is Public | Share Type Name | Host | Availability Zone |
+--------------------------------------+--------+------+-------------+--------+-----------+--------------------+---------------------------+-------------------+
| 4e2a2bfe-9545-4c07-8706-1320c0a06821 | share1 | 1 | NFS | error| False | default_share_type | block@lvm#lvm-single-pool | nova |
+--------------------------------------+--------+------+-------------+--------+-----------+--------------------+---------------------------+-------------------+
#
不完美的是本次实验做到这一步时共享创建状态一直为error，查证无果，所以共享服务没有给大家一个完美的结局，如果有配置成功的大神，千万记得留言反馈我一下，让我把这儿补充一下，本人甚是感谢。
页:
[1]