G023-OP-INS-RHEL-03 RedHat OpenStack Cinder 对接 NFS 后端存储

Posted 2022-06-17 CloudCS

1 您需要了解

• 通过 Packstack 部署的 OpenStack Block Storage 服务使用的后端存储默认为 LVM（应答文件参数 CONFIG_CINDER_BACKEND=lvm）
• 所有节点关闭 SELinux，如控制节点、计算节点开启 SELinux，并作为客户端访问 NFS 卷，则需要在这些节点上启用 virt_use_nfs 布尔值。执行命令 setsebool -P virt_use_nfs on
• NFS 服务端 OS ：CentOS 7.8
• 您可参考下列相关文章 G019 和 G020 完成 RedHat OpenStack 环境搭建及基础操作

---

• 相关文章
• G019-OP-INS-RHEL-01 PackStack 安装 RedHat OpenStack
• G020-OP-INS-RHEL-02 RedHat OpenStack 发放云主机（命令行）

2 配置NFS服务

2.1 安装 NFS

• 查询是否已安装

[root@cloudcs ~]# rpm -qa |grep nfs
nfs-utils-1.3.0-0.66.el7.x86_64

• 如没有安装，执行以下命令

[root@cloudcs ~]# yum install -y nfs-utils

2.2 启动 NFS 服务

[root@cloudcs ~]# systemctl start nfs-server.service 
[root@cloudcs ~]# systemctl enable nfs-server.service 
[root@cloudcs ~]# systemctl status nfs-server.service 

2.3 关闭防火墙及SELinux

[root@cloudcs ~]# setenforce 0
[root@cloudcs ~]# systemctl stop firewalld.service 
[root@cloudcs ~]# systemctl disable firewalld.service 
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@cloudcs ~]# vim /etc/selinux/config 
[root@cloudcs ~]# cat /etc/selinux/config 

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled

2.4 创建共享目录

[root@cloudcs ~]# mkdir /nfs_share
[root@cloudcs ~]# mount /dev/sdb1 /nfs_share/
[root@cloudcs ~]# df -Th
Filesystem     Type      Size  Used Avail Use% Mounted on
devtmpfs       devtmpfs  3.9G     0  3.9G   0% /dev
tmpfs          tmpfs     3.9G     0  3.9G   0% /dev/shm
tmpfs          tmpfs     3.9G   13M  3.9G   1% /run
tmpfs          tmpfs     3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/sda3      xfs        52G  4.5G   48G   9% /
/dev/sda1      xfs       497M  162M  336M  33% /boot
tmpfs          tmpfs     797M   20K  797M   1% /run/user/0
/dev/sr0       iso9660   4.5G  4.5G     0 100% /mnt
/dev/sdb1      xfs       100G   33M  100G   1% /nfs_share

[root@cloudcs ~]# chmod 777 -R /nfs_share/

2.5 修改配置文件

[root@cloudcs ~]# vim /etc/exports
[root@cloudcs ~]# cat /etc/exports
/nfs_share 192.168.45.0/24(rw)

2.6 重启服务

[root@cloudcs ~]# systemctl restart nfs-server.service 

2.7 客户端测试挂载

• 使用控制节点测试挂载

[root@controller ~]# rpm -qa |grep nfs
libnfsidmap-0.25-11.el7.x86_64
nfs-utils-1.3.0-0.8.el7.x86_64
[root@controller ~]# systemctl start nfs
[root@controller ~]# mkdir /nfs_test
[root@controller ~]# mount -t nfs 192.168.45.142:/nfs_share /nfs_test/
[root@controller ~]# df -Th
Filesystem                Type      Size  Used Avail Use% Mounted on
/dev/sda3                 xfs        92G  5.1G   87G   6% /
devtmpfs                  devtmpfs  7.9G     0  7.9G   0% /dev
tmpfs                     tmpfs     7.9G  144K  7.9G   1% /dev/shm
tmpfs                     tmpfs     7.9G   17M  7.9G   1% /run
tmpfs                     tmpfs     7.9G     0  7.9G   0% /sys/fs/cgroup
/dev/loop0                ext4      1.9G  6.1M  1.7G   1% /srv/node/swiftloopback
/dev/sda1                 xfs       497M  106M  391M  22% /boot
tmpfs                     tmpfs     7.9G   17M  7.9G   1% /run/netns
192.168.45.142:/nfs_share nfs4      100G   33M  100G   1% /nfs_test
[root@controller ~]# cd /nfs_test/
[root@controller nfs_test]# touch a,b1,2.txt
[root@controller nfs_test]# ls
a1.txt  a2.txt  b1.txt  b2.txt

[root@controller ~]# umount /nfs_test/

3 控制节点配置NFS共享

3.1 创建文件并配置

[root@controller ~]# vim /etc/cinder/nfs_share
[root@controller ~]# cat /etc/cinder/nfs_share 
192.168.45.142:/nfs_share

3.2 修改文件权限

[root@controller ~]# chown root:cinder /etc/cinder/nfs_share
[root@controller ~]# chmod 0640 /etc/cinder/nfs_share
[root@controller ~]# ll /etc/cinder/
total 96
-rw-------. 1 cinder cinder  2117 May  7 11:49 api-paste.ini
-rw-------. 1 cinder cinder 79364 May  7 11:49 cinder.conf
-rw-r-----  1 root   cinder    26 Jun 16 12:16 nfs_share
-rw-r-----. 1 root   cinder  3200 Feb  6  2015 policy.json
-rw-r-----. 1 root   cinder   942 Feb  6  2015 rootwrap.conf
drwxr-xr-x. 2 cinder root       6 Feb  7  2015 volumes

4 创建新的后端定义

4.1 增加 NFS 后端参数

• 增加 nfs 后端参数，其中 lvm 默认存在，nfs 为新增

[root@controller ~]# vim /etc/cinder/cinder.conf
[root@controller ~]# cat /etc/cinder/cinder.conf

[lvm]
iscsi_helper=lioadm
volume_group=cinder-volumes
iscsi_ip_address=192.168.45.150
volume_driver=cinder.volume.drivers.lvm.LVMISCSIDriver
volume_backend_name=lvm

[nfs] 
nfs_shares_config = /etc/cinder/nfs_share 
volume_driver = cinder.volume.drivers.nfs.NfsDriver 
volume_backend_name = nfsbackend

4.2 增加 NFS 后端定义

• 修改参数 enabled_backends

enabled_backends=lvm,nfs

4.3 重启卷服务

[root@controller ~]# source keystonerc_admin
[root@controller cinder(keystone_admin)]# systemctl restart openstack-cinder-api  openstack-cinder-scheduler  openstack-cinder-volume

5 创建云硬盘类型

• 创建云硬盘类型

[root@controller ~(keystone_admin)]# cinder type-create nfstype
+--------------------------------------+---------+
|                  ID                  |   Name  |
+--------------------------------------+---------+
| 8a2c71c6-3f28-4504-8441-51c128ae2ed1 | nfstype |
+--------------------------------------+---------+

[root@controller ~(keystone_admin)]# cinder type-list 
+--------------------------------------+---------+
|                  ID                  |   Name  |
+--------------------------------------+---------+
| 2f222c5f-3a8b-48f4-a0b1-75992722ef6d |  iscsi  |
| 8a2c71c6-3f28-4504-8441-51c128ae2ed1 | nfstype |
+--------------------------------------+---------+

• 设置后端名称

[root@controller ~(keystone_admin)]# cinder type-key nfstype set volume_backend_name=nfsbackend

6 创建云硬盘

• 使用 NFS 类型创建云硬盘

[root@controller ~(keystone_admin)]# source keystonerc_cloudcs
[root@controller cinder(keystone_cloudcs)]# cinder create --display-name evs001 --volume-type nfstype 2 
+---------------------+--------------------------------------+
|       Property      |                Value                 |
+---------------------+--------------------------------------+
|     attachments     |                  []                  |
|  availability_zone  |                 nova                 |
|       bootable      |                false                 |
|      created_at     |      2022-06-16T05:06:33.321538      |
| display_description |                 None                 |
|     display_name    |                evs001                |
|      encrypted      |                False                 |
|          id         | b0932857-3ef4-433d-94ad-d5ad10f0aa32 |
|       metadata      |                                    |
|         size        |                  2                   |
|     snapshot_id     |                 None                 |
|     source_volid    |                 None                 |
|        status       |               creating               |
|     volume_type     |               nfstype                |
+---------------------+--------------------------------------+

• 尝试挂载给主机

[root@controller cinder(keystone_cloudcs)]# cinder list
+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
|                  ID                  |   Status  | Display Name | Size | Volume Type | Bootable | Attached to |
+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+
| b0932857-3ef4-433d-94ad-d5ad10f0aa32 | available |    evs001    |  2   |   nfstype   |  false   |             |
+--------------------------------------+-----------+--------------+------+-------------+----------+-------------+

[root@controller cinder(keystone_cloudcs)]# nova list
+--------------------------------------+-------+--------+------------+-------------+------------------------+
| ID                                   | Name  | Status | Task State | Power State | Networks               |
+--------------------------------------+-------+--------+------------+-------------+------------------------+
| ade7bdfd-6cba-494e-8da8-780c5ff0cb85 | ecs01 | ACTIVE | -          | Running     | private=192.168.100.10 |
+--------------------------------------+-------+--------+------------+-------------+------------------------+

[root@controller cinder(keystone_cloudcs)]# nova volume-attach ecs01 b0932857-3ef4-433d-94ad-d5ad10f0aa32
+----------+--------------------------------------+
| Property | Value                                |
+----------+--------------------------------------+
| device   | /dev/vdb                             |
| id       | b0932857-3ef4-433d-94ad-d5ad10f0aa32 |
| serverId | ade7bdfd-6cba-494e-8da8-780c5ff0cb85 |
| volumeId | b0932857-3ef4-433d-94ad-d5ad10f0aa32 |
+----------+--------------------------------------+

---

• END