创建基于SRIOV的KVM虚拟机
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了创建基于SRIOV的KVM虚拟机相关的知识，希望对你有一定的参考价值。
参考技术A 本文来自微信公众号“ICT大融合”。一、利用 iso 创建虚机
1. 安装环境：
适用于centos/redhat.
2. 安装准备：
yum -y install kvm
yum -y install qemu-kvmpython-virtinst libvirt libvirt-python virt-manager libguestfs-toolsbridge-utils virt-install
systemctl startlibvirtd.service
systemctl enablelibvirtd.service
3. 构建虚拟网络：
brctl addbr br0
brctl addif br0 ens6np0
注：ens6np0是netreonome CX-25G智能网卡，
[root@test4 vms]# ethtool -i ens6np0
driver: nfp
version: rev-2020.01.13.0820.a3cf8e9 (o-
firmware-version: 0.0.3.5 0.30 sriov-2.1.16.1nic
expansion-rom-version:
bus-info: 0000:02:00.0
4. 安装虚拟机
mkdir -p /home/vms/
virt-install --virt-type=kvm --name=centos88 --vcpus=2--memory=4096 --location=/home/CentOS-7-x86_64-DVD-1908.iso --diskpath=/home/vms/centos88.qcow2,size=40,format=qcow2 --network bridge=br0--graphics none --extra-args='console=ttyS0' –force
注：
安装镜像iso一定放在/home/或其他可访问目录，否则提示没有操作权限。
然后一路按照提示，设置相应参数，直到虚拟机构建完成。
安装完成之后，用virshlish查看虚拟机状态。
[root@test4 vms]# virsh list
Id Name State
----------------------------------------------------
2 centos88 running
以上是虚拟机的网卡是基于virtio并接入主机的bridge，用ps可以看到相应进程。
[root@test4 ~]# ps -e |grep kvm
15484 ? 00:00:38 qemu-kvm
15491 ? 00:00:00 kvm-pit/15484
18406 ? 00:00:14 qemu-kvm
18413 ? 00:00:00 kvm-pit/18406
[root@test4 ~]# ps -e |grep vhost
15486 ? 00:00:00 vhost-15484
18408 ? 00:00:00 vhost-18406
二、用通过镜像创建基于 SRIOV 虚拟机。
1. 创建vf ：
echo 2 > /sys/class/net/ens6np0/device/sriov_numvfs
可以用如下命令查看，
lspci -kd 19ee:
lspci | grep -i ethernet
注意：19ee是netreonome的PCI vendor ID号。
2. 创建虚拟机镜像文件：
cp centos88.qcow2 centos7.200.qcow2
chown qemu:qemu centos7.200.qcow2
cp centos88.xml centos7.200.xml
3. 修改配置文件
vi centos7.200.xml
其中，uuid采用uuidgen 命令生成。
选项Interface进行修改，Mac地址随便改一个，不重复就行。关键是修改vf的PCI地址，用lspci获取。
virsh define centos7.200.xml
4. 创建磁盘：
qemu-img create /home/vms/centos200.img 40G
5. 启动虚机：
virsh start centos200
三、常见命令
virsh基本命令：
virsh list # 查看在运行的虚拟机
virsh dumpxml vm-name # 查看kvm虚拟机配置文件
virsh start vm-name # 启动kvm虚拟机
virsh shutdown vm-name # 正常关机
virsh destroy vm-name # 非正常关机，强制关闭虚拟机（相当于物理机直接拔掉电源）
virsh undefine vm-name # 删除vm的配置文件
ls /etc/libvirt/qemu
# 查看删除结果，Centos-6.6的配置文件被删除，但磁盘文件不会被删除
virsh define file-name.xml # 根据配置文件定义虚拟机
virsh suspend vm-name # 挂起，终止
virsh resumed vm-name # 恢复被挂起的虚拟机
virsh autostart vm-name # 开机自启动vm
virsh console <虚拟机名称> # 连接虚拟机
virt-install参数：
–name指定虚拟机名称
–memory分配内存大小。
–vcpus分配CPU核心数，最大与实体机CPU核心数相同
–disk指定虚拟机镜像，size指定分配大小单位为G。
–network网络类型，此处用的是默认，一般用的应该是bridge桥接。
–accelerate加速
–cdrom指定安装镜像iso
–vnc启用VNC远程管理，一般安装系统都要启用。
–vncport指定VNC监控端口，默认端口为5900，端口不能重复。
–vnclisten指定VNC绑定IP，默认绑定127.0.0.1，这里改为0.0.0.0。
–os-type=linux,windows
–os-variant=rhel6
--name 指定虚拟机名称
--ram 虚拟机内存大小，以 MB 为单位
--vcpus 分配CPU核心数，最大与实体机CPU核心数相同
–-vnc 启用VNC远程管理，一般安装系统都要启用。
–-vncport 指定VNC监控端口，默认端口为5900，端口不能重复。
–-vnclisten 指定VNC绑定IP，默认绑定127.0.0.1，这里改为0.0.0.0。
--network 虚拟机网络配置
#其中子选项，bridge=br0 指定桥接网卡的名称。
–os-type=linux,windows
–os-variant=rhel7.2
--disk 指定虚拟机的磁盘存储位置
#size，初始磁盘大小，以 GB 为单位。
--location 指定安装介质路径，如光盘镜像的文件路径。
--graphics 图形化显示配置
#全新安装虚拟机过程中可能会有很多交互操作，比如设置语言，初始化 root 密码等等。
--extra-args 根据不同的安装方式设置不同的额外选项
42-KVM虚拟化-基于现有虚拟机磁盘为模版创建新的虚拟机
基于现有虚拟机磁盘为模版创建新的虚拟机
使用 virt-install 创建虚拟机
- 虽然使用virt-manager 可以方便的管理虚拟机,但如果需要批量进行虚拟机的创建管理,命令行工具virtinstall更加方便和适合
案例：
利用 qemu-img命令创建虚拟磁盘
#注意: qemu-img create 一定要确认对应路径下没有此文件,如果存在将覆盖原文件
1. 确认现有的虚拟磁盘文件
[root@ubuntu2204 ~]#ll /var/lib/libvirt/images/centos7-mooreyxia.qcow2 -h
-rw------- 1 libvirt-qemu kvm 11G Dec 30 22:03 /var/lib/libvirt/images/centos7-mooreyxia.qcow2
2. 创建新的虚拟磁盘文件
[root@ubuntu2204 ~]#qemu-img create -f qcow2 /var/lib/libvirt/images/centos7-mooreyxia2.qcow2 20G
Formatting /var/lib/libvirt/images/centos7-mooreyxia2.qcow2, fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=21474836480 lazy_refcounts=off refcount_bits=16
3. 查看虚拟磁盘文件占用空间
[root@ubuntu2204 ~]#ll /var/lib/libvirt/images/centos7-mooreyxia* -h
-rw-r--r-- 1 root root 193K Dec 30 22:05 /var/lib/libvirt/images/centos7-mooreyxia2.qcow2
-rw------- 1 libvirt-qemu kvm 11G Dec 30 22:03 /var/lib/libvirt/images/centos7-mooreyxia.qcow2
#发现虚拟磁盘占用的空间并不大只有193k,其实是一种稀疏格式的文件。类似下面这种：
[root@ubuntu2204 ~]#dd if=/dev/zero of=f1.img bs=1G count=0 seek=1
0+0 records in
0+0 records out
0 bytes copied, 0.000133668 s, 0.0 kB/s
[root@ubuntu2204 ~]#ll f1.img -h
-rw-r--r-- 1 root root 1.0G Dec 30 22:08 f1.img
#占用实际空间也就一个字节
[root@ubuntu2204 ~]#du -sh f1.img
0 f1.img
使用虚拟磁盘创建新的虚拟机模板
#利用 osinfo-query命令查看支持的OS版本
[root@ubuntu2204 ~]#apt install -y libosinfo-bin
[root@ubuntu2204 images]#osinfo-query os|grep -i rocky
rocky-unknown | Rocky Linux Unknown | unknown | http://rockylinux.org/rocky/unknown
rocky8-unknown | Rocky Linux 8 Unknown | 8-unknown | http://rockylinux.org/rocky/8-unknown
rocky8.4 | Rocky Linux 8.4 | 8.4 | http://rockylinux.org/rocky/8.4
rocky8.5 | Rocky Linux 8.5 | 8.5 | http://rockylinux.org/rocky/8.5
rocky8.6 | Rocky Linux 8.6 | 8.6 | http://rockylinux.org/rocky/8.6
rocky9-unknown | Rocky Linux 9 Unknown | 9-unknown | http://rockylinux.org/rocky/9-unknown
rocky9.0 | Rocky Linux 9.0 | 9.0 | http://rockylinux.org/rocky/9.0
#利用复制的qcow2虚拟磁盘创建虚拟机
[root@ubuntu2204 ~]#ll /var/lib/libvirt/images/*
-rw-r--r-- 1 root root 196928 Dec 30 22:05 /var/lib/libvirt/images/centos7-mooreyxia2.qcow2
-rw------- 1 libvirt-qemu kvm 10739318784 Dec 30 22:22 /var/lib/libvirt/images/centos7-mooreyxia.qcow2
[root@ubuntu2204 ~]#cd /var/lib/libvirt/images/
[root@ubuntu2204 images]#mv centos7-mooreyxia2.qcow2 rocky8.qcow2
#查看虚拟硬盘大小,注意到只要正在运行的虚拟对应的硬盘文件所有者和组为qemu,而虚拟机关机的为root
[root@ubuntu2204 images]#ll
total 1676904
drwx--x--x 2 root root 4096 Dec 30 22:28 ./
drwxr-xr-x 7 root root 4096 Dec 29 23:01 ../
-rw------- 1 libvirt-qemu kvm 10739318784 Dec 30 22:22 centos7-mooreyxia.qcow2
-rw-r--r-- 1 root root 196928 Dec 30 22:05 rocky8.qcow2
[root@ubuntu2204 images]#ls /data/isos/
CentOS-7-x86_64-Minimal-2207-02.iso Rocky-x86_64-minimal.iso
#创建默认NAT模式的虚拟机,并不自动打开virt-viewer连接console,需要手动打开virt-manager 连接,并手动安装系统
[root@ubuntu2204 images]#virt-install --virt-type kvm --name rocky8 --ram 2048 --vcpus 2 --cdrom=/data/isos/Rocky-x86_64-minimal.iso --disk path=/var/lib/libvirt/images/rocky8.qcow2 --network network=default --graphics vnc,listen=0.0.0.0 --noautoconsole --os-variant=rocky8.6
Starting install...
Creating domain... | 0 B 00:00:00
Domain is still running. Installation may be in progress.
You can reconnect to the console to complete the installation process.
[root@ubuntu2204 images]#virsh list
Id Name State
-----------------------------------
1 centos7-mooreyxia running
2 rocky8 running
#一个虚拟机表现为一个进程
[root@ubuntu2204 images]#ps aux
...
libvirt+ 28601 2.2 3.2 3799436 537036 ? Sl 21:45 1:36 /usr/bin/qemu-system-x86_64 -name guest=centos7-mooreyxia,debug-threads=on -S -object "qom-type":"secret","id":
...
libvirt+ 28950 60.9 13.2 4549116 2164032 ? Sl 22:44 7:11 /usr/bin/qemu-system-x86_64 -name guest=rocky8,debug-threads=on -S -object "qom-type":"secret","id":"masterKey0
root 28956 0.0 0.0 0 0 ? S 22:44 0:00 [kvm-nx-lpage-re]
root 28957 0.0 0.0 0 0 ? S 22:44 0:00 [vhost-28950]
[root@ubuntu2204 images]#apt install -y psmisc
...
├─qemu-system-x86(28601)─┬─qemu-system-x86(28605)
│ ├─qemu-system-x86(28609)
│ ├─qemu-system-x86(28610)
│ ├─qemu-system-x86(28611)
│ └─qemu-system-x86(28613)
├─qemu-system-x86(28950)─┬─qemu-system-x86(28954)
│ ├─qemu-system-x86(28958)
│ ├─qemu-system-x86(28960)
│ ├─qemu-system-x86(28961)
│ ├─qemu-system-x86(28963)
│ ├─qemu-system-x86(29040)
│ └─qemu-system-x86(29075)
#宿主机远程连接
[root@ubuntu2204 ~]#ssh 192.168.122.14
The authenticity of host 192.168.122.14 (192.168.122.14) cant be established.
ED25519 key fingerprint is SHA256:zIUbzb+J5KknMgrYqJpK/IgOfCfnojkR2mZoTm74j18.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 192.168.122.14 (ED25519) to the list of known hosts.
root@192.168.122.14s password:
Last login: Fri Dec 30 23:09:20 2022
[root@Rocky8 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:54:00:b4:c5:4b brd ff:ff:ff:ff:ff:ff
inet 192.168.122.14/24 brd 192.168.122.255 scope global dynamic noprefixroute enp1s0
valid_lft 3427sec preferred_lft 3427sec
inet6 fe80::5054:ff:feb4:c54b/64 scope link noprefixroute
valid_lft forever preferred_lft forever
#系统初始化
[root@Rocky8 ~]# getenforce
Enforcing
[root@Rocky8 ~]# vi /etc/selinux/config
[root@Rocky8 ~]# cat /etc/selinux/config
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these three values:
# targeted - Targeted processes are protected,
# minimum - Modification of targeted policy. Only selected processes are protected.
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
[root@Rocky8 ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2022-12-30 23:08:14 CST; 7min ago
Docs: man:firewalld(1)
Main PID: 904 (firewalld)
Tasks: 2 (limit: 11352)
Memory: 38.1M
CGroup: /system.slice/firewalld.service
└─904 /usr/libexec/platform-python -s /usr/sbin/firewalld --nofork --nopid
[root@Rocky8 ~]# systemctl disable --now firewalld
Removed /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@Rocky8 ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
Active: inactive (dead)
Docs: man:firewalld(1)
Dec 30 23:08:13 localhost.localdomain systemd[1]: Starting firewalld - dynamic firewall daemon...
Dec 30 23:08:14 localhost.localdomain systemd[1]: Started firewalld - dynamic firewall daemon.
Dec 30 23:08:14 localhost.localdomain firewalld[904]: WARNING: AllowZoneDrifting is enabled. This is considered an insecure configuration option. It will be removed in a future r>
Dec 30 23:15:31 Rocky8 systemd[1]: Stopping firewalld - dynamic firewall daemon...
Dec 30 23:15:31 Rocky8 systemd[1]: firewalld.service: Succeeded.
Dec 30 23:15:31 Rocky8 systemd[1]: Stopped firewalld - dynamic firewall daemon.
[root@Rocky8 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 52:54:00:b4:c5:4b brd ff:ff:ff:ff:ff:ff
inet 192.168.122.14/24 brd 192.168.122.255 scope global dynamic noprefixroute enp1s0
valid_lft 2999sec preferred_lft 2999sec
inet6 fe80::5054:ff:feb4:c54b/64 scope link noprefixroute
valid_lft forever preferred_lft forever
[root@Rocky8 ~]# vi /etc/default/grub
[root@Rocky8 ~]# cat /etc/default/grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed s, release .*$,,g /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="crashkernel=auto resume=/dev/mapper/rl-swap rd.lvm.lv=rl/root rd.lvm.lv=rl/swap net.ifnames=0"
GRUB_DISABLE_RECOVERY="true"
GRUB_ENABLE_BLSCFG=true
[root@Rocky8 ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
done
[root@Rocky8 ~]# cd /etc/yum.repos.d/
[root@Rocky8 yum.repos.d]# ls
Rocky-AppStream.repo Rocky-Debuginfo.repo Rocky-Extras.repo Rocky-Media.repo Rocky-Plus.repo Rocky-ResilientStorage.repo Rocky-Sources.repo
Rocky-BaseOS.repo Rocky-Devel.repo Rocky-HighAvailability.repo Rocky-NFV.repo Rocky-PowerTools.repo Rocky-RT.repo
[root@Rocky8 yum.repos.d]# mkdir -pv bak
mkdir: created directory bak
[root@Rocky8 yum.repos.d]# mv *.repo bak/
[root@Rocky8 yum.repos.d]# ls
bak
#从远程备份拉取yum仓配置
[root@rocky8 ~]#cat /etc/yum.repos.d/base.repo
[BaseOS]
name=BaseOS
baseurl=https://mirrors.aliyun.com/rockylinux/$releasever/BaseOS/x86_64/os/
http://mirrors.163.com/rocky/$releasever/BaseOS/x86_64/os/
https://mirrors.nju.edu.cn/rocky/$releasever/BaseOS/x86_64/os/
https://mirrors.sjtug.sjtu.edu.cn/rocky/$releasever/BaseOS/x86_64/os/
http://mirrors.sdu.edu.cn/rocky/$releasever/BaseOS/x86_64/os/
gpgcheck=0
[AppStream]
name=AppStream
baseurl=https://mirrors.aliyun.com/rockylinux/$releasever/AppStream/x86_64/os/
http://mirrors.163.com/rocky/$releasever/AppStream/x86_64/os/
https://mirrors.nju.edu.cn/rocky/$releasever/AppStream/x86_64/os/
https://mirrors.sjtug.sjtu.edu.cn/rocky/$releasever/AppStream/x86_64/os/
http://mirrors.sdu.edu.cn/rocky/$releasever/AppStream/x86_64/os/
gpgcheck=0
[extras]
name=extras
baseurl=https://mirrors.aliyun.com/rockylinux/$releasever/extras/$basearch/os
http://mirrors.163.com/rocky/$releasever/extras/$basearch/os
https://mirrors.nju.edu.cn/rocky/$releasever/extras/$basearch/os
https://mirrors.sjtug.sjtu.edu.cn/rocky/$releasever/extras/$basearch/os
http://mirrors.sdu.edu.cn/rocky/$releasever/extras/$basearch/os
gpgcheck=0
enabled=1
[PowerTools]
name=CentOS-$releasever - PowerTools
baseurl=https://mirrors.aliyun.com/rockylinux/$releasever/PowerTools/$basearch/os/
http://mirrors.163.com/rocky/$releasever/PowerTools/$basearch/os/
http://mirrors.sdu.edu.cn/rocky/$releasever/PowerTools/$basearch/os/
https://mirrors.sjtug.sjtu.edu.cn/rocky/$releasever/PowerTools/$basearch/os/
http://mirrors.sdu.edu.cn/rocky/$releasever/PowerTools/$basearch/os/
gpgcheck=0
enabled=0
[epel]
name=EPEL
baseurl=https://mirror.tuna.tsinghua.edu.cn/epel/$releasever/Everything/$basearch
https://mirrors.cloud.tencent.com/epel/$releasever/Everything/$basearch
https://mirrors.huaweicloud.com/epel/$releasever/Everything/$basearch
https://mirrors.aliyun.com/epel/$releasever/Everything/$basearch
gpgcheck=0
enabled=1
#cp到宿主机
[root@rocky8 ~]#scp /etc/yum.repos.d/base.repo 10.0.0.200:
root@10.0.0.200s password:
base.repo
#再cp到虚拟机
[root@Rocky8 yum.repos.d]# scp 10.0.0.200:/root/base.repo .
The authenticity of host 10.0.0.200 (10.0.0.200) cant be established.
ECDSA key fingerprint is SHA256:tnqFcM0vAKUTFBp39MI8wNU3P0OgZphFCM4/jXbcSDM.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 10.0.0.200 (ECDSA) to the list of known hosts.
root@10.0.0.200s password:
base.repo 100% 2062 975.1KB/s 00:00
[root@Rocky8 yum.repos.d]# ll
total 8
drwxr-xr-x. 2 root root 4096 Dec 30 23:21 bak
-rw-r--r--. 1 root root 2062 Dec 30 23:30 base.repo
[root@Rocky8 yum.repos.d]# yum update;yum install -y wget vim lrzsz net-tools
...
Installed:
gpm-libs-1.20.7-17.el8.x86_64 libmetalink-0.1.3-7.el8.x86_64 lrzsz-0.12.20-43.el8.x86_64 net-tools-2.0-0.52.20160912git.el8.x86_64
vim-common-2:8.0.1763-19.el8_6.4.x86_64 vim-enhanced-2:8.0.1763-19.el8_6.4.x86_64 vim-filesystem-2:8.0.1763-19.el8_6.4.noarch wget-1.19.5-10.el8.x86_64
Complete!
#至此Rocky8.6虚拟机模板完成
[root@Rocky8 yum.repos.d]# reboot
Connection to 192.168.122.14 closed by remote host.
Connection to 192.168.122.14 closed.
利用virt-clone克隆新的虚拟机
#基于已有的虚拟机克隆生成新的虚拟机
[root@ubuntu2204 images]#virsh list --all
Id Name State
------------------------------------
1 centos7-mooreyxia running
- rocky8 shut off
[root@ubuntu2204 images]#virt-clone -o rocky8 -f /var/lib/libvirt/images/rocky8-template.qcow2 -n rocky8-template
Allocating rocky8-template.qcow2 | 3.5 GB 00:10:29 ...
Clone rocky8-template created successfully.
-o rocky8 #指已存在的虚拟机的名称
-f /var/lib/libvirt/images/rocky8-3.qcow2 #新虚拟机磁盘文件路径，此文件自动生成，不需要
事先创建
-n rocky8-3 #新虚拟机的名称
[root@ubuntu2204 images]#virsh list --all
Id Name State
------------------------------------
- centos7-mooreyxia shut off
- rocky8 shut off
- rocky8-template shut off
至此，就可以在裸机服务器上安装Linux，再KVM虚拟化安装任意操作系统的虚拟机了。
我是moore,大家一起加油！
以上是关于创建基于SRIOV的KVM虚拟机的主要内容，如果未能解决你的问题，请参考以下文章