OpenStack Mitaka Neutron SR-IOV配置

Posted 2020-10-30 wanstack

### 一、在所有节点（控制节点、计算节点）
1、修改Bios

```
BOIS里面开启SR-IOV功能
开启 VT-d （inter virtualization technology）和 SR-IOV
```

2、修改系统配置

```
# cat /etc/default/grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed ‘s, release .*$,,g‘ /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="rd.lvm.lv=cl/root rd.lvm.lv=cl/swap rhgb quiet intel_iommu=on ixgbe.max_vfs=16"
GRUB_DISABLE_RECOVERY="true"
其中intel_iommu=on ixgbe.max_vfs=16是新添加的
```

```
grub2-mkconfig -o /boot/grub2/grub.cfg
```

3、安装配置软件包

```
yum -y install openstack-neutron-sriov-nic-agent.noarch

```

4、所有节点全部重启

为了保证控制节点集群稳定性，控制节点请一台一台重启

### 二、控制节点配置


```
# 增加sriov支持
vi /etc/neutron/plugins/ml2/ml2_conf.ini

mechanism_drivers = openvswitch,sriovnicswitch


# 查看网卡
lspci -nn | grep -i ethernet
[[email protected] ~]# lspci -nn | grep -i ethernet
04:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)
04:00.1 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521] (rev 01)...
08:12.6 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
08:13.0 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
08:13.2 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
08:13.4 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
08:13.6 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
82:00.0 Ethernet controller [0200]: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb] (rev 0...
82:10.0 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)
82:13.6 Ethernet controller [0200]: Intel Corporation 82599 Ethernet Controller Virtual Function [8086:10ed] (rev 01)


# 配置PCI
vi /etc/neutron/plugins/ml2/ml2_conf_sriov.ini

[ml2_sriov]
supported_pci_vendor_devs = 8086:10ed

# 增加sriov配置文件
vi /usr/lib/systemd/system/neutron-server.service

--config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugin.ini
--config-file /etc/neutron/plugins/ml2/ml2_conf_sriov.ini

# 增加调度算法支持sriov
vi /etc/nova/nova.conf

[DEFAULT]
scheduler_available_filters=nova.scheduler.filters.all_filters
scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter,CoreFilter, PciPassthroughFilter


# 绑定物理网卡
vi /etc/neutron/plugins/ml2/sriov_agent.ini
[sriov_nic]
physical_device_mappings = external:enp130s0


# 重启服务
pcs resource restart openstack-nova-consoleauth-clone
pcs resource restart neutron-server-clone


```


### 三、计算节点配置

```
# 计算节点开启SR-IOV agent
vi /etc/neutron/plugins/ml2/sriov_agent.ini

[securitygroup]
firewall_driver = neutron.agent.firewall.NoopFirewallDriver
[sriov_nic]
physical_device_mappings = external:enp130s0
exclude_devices =


# 计算节点开放白名单
vi /etc/nova/nova.conf

pci_passthrough_whitelist = [{"devname":"enp130s0","physical_network":"external"}]

# 重启服务
systemctl start neutron-sriov-nic-agent
systemctl enable neutron-sriov-nic-agent

systemctl restart openstack-nova-compute

```

### 四、测试

```
# 创建相关主机
neutron port-create <net-id> --binding:vnic-type direct 
#nova boot --flavor m1.large --image <image>--nic port-id=<port1> --nic port-id=<port2> <vm name>


ip link show p4p1

[[email protected] ~]# ip link show enp130s0
5: enp130s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT qlen 1000
    link/ether 00:e0:ed:2e:20:c6 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
    vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
    vf 2 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
    vf 3 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
    vf 4 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
    vf 5 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
    vf 6 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
    vf 7 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
    vf 8 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
    vf 9 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
    vf 10 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
    vf 11 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
    vf 12 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
    vf 13 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
    vf 14 MAC 00:00:00:00:00:00, spoof checking on, link-state auto
    vf 15 MAC fa:16:3e:73:2c:ee, vlan 46, spoof checking on, link-state auto
```