This article describes how to use single root I/O virtualization (SR-IOV), which enables configuration of a single physical network port to provide virtual functions (VFs) to a set of virtual machines (VMs). Depending on the Ethernet controller, you can create 63 or more VFs per physical port. Then with network data transferred directly between port and VM, bypassing the hypervisor network stack (and switch layer), traffic on the bus and processor interrupts are significantly reduced.
SR-IOV is available on a variety of Ethernet controllers across multiple operating systems (both hypervisor and guest OS). This article covers the basic steps to create VFs using SR-IOV on Fedora* 4.0.3.
Basic Steps to Configure SR-IOV
In this section, we walk through the basic steps required to configure SR-IOV.
- Confirm that your Ethernet controller, hypervisor, and guest OS are supported by checking the FAQ for Intel® Ethernet Server Adapters with SR-IOV.
Confirm iommu (or Intel VT-d) is enabled in the BIOS.
Note: iommu enables mapping of virtual memory addresses to physical addresses.
- In the grub.conf file, turn on iommu and set it to passthrough mode:
- Be sure to update grub file, then reboot. The grub file gets updated automatically if using you’re using a desktop system, however If you’re using a server system, you must enter the update-grub command before rebooting system.
- Verify the setup by typing the steps below:
mesg | grep Virtualization ##shows if VT-d is enabled cat /proc/cmdline ##checks that iommu parameters were passed cd /root/DPDK/dpdk-16.07 ##checks for DPDK and moves to next step tools/dpdk-devbind.py --st ##checks network devices status
- Load the driver
modprobe uio insmod igb_uio.ko
- Set up the virtual functions (VFs) - in this case on device 04, first 2 ports:
echo 1 > /sys/bus/pci/devices/0000\:04\:00.0/SR-IOV_numvfs echo 1 > /sys/bus/pci/devices/0000\:04\:00.1/SR-IOV_numvfs
- Confirm the VF configuration
lspci | grep Eth OR ./dpdk-devbind.py --st cat /sys/bus/pci/devices/0000\:04\:00.0/SR-IOV_numvfs cat /sys/bus/pci/devices/0000\:04\:00.1/SR-IOV_numvfs
Note: In these steps we created one VF for each port of a physical NIC. You can create many VFs depending upon your requirements and the capacity of your physical NIC. For example, if you're using the Intel 82599 10 Gigabit Ethernet NIC, you can enable maximum of 63 Virtual Functions per port. Find your NIC specification at http:/ark.intel.com. Look at the product brief for information about number of supported VFs.
The video below is a companion to this article.
This article provided a definition of SR-IOV, and provided the steps required to configure a system to share one physical NIC among multiple VMs.
About the Author
Nancy Yadav is a Software Dataplane (Platform) Application Engineer at Intel. She has worked on software development, algorithm optimization, and now helping customers to optimize network functions for SDN & NFV.