SR-IOV – The way to share real virtualized host devices with a virtual machine

Single Root I/O Virtualization or SR-IOV is a specification that allows the isolation of PCI Express (PCIe) resources. It allows a physical PCIe device or adapter to appear as one or more separate physical/virtual PCIe devices. These virtualized devices are also known as Virtual Function (VF) devices. Two main reasons or purposes for SR-IOV are: 1) to provide better virtualization network manageability and 2) to increase Virtual Machine (VM) performance. It is important to note that when you create multiple VF devices on a single physical adapter, all of the VF devices on that adapter are still sharing the same physical network port. If you have a multiport adapter, VF devices can be created on each adapter/port. The maximum number of VF devices that can be created on a single physical PCIe adapter is determined by the manufacturers’ hardware and driver design specifications.

Single Root I/O Virtualization and YES Certification

SR-IOV, using a network adapter, is part of the YES Certification test suite. It is a mandatory test on all virtualization servers, certified with SUSE Linux Enterprise Server 12 (the GA release) and newer. SR-IOV works well and is supported on SUSE Linux Enterprise Server 11, but validating the functionality during the YES Certification is optional. The sever manufacturer decides whether they want SR-IOV certified (and supported) on their system with SUSE Linux Enterprise Server 11.

All servers that are YES Certified with Xen or KVM, and thus tested with SR-IOV, have a configuration note on the certification bulletin that describes the network adapter and guest used during testing. Some server configurations do not provide support; the certification bulletins for these servers will include a configuration note stating that the current system configuration does not support SR-IOV. The best place to begin your search for YES Certified hardware is at https://www.suse.com/yessearch/. This is where you’ll find the most current YES Certification bulletins, including the Xen and KVM virtualization servers that are compatible with Single Root I/O Virtualization.

SR-IOV vs. PCI pass-through

Servers that do not support Single Root I/O Virtualization might still be able to pass through a network adapter to a VM guest if they support the legacy technology of PCI pass-through. One of the ways a host network adapter can be shared with a VM is to use PCI pass-through technology. PCI pass-through is an older technology and a predecessor technology to Single Root I/O Virtualization. With PCI pass-through, the physical network adapter (in the host) is completely dedicated to a single VM guest. The virtualized host adapter is configured to be exclusively assigned to one VM guest. SUSE Linux Enterprise still supports this legacy technology in addition to newer technologies. PCI pass-through is intended to increase a single VM guest’s network performance. The drawback is that the network adapter can no longer be shared among multiple VM guests. PCI pass-through can still be used to YES Certify a virtualization server, but new technology is the recommended method. Not all network adapters support SR-IOV. So, to reiterate, all YES Certification bulletins for Xen and KVM will have a configuration note stating one of the following: 1) PCI pass-through support, 2) SR-IOV support, or 3) SR-IOV/PCI pass-through not supported.

SR-IOV Implementation

Because SR-IOV is newer, it has become standard functionality for server-class network adapters/drivers on server-class systems. The firmware in a server must also provide SR-IOV functionality in order for it to be supported on that server. Some network adapters provide two drivers: one for the physical adapter and a second “add-on” driver for SR-IOV functionality. An example of this is the Intel® i40e (which is the base network driver) and the Intel i40evf (which is the Single Root I/O Virtualization network driver). Both of these Intel network drivers install simultaneously and work together on the same network adapter with SUSE Linux Enterprise. As virtualization and cloud-based virtualization technologies become more prevalent, SR-IOV has the potential to become a vital and integral part of the network infrastructure.

SR-IOV Limitation

Most SR-IOV network implementations have a minor limitation: the VF device that is created and the driver used for that VF device do not provide the same feature set as that of the physical adapter and driver. VF devices and drivers will typically only provide a subset of high-level networking features. This means that with a VF device attached to your guest, the network connection in the guest might not have the same advanced capabilities available on the physical adapter.

The Future

So, what does the future hold for SR-IOV? What is on the horizon that will extend SR-IOV beyond its current functionality? The answer to both is Multi Root I/O Virtualization, or MR-IOV. The idea behind MR-IOV is that it will build upon SR-IOV technology by sharing VF devices across multiple network servers or cloud servers. MR-IOV will allow the VF devices in one server to be used in another server on the network. SR-IOV is limited to a single server or host configuration. At this point, I am aware of only a small number of very high-end configuration designs utilizing MR-IOV functions today.

We hope YES Certification, YES bulletins and the valuable data on these bulletins, like SR-IOV, help you make better decisions when buying new servers. Our goal is to support you early in the process so that you have fewer hardware-related headaches later as you purchase servers (and workstations).

You can find more information about SUSE YES Certification at SUSE YES Program. As mentioned above, you can search for YES Certified hardware at YES Search. Finally, you can review all of the YES Certification blogs at YES Certification blog posts.