The virtio-nvgpu project develops a virtio device that allows access to NVIDIA GPUs from guest systems running under the KVM hypervisor, with minimal performance loss. It includes a kernel module loaded in the guest system and a virtio device running on the host environment side. The kernel module code is written in C and distributed under the GPLv2 license, while the virtio device code is written in Rust and distributed under the Apache 2.0 license.
According to developers' estimates, the performance loss when accessing the GPU does not exceed 2%. High speed is achieved by redirecting ioctl calls to the native NVIDIA driver and operating at the NVIDIA driver ABI level, without translating requests at the API level. Standard libraries and high-level NVIDIA drivers operating in user space are used on the guest system.
In the guest system, a kernel module is loaded that provides device files /dev/nvidiactl, /dev/nvidia0…N, /dev/nvidia-uvm and passes ioctl() and mmap() requests through a virtqueue to a handler on the host environment side, which sends them to the host's NVIDIA driver. To eliminate context switches between the host system and the virtual machine, command buffers are mirrored (mmap) in shared memory accessible to the host environment and the GPU.
Unlike exclusive GPU passthrough (VFIO Passthrough), the virtio-nvgpu project provides shared access to a single GPU from multiple of virtual machines, and differs from virtio-gpu (Venus, VirGL) by operating through the ABI passthrough to the NVIDIA driver, rather than at the level of translating Vulkan and OpenGL graphics APIs. The downsides of virtio-nvgpu include the lack of full isolation based on IOMMU (guest systems operate through the host's NVIDIA driver and are limited only to filtering ioctl calls) and its dependence on the NVIDIA driver ABI (the ABI changes in each version and requires adaptation of virtio-nvgpu).
The capabilities of virtio-nvgpu are sufficient for rendering, video encoding, and computation using the Vulkan, OpenGL, NVENC, and CUDA APIs. Its main application area is mentioned to be the creation of cloud gaming services or virtual workstations, where a game or graphical application is launched in virtual machine, where rendering, hardware encoding of the result using NVENC, and streaming the output to a remote user in the form of a compressed video stream are done.
Source: opennet.ru
