Table of Contents
Ubuntu - GPU - GPU pass-through - Configuring the host
Verify the host environment supports GPU pass-through
TODO:
Enable IOMMU
IOMMU is usually disabled by default.
- It needs to be enable at boot time in the /etc/default/grub configuration file.
For AMD-based hosts:
- /etc/default/grub
GRUB_CMDLINE_LINUX="iommu=pt amd_iommu=on rd.driver.pre=vfio-pci"
For Intel-based hosts:
- /etc/default/grub
GRUB_CMDLINE_LINUX="intel_iommu=on iommu=pt rd.driver.pre=vfio-pci"
Re-generate the main GRUB 2 configuration file
The GRUB 2 configuration file is /boot/grub2/grub.cfg.
sudo grub2-mkconfig -o /boot/grub2/grub.cfg
Reboot
Verify that IOMMU is enabled
sudo dmesg | grep -e DMAR -e IOMMU
returns:
[ 2.172228] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported [ 2.176588] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40 [ 2.176924] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank). [ 3.875165] AMD-Vi: AMD IOMMUv2 loaded and initialized
Blacklist the Nouveau driver
Prevent the host OS from loading the built-in nouveau driver.
Create the file /etc/modprobe.d/60-blacklist-nouveau.conf:
- /etc/modprobe.d/60-blacklist-nouveau.conf
blacklist nouveau
Configure VFIO and isolate the GPU used for pass-through
Find the card vendor and model IDs.
lspci -k | grep VGA
returns:
0c:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Navi 31 [Radeon RX 7900 XT/7900 XTX] (rev c8)
NOTE: This shows the Bus number is 0c:00.0.
Get the device ID
Using the Bus number identified earlier.
lspci -nn | grep 0c:00.0
returns:
0c:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 31 [Radeon RX 7900 XT/7900 XTX] [1002:744c] (rev c8)
NOTE: This shows the ID is 1002:744c.
Create a module VFIO file
Create the file /etc/modprobe.d/vfio.conf:
- /etc/modprobe.d/vfio.conf
options vfio-pci ids=1002:744c
NOTE: Verify that the card does not need an extra ids= parameter.
- For some cards, the audio device must be specified too, so that ID must also be added to the list, otherwise the card will be unusable.
Load the VFIO driver
There are various ways you can load the VFIO driver:
Including the driver in the initrd file
Create the file /etc/dracut.conf.d/gpu-passthrough.conf:
add_drivers+=" vfio vfio_iommu_type1 vfio_pci vfio_virqfd"
NOTE: Mind the leading whitespace.
Re-generate the initrd file:
sudo dracut --force /boot/initrd $(uname -r)
Adding the driver to the list of auto-loaded modules
Create the file /etc/modules-load.d/vfio-pci.conf:
- /etc/modules-load.d/vfio-pci.conf
vfio vfio_iommu_type1 vfio_pci kvm kvm_intel
Load the driver manually
To load the driver manually at runtime:
sudo modprobe vfio-pci
Disable MSR for Microsoft Windows guests
For Microsoft Windows guests, it is recommended to disable MSR (model-specific register) to avoid the guest crashing.
Create the file /etc/modprobe.d/kvm.conf:
- /etc/modprobe.d/kvm.conf
options kvm ignore_msrs=1
Install UEFI firmware
For proper GPU Pass-Through functionality, the host needs to boot using UEFI firmware.
- Not using a legacy-style BIOS boot sequence.
Install the qemu-ovmf package if not already installed:
sudo apt install qemu-ovmf
Reboot the host machine
For most of the changes in the above steps to take effect, the host machine needs to be rebooted.
sudo shutdown -r now