This is an old revision of the document!
Table of Contents
Proxmox - GPU Passthrough
PCI passthrough allows you to use a physical PCI device (graphics card, network card) inside a VM (KVM virtualization only).
If you “PCI passthrough” a device, the device is not available to the host anymore.
The hardware should, at the very least, support: VT-d, interrupt mapping, and UEFI BIOS.
Update Grub
SSH directly into the Proxmox server.
Edit /etc/default/grub file.
Change the line:
- /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet"
For an Intel CPU
- /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
For an AMD CPU
- /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"
NOTE: Additional commands may be needed as well in this line, if the passthrough ends up failing.
For example:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb:off,efifb:off"
- Disabling the Framebuffer: video=vesafb:off,efifb:off.
- ACS Override for IOMMU groups: pcie_acs_override=downstream,multifunction.
When editing is finished run this command:
update-grub
VFIO Modules
Add a few VFIO (Virtual Function I/O) modules to your Proxmox system.
Add the following to the /etc/modules file:
- /etc/modules
vfio vfio_iommu_type1 vfio_pci vfio_virqfd
Save and exit.
IOMMU interrupt remapping
Run the following commands:
echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf
Blacklist Drivers
To prevent the Proxmox host system utilizing the GPU(s), some drivers need to be blacklisted.
echo "blacklist radeon" >> /etc/modprobe.d/blacklist.conf echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf echo "blacklist nvidia" >> /etc/modprobe.d/blacklist.conf
Adding GPU to VFIO
Determine the GPU
lspci -v
returns:
01:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] (rev a1) (prog-if 00 [VGA controller]) 01:00.1 Audio device: NVIDIA Corporation GP104 High Definition Audio Controller (rev a1)
NOTE: Look for the line(s) that show the video card.
Make note of the first set of numbers (e.g. 01:00.0 and 01:00.1).
Determine the GPU Vendor IDs
Run the command below.
NOTE: Replace 01:00 with whatever number was next to your GPU when you ran the previous command:
lspci -n -s 01:00
returns:
01:00.0 0000: 10de:1b81 (rev a1) 01:00.1 0000: 10de:10f0 (rev a1)
NOTE: This should output the GPU Vendor IDs, usually one ID for the GPU and one ID for the Audio bus.
Make a note of the vendor id codes: 10de:1b81 and 10de:10f0.
Add the GPU Vendor IDs to the VFIO
echo "options vfio-pci ids=10de:1b81,10de:10f0 disable_vga=1"> /etc/modprobe.d/vfio.conf ---- Update <code bash> update-initramfs -u
Restart
sudo reboot
NOTE: The Proxmox host should be ready to passthrough GPUs!
Configuring the VM
Edit the VM Config File
Edit /etc/pve/qemu-server/<vmid>.conf, where <vmid> is the VM ID Number you used during the VM creation (General Tab).
Add these lines.
machine: q35 cpu: host,hidden=1,flags=+pcid args: -cpu 'host,+kvm_pv_unhalt,+kvm_pv_eoi,hv_vendor_id=NV43FIX,kvm=off'
Add PCI Devices (Your GPU) to VM
Select the VM.
Under the Hardware Tab, click on the Add button towards the top.
Then under the drop-down menu, click PCI Device.
Look for the GPU in the list, and select it.
On the PCI options screen, you should only need to configure it like so:
- All Functions: Checked.
- Rom-Bar: Checked.
- Primary GPU: Not Checked.
- PCI-Express: Checked. This requires machine: q35 in the VM config file.