Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Proxmox - GPU Passthrough - Configure Proxmox ======

===== Update Grub =====

SSH directly into the Proxmox server.

Edit /etc/default/grub file.

Change the line:

<file bash /etc/default/grub>
GRUB_CMDLINE_LINUX_DEFAULT="quiet"
</file>

==== For an Intel CPU ====

<file bash /etc/default/grub>
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on"
</file>

==== For an AMD CPU ====


<file bash /etc/default/grub>
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on"
</file>

<WRAP info>
**NOTE:**  Additional commands may be needed as well in this line, if the passthrough ends up failing.

For example:

<code>
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction nofb nomodeset video=vesafb:off,efifb:off"
</code>

  * Disabling the Framebuffer: **[[https://passthroughpo.st/explaining-csm-efifboff-setting-boot-gpu-manually/|video=vesafb:off,efifb:off]]**.
  * ACS Override for IOMMU groups:  **[[http://vfio.blogspot.com/2014/08/vfiovga-faq.html|pcie_acs_override=downstream,multifunction]]**.
</WRAP>

----

When editing is finished run this command:

<code bash>
update-grub
</code>

----

===== VFIO Modules =====

Add a few VFIO (Virtual Function I/O) modules to your Proxmox system.

Add the following to the /etc/modules file:

<file bash /etc/modules>
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
</file>

Save and exit.

----

===== IOMMU interrupt remapping =====

Run the following commands:

<code bash>
echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf
</code>

----

===== Blacklist Drivers =====

To prevent the Proxmox host system utilizing the GPU(s), some drivers need to be blacklisted.

<code bash>
echo "blacklist radeon" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
echo "blacklist nvidia" >> /etc/modprobe.d/blacklist.conf
</code>

----

===== Adding GPU to VFIO =====

==== Determine the GPU ====

<code bash>
lspci -v
</code>

returns:

<code bash>
01:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] (rev a1) (prog-if 00 [VGA controller])
01:00.1 Audio device: NVIDIA Corporation GP104 High Definition Audio Controller (rev a1)
</code>


<WRAP info>
**NOTE:**  Look for the line(s) that show the video card.

Make note of the first set of numbers (e.g. 01:00.0 and 01:00.1).

</WRAP>

----

==== Determine the GPU Vendor IDs ====

Run the command below.

<WRAP info>
**NOTE:**  Replace 01:00 with whatever number was next to your GPU when you ran the previous command:
</WRAP>


<code bash>
lspci -n -s 01:00
</code>

returns:

<code bash>
01:00.0 0000: 10de:1b81 (rev a1)
01:00.1 0000: 10de:10f0 (rev a1)
</code>

<WRAP info>
**NOTE:**  This should output the GPU Vendor IDs, usually one ID for the GPU and one ID for the Audio bus.

Make a note of the vendor id codes: 10de:1b81 and 10de:10f0.

</WRAP>

----
    
==== Add the GPU Vendor IDs to the VFIO ====

<code bash>
echo "options vfio-pci ids=10de:1b81,10de:10f0 disable_vga=1"> /etc/modprobe.d/vfio.conf
</code>

----

===== Update images =====

<code bash>
update-initramfs -u
</code>

<WRAP info>
**NOTE:**  **initramfs** is a tiny version of the OS that gets loaded by the bootloader, right after the kernel.

  * It lives in RAM, and it provides **just** enough tools and instructions to tell the kernel how to set up the real filesystem, mount the HDD read/write, and start loading all the system services.
  * It includes the stub of init, PID #1.
  * If your initramfs is broken, your boot fails.

**update-initramfs** is a script that updates initramfs to work with a new kernel.

Updating the initramfs prevents some modules from being loaded.

</WRAP>

----

===== Restart =====

<code bash>
sudo reboot
</code>

<WRAP info>
**NOTE:**  The Proxmox host should be ready to passthrough GPUs!
</WRAP>