QEMU is a powerful free and open source emulator which when paired with kvm can be used to create almost bare-metal performance virtual machines. In this guide I will be detailing some tips and tricks to configuring a setup on your linux system to allow a PCI device (typically a graphics card) to be passed through to a virtual machine.
Countless guides already exist on this topic but they all rely on using virt-manager and other Redhat software, which, depending on your use case, may be completely overkill. This guide assumes that you will not be using the target GPU as a video output on your host machine from boot, so will only work in configurations where you are able to remote connect or where you have **multiple graphics cards**. However the process is mostly similar for single GPU passthrough, with extra steps if you want to bind and unbind display drivers from the host.
This is a generic guide written to support any semi-standard Linux distributions, so adapt any instructions as you see fit to your current system.
If you do not trust this guide or need any clarification, feel free to follow steps from other guides, or just skip to the tips at the bottom of this guide.
## Prerequisites
Make sure that your motherboard is:
- supports hardware virtualisation
- supports IOMMU
These options can be enabled within the motherboard's BIOS settings usually.
You may also want a spare monitor or a monitor with multiple inputs so that you can switch between GPU outputs.
Once you have enabled IOMMU, you will need to ensure that your PCI slot can actually be passed through. This is only possible if the GPU appears in its own single IOMMU group, with no other devices in that group.
To list IOMMU groups, you may use this script from the [archwiki](https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF#Ensuring_that_the_groups_are_valid)
If you see that there PCI devices other than ones that correspond to your graphics card, then try using a different PCI slot. Unfortunately some motherboards don't have any isolated PCI slots, in this case you are out of luck; PCI passthrough will not be possible in this method.
## Setting up VFIO
The core principle behind PCI passthrough is that you don't want the kernel from having any control over the device in the PCI slot. To do this, we tell the kernel to bind a *dummy driver* to the gpu so that when we want to pass it through to the VM, it is not in use.
To do this, you want to find out the Device IDs of your PCI device, you can do this using the following command:
Make sure take notes of all of the devices that appeared within the isolated IOMMU group from the previous stage. If you miss out any PCI devices, this will not work. IDs often look like this: `10de:13c2`
Next add the following to your kernel's cmdline arguments. This can usually be found somewhere in your bootloaders settings, for example for GRUB, you can add it to `GRUB_CMDLINE_LINUX_DEFAULT=""` in `/etc/default/grub`
where *id1* and *id2* represent the ids that you collected in the previous step.
Next you need to tell your initramfs (if applicable) to load the vfio modules. This is done to make sure that the vfio module is loaded and assigned to these pci devices *before* your video drivers. The process of doing this depends on your initramfs system:
In this guide I will be using qemu from the command line. I feel as if this is the easiest way to create virtual machines without any overhead, but you can use libvirt if you wish.
Here is a script that use a to run a linux virtual machine, if you want to copy it blindly and not understand it, then thats alright, but I will include a description for each argument and why its used.
Most PCI cards require UEFI firmware to function properly. For this reason we will be using the OVMF firmware. You may need to install this on your system, typically the package is called ovmf, edk2-ovmf or something similar. Learn to use your package manager's search functionality to find it.
Make a copy of the OVMF variables file, typically located at `/usr/share/OVMF/OVMF_VARS.fd` and place it somewhere with the rest of your virtual machine's files. You should keep these separate and unique for each virtual machine you wish to create.
In my example script, make sure you replace `/path/to/your/ovmf/vars/image` to the path to your copy of `OVMF_VARS.fd`
Next you need to create a virtual disk for your virtual machine. You can create this using the following command, replacing `32G` with the desired size of this disk.
In this example I use `if=virtio,` which provides better performance than the default drive type, however this will only work with linux guests which have the virtio module. **Remove this if you are using windows in your virtual machine**
For your first boot, you may want to add the following before your primary virtual disk:
Ensure that you have set the `ISO` and `ROOT` variables appropriately with the paths to the corresponding images.
### Disabling virtual video output
This next step is to ensure that qemu doesn't create a virtual VGA output for your virtual machine, nor opens a window, allowing this to be run from outside an X11 session
I use the virtio network card, which only works for Linux guests. **If you have a windows guest, do not include this line** this will use the default network card for qemu.
Then when your virtual machine is running, you will be able to switch to and from the host's control by pressing both left and right ctrl keys at the same time on your keyboard.
### Running as an ordinary user
To do this, I would recommend creating a group named `kvm` and adding your user to it.
Next you will probably want to increase memory limits for users of the kvm group, to allow them to allocate potentially GB for the virtual machine. To make things easier, you might want to just set this to the maximum number of megabytes available in the system. You can find this out using `free`, for example for a system with 8GB ram, its: `8100452`
In `/etc/security/limits.d/99-memlock.conf` write:
You may need to reboot for these changes to take effect, especially ones relating to udev rules.
### Using ddcutil to switch between monitor inputs
Most monitors, other than laptop displays, have a Virtual Control Panel which can be controlled through i2c as per the Display Data Channel/ Command Interface Standard... DDC/CI
Setting up ddcutil to work on your monitor will depend on a case to case basis depending on your monitor and video card.
Depending on your monitor, you may need to enable DDC/CI in its settings.
Make sure you have installed ddcutil and i2c-dev, again exact package names may vary.
To detect the montors available, use `ddcutil detect`. If this doesn't work, ensure that the i2c-dev module is loaded.
To enable the i2c module on load, you may need to add the following to `/etc/modules-load.d/i2c-dev.conf`
To allow users of the i2c group to control i2c devices add the following to `/etc/udev/rules.d/10-i2c-group.rules`. Make sure you create this group and add your user to it.
I have this bound to a hotkey using `sxhkd` so I can easily switch between inputs without having to reach over to buttons on my monitor.However you can configure this whichever way you want: for example, you can switch to the display output of your passthrough GPU when the virtual machine starts, and back when it shuts down.
## Conclusion
Hopefully, this guide has helped you set up a virtual machine with PCI passthrough. All thats left now is to install the software you want in your virtual machine and have fun.
If there is anything that isn't clear in this guide, please contact me, or look at other guides if you need any help. The archwiki is a pretty good place to look if you are, *or aren't* using archlinux.