CPU-pinning and further performance tweaks for virtual machines on AMD Ryzen CPUs

Considering gaming as the main use case for my passthrough setup I try to minimize the latency on the guest (don’t bother, straight to the config). One knob to turn in order to squeeze out more snappiness is CPU-pinning. 

This will allocate CPU-cores for mainly (or solely) Guest tasks, when the Guest is running. One could go one even further and restrict access to the guest cores completely, even if the guest isn’t running. This would use the isolcpus kernel command line flag at boot time. I do not use this feature as I would like to have maximum Host performance if the Guest is not running.

My test system runs:

  • AMD Ryzen 7 1800X (max boost clock 4GHz)

Decisions

I started with benchmarking several setups (found on level1tech, reddit and the arch wiki) – used cinebench, superpossition etc… In the end the results were, unfortunately, not as meaningful as I have hoped. Sure, more cores lead to better CPU bench results, but in terms of pure FPS performance the results were not significantly different. Thus I went with common sense.

Ryzen CPU architecture
Ryzen CPU architecture

The AMD Ryzen architecture houses 8 physical cores, each core capable of handling two threads. This leads to a total of 16 cores available for pinning. The 8 cores are separated into two complexes of 4 cores called CCX. Each CCX has its own L3 cache.

The plan is to have one CCX for the host, and one CCX for the guest. As the hosts runs first, ill assume it will use the (first) CCX with cores 0-3. The second CCX (cores 4-7) shall be used for the virtual machine.

I used a 12 pin setup to the Guest (6 cores) for half a year. Somtimes I encountered micro lag-spikes, which I was unable to track down the source of it.

Then I switched to 8 cpus (4 cores). The benchmarks indicated that the 6 core pinning had better CPU mark results, and slightly higher FPS, but since I switched to the CCX seperation the lag spikes went away. Here are my settings:

Core separation between host and guest system. Second core pins marked for CPU-pinning
Guest cores marked red

Virtual machine CPU configuration

In order edit the virtual machine

cd /etc/libvirt/qemu
sudo virsh define windows10.xml (change this according to your virtual machine name)
sudo virsh edit windows10

once your done with the edits and have saved your config re-run

sudo virsh define windows10.xml

First of all find the very first line, which should read:

<domain type='kvm'>

and replace it with:

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

Now find the line which ends with </vcpu>and add the following block in the next line:

<vcpu placement='static'>8</vcpu>   
<iothreads>1</iothreads>   
<cputune>     
   <vcpupin vcpu='0' cpuset='8'/>     
   <vcpupin vcpu='1' cpuset='9'/>     
   <vcpupin vcpu='2' cpuset='10'/>     
   <vcpupin vcpu='3' cpuset='11'/>     
   <vcpupin vcpu='4' cpuset='12'/>     
   <vcpupin vcpu='5' cpuset='13'/>     
   <vcpupin vcpu='6' cpuset='14'/>     
   <vcpupin vcpu='7' cpuset='15'/>     
   <emulatorpin cpuset='0-1'/>     
   <iothreadpin iothread='1' cpuset='0-1'/>   
</cputune>

Attention:Make sure <vcpu>, <iothreads> and <cputune> have the same indent.

Find the block <features>
and add the following block in parallel to the <acpi> block:

<hyperv> 
  <relaxed state='on'/>
   <vapic state='on'/>
   <spinlocks state='on' retries='8191'/>
</hyperv>

Attention:Make sure <hyperv>and <acpi> have the same indent.

Find the block <CPU> and adapt it to look like this:

  <cpu mode='host-passthrough' check='none'>
    <topology sockets='1' cores='4' threads='2'/>
    <cache level='3' mode='emulate'/>
  </cpu> 

Leave a Reply

Wir benutzen Cookies um die Nutzerfreundlichkeit der Webseite zu verbessen. Durch Deinen Besuch stimmst Du dem zu.