Configuring static Hugepages for virtual machine usage

Static Hugepages can be used to increases the performance of a virtual machine. The downside is that the RAM allocated for Hugepages can not be used on the host system, even if the guest isn’t running.

The minimum size should be not smaller then 4GB.

Ubuntu 20.04 and newer

Get your Hugepage size via:

grep Huge /proc/meminfo

AnonHugePages:        0 kB
ShmemHugePages:        0 kB
FileHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB

Important for the calculation is the line Hugepagesize: 2048 kB

Calculate number of Hugepages

My total RAM size is 32GB or 32801012 kB, this was acquired via grep MemTotal /proc/meminfo.

I want half of it to a virtual machine as static Hugepages. This leaves us with: 16400506 kB

In order to reserve this for static huge pages we have to calculate the number of Hugepages, based on the Hugepagesize gathered in the previous step.

The calulation is 16400506kB/2048kB = 8008.059 in order to make sure we have enough space for overhead we add some % extra spac (some say 2%, some say 10%), to be on the safe side we use 8800.

Reserve the Hugepages

Edit the /etc/sysctl.conf file via sudo nano /etc/sysctl.conf and add the following line:

vm.nr_hugepages = 8800

Save the file and reboot.

Ubuntu 18.04 and older

Check if hugepages are installed

hugeadm --explain

If you get the error message:
hugeadm:ERROR: No hugetlbfs mount points found

install the missing package first before continuing:
sudo apt install hugepages

Edit

sudo nano /etc/default/qemu-kvm

and add or uncomment
KVM_HUGEPAGES=1

-> Reboot!

After the reboot, in a terminal window, enter again:
hugeadm --explain

Total System Memory: 32098 MB

Mount Point Options
/dev/hugepages rw,relatime,pagesize=2M
/run/hugepages/kvm rw,relatime,gid=130,mode=775,pagesize=2M

Huge page pools: 
 Size Minimum Current Maximum Default 
 2097152 0 0 0 * 
 1073741824 0 0 0 …

As you can see, hugepages are now mounted to /run/hugepages/kvm, and the hugepage size is 2097152 Bytes/(1024*1024)=2MB.

Another way to determine the hugepage size is:
grep "Hugepagesize:" /proc/meminfo

Hugepagesize: 2048 kB

Some math:
We want to reserve 8GB for Windows:
8GB = 8x1024MB = 8192MB

Our hugepage size is 2MB, so we need to reserve:
8192MB/2MB = 4096 hugepages

We need to add some % extra space for overhead (some say 2%, some say 10%), to be on the safe side you can use 4500.

Configure the hugepage pool

Again, run:

sudo nano /etc/sysctl.conf

and add the following lines into the file (a 16GB example):
# Set hugetables / hugepages for KVM single guest using 16GB RAM
vm.nr_hugepages = 8600

-> Reboot!

Set shmmax value

test again:
hugeadm --explain

Total System Memory: 32098 MB

Mount Point Options
/dev/hugepages rw,relatime,pagesize=2M
/run/hugepages/kvm rw,relatime,gid=130,mode=775,pagesize=2M

Huge page pools: 
 Size Minimum Current Maximum Default 
 2097152 8600 8600 8600 * 
 1073741824 0 0 0 
Huge page sizes with configured pools:  
 …

find the part that reads:
The recommended shmmax for your currently allocated huge pages is 18035507200 bytes.
To make shmmax settings persistent, add the following line to /etc/sysctl.conf:
kernel.shmmax = 18035507200

->  Do it!

sudo nano /etc/sysctl.conf
add the recommended line
kernel.shmmax = 18035507200
-> yours may differ!

Add Hugepages to your virtual machine

In order to use the previous configured static Hugepages in a virtual machine, add

<memoryBacking>
  <hugepages/>
</memoryBacking>

in your virsh edit.

Sources:

• Glorious arch wiki – KVM

• https://linuxconfig.org/how-to-enable-hugepages-on-linux

• https://help.ubuntu.com/community/KVM%20-%20Using%20Hugepages