Table of Contents
KVM - Set CPU Shares for Live Running Instance Permanently
virsh schedinfo VM_ID \ --set cpu_shares=[0-262144] \ --live \ --current \ --config
NOTE: KVM represents each virtual CPU as a thread in the host Linux system, actually as a thread in the QEMU process.
On Linux, you can use taskset to force specific threads onto specific CPUs. So that will let you assign one VCPU to one physical CPU and two VCPUs to another.
A virtual CPU equates to 1 physical core, but when your VM attempts to process something, it can potentially run on any of the actual cores that happen to be available at that moment. The scheduler handles this, and the VM is not aware of it. You can assign multiple vCPUs to a VM which allows it to run concurrently across several cores.
Cores are shared between all VMs as needed, so you could have a 4-core system, and 10 VMs running on it with 2 vCPUs assigned to each. VMs share all the cores in your system quite efficiently as determined by the scheduler. This is one of the main benefits of virtualization - making the most use of under-subscribed resources to power multiple OS instances.
If your VMs are so busy that they have to contend for CPU time, the outcome is that VMs may have to wait for CPU time. Again, this is transparent to the VM and handled by the scheduler.
Running your VM on specific CPUs
One of the benefits of the qemu/kvm model is management of the qemu process using linux userspace utilities as qemu runs in userspace as a normal linux process.
One of these useful linux utilities is called taskset which allows setting the cpu affinity of a process. This is just a fancy way of saying bonding a process to a specific set of cpu(s).
Why would you want to do this? The short answer is Load balancing.
With multicore cpus becoming the de facto standard, there are some very practical uses of this when it comes to virtualization. On a typical multicore cpu today, each core has enough power to easily run a physical machine so you can see the practical application of this; pin a virtual machine to a cpu core. This is applicable for most applications. If your application needs more cpu cycles you can scale up by pinning your virtual machine to multiple cores. Using the Linux taskset utility you have two options in applying it to your virtual machine process.
- You can set the cpu affinity of an already running process; or
- You can start the process using a specific cpu affinity.
The fact that you can specify the cpu affinity of an already running process really adds to the flexibility and usefulness of this utility.
Let’s look at both options for manipulating your kvm virtual machines processes.
Start a VM with specific CPU affinity
To start your qemu/kvm virtual machine on logical cpus 0 and 1, start your virtual machine with the following command.
taskset 0x00000003 qemu-system-x86_64 –hda windows.img –m 512
NOTE: When starting a new process using taskset, you have to specify one argument which represents a bitmask of the cpus you want to bond the process to.
As you can see this mask argument is in hexadecimal.
The mask value 0x00000003 represents logical cpu numbers 0 and 1 as it starts counting from 0.
To verify that your VM is running on logical cpus 0 and 1, use the taskset to verify this using the process id of your virtual machine.
To get the process id of your virtual machine process use the following command.
sudo ps -eo pid,comm | grep qemu 1234 qemu-system-x86
NOTE: From the above command the process id of the qemu/kvm process is 1234.
Run the following command to verify the cpu affinity of the virtual machine process.
sudo taskset -p 1234 pid 1234's current affinity mask: 3
NOTE: This is saying that the bitmask representing the cpu affinity of the process is 3.
This bitmask is more machine friendly.
To get a more human friendly verification run the following command instead.
sudo taskset -c -p 1234 pid 1234's current affinity list: 0,1
NOTE: This is much more human friendly and verifies that your virtual machine is running on logical cpus 0 and 1.
Modify an already running VM process
As mentioned earlier, taskset also allows you to modify your already running virtual machine process so that you can change the cpu affinity.
Let’s change it so that it only runs on logical cpu 0.
Using the same process id in the section above, run the following command.
sudo taskset -p 0x00000001 1234 pid 1234's current affinity mask: 3 pid 1234's new affinity mask: 1
NOTE: taskset sets the affinity and shows the old and new bitmask.
To also verify using the human friendly way with the following command:
sudo taskset -c -p 1234 pid 1234's current affinity list: 0
NOTE: You can see that your virtual machine process is running on logical cpu 0 only.
Summary
As multi-core technology continue to increase the number of cores, the application of bonding virtual machine processes to specific cpus will become more commonplace.
This is one clear example of how the qemu/kvm model leverages Linux to its advantage.
There was absolutely no development needed by KVM maintainers in order for users to have access to this practical facility.
References
virsh kvm CPU affinity problems
Some new kvm options: boot order and process name
https://stackoverflow.com/questions/47621257/kvm-cpu-share-priority-overselling
https://serverfault.com/questions/313333/kvm-and-virtual-to-physical-cpu-mapping
https://groups.google.com/forum/#!topic/linuxkernelnewbies/qs5IiIA4xnw