CPU Hotplug¶
The CPU hotplug feature was introduced in KubeVirt v1.0, making it possible to configure the VM workload to allow for adding or removing virtual CPUs while the VM is running.
Abstract¶
A virtual CPU (vCPU) is the CPU that is seen to the Guest VM OS. A VM owner can manage the amount of vCPUs from the VM spec template using the CPU topology fields (spec.template.spec.domain.cpu
). The cpu
object has the integers cores,sockets,threads
so that the virtual CPU is calculated by the following formula: cores * sockets * threads
.
Before CPU hotplug was introduced, the VM owner could change these integers in the VM template while the VM is running, and they were staged until the next boot cycle. With CPU hotplug, it is possible to patch the sockets
integer in the VM template and the change will take effect right away.
Per each new socket that is hot-plugged, the amount of new vCPUs that would be seen by the guest is cores * threads
, since the overall calculation of vCPUs is cores * sockets * threads
.
Configuration¶
Enable feature-gate¶
In order to enable CPU hotplug we need to add the VMLiveUpdateFeatures
feature gate in Kubevirt CR:
apiVersion: kubevirt.io/v1
kind: KubeVirt
spec:
configuration:
developerConfiguration:
featureGates:
- VMLiveUpdateFeatures
Configure the workload update strategy¶
Current implementation of the hotplug process requires the VM to live-migrate.
The migration will be triggered automatically by the workload updater. The workload update strategy in the KubeVirt CR must be configured with LiveMigrate
, as follows:
apiVersion: kubevirt.io/v1
kind: KubeVirt
spec:
workloadUpdateStrategy:
workloadUpdateMethods:
- LiveMigrate
Configure the VM rollout strategy¶
Hotplug requires a VM rollout strategy of LiveUpdate
, so that the changes made to the VM object propagate to the VMI without a restart.
This is also done in the KubeVirt CR configuration:
More information can be found on the VM Rollout Strategies page
[OPTIONAL] Set maximum sockets or hotplug ratio¶
You can explicitly set the maximum amount of sockets in three ways:
- with a value VM level
- with a value at the cluster level
- with a ratio at the cluster level (
maxSockets = ratio * sockets
).
Note: the third way (cluster-level ratio) will also affect other quantitative hotplug resources like memory.
VM level |
Cluster level value |
Cluster level ratio |
---|---|---|
The VM-level configuration will take precedence over the cluster-wide configuration.
Hotplug process¶
Let's assume we have a running VM with the 4 vCPUs, which were configured with sockets:4 cores:1 threads:1
In the VMI status we can observe the current CPU topology the VM is running with:
apiVersion: kubevirt.io/v1
kind: VirtualMachineInstance
...
status:
currentCPUTopology:
cores: 1
sockets: 4
threads: 1
kubectl patch vm vm-cirros --type='json' \
-p='[{"op": "replace", "path": "/spec/template/spec/domain/cpu/sockets", "value": 5}]'
status:
conditions:
- lastProbeTime: null
lastTransitionTime: null
status: "True"
type: LiveMigratable
- lastProbeTime: null
lastTransitionTime: null
status: "True"
type: HotVCPUChange
currentCPUTopology:
cores: 1
sockets: 4
threads: 1
Please note the condition HotVCPUChange
that indicates the hotplug process is taking place.
Also you can notice the VirtualMachineInstanceMigration object that was created for the VM in subject:
currentCPUTopology
will be updated with the new number of sockets and the migration
is marked as successful.
#kubectl get vmi vm-cirros -oyaml
apiVersion: kubevirt.io/v1
kind: VirtualMachineInstance
metadata:
name: vm-cirros
spec:
domain:
cpu:
cores: 1
sockets: 5
threads: 1
...
...
status:
currentCPUTopology:
cores: 1
sockets: 5
threads: 1
#kubectl get vmim -l kubevirt.io/vmi-name=vm-cirros
NAME PHASE VMI
kubevirt-workload-update-cgdgd Succeeded vm-cirros
Limitations¶
- VPCU hotplug is currently not supported by ARM64 architecture.
- Current hotplug implementation involves live-migration of the VM workload.