| Table of Contents | ||
|---|---|---|
|
...
In details, KubeVirt technology addresses the needs of development teams that have adopted or want to adopt Kubernetes but possess existing Virtual Machine-based workloads that cannot be easily containerized. So KubeVirt extends Kubernetes by adding additional virtualization resource types (especially the VM/VMI type) through Kubernetes's Custom Resource Definitions API. By using this mechanism, the Kubernetes API can be used to manage these VM resources alongside all other resources Kubernetes provides. The resources themselves in Kubernetes are not enough to launch virtual machines. For this to happen, the functionality and business logic needs to be added to the cluster. Scheduling, networking, and storage are all delegated to Kubernetes, while KubeVirt provides the virtualization functionality. The functionality is not added to Kubernetes itself, but rather added to a Kubernetes cluster by running additional controllers and agents on an existing cluster. And these necessary controllers and agents are all provided by KubeVirt.
...
So to enable QAT in Kubevirt, it is necessray to create related segments to the validation service and add the QAT feature Gate verified method.
- Add necessary information to swagger.json which is an add on for kubenetes API used in Kubevirt
... "qats": { ... "v1.QAT": { |
|---|
- Add Feature Gate to webhook validation service tp
- to config it in configmap
... if spec.Domain.Devices.QATs != nil && !config.QATPassthroughEnabled() { ... |
|---|
...
Non-privileged
To meet the Kubernetes and Kubevirt community specifications, the pod should be non-privileged. So we should mount the assigned QAT pci device to the VM through the interfaces Kubevirt provided.
| if util.IsQATVMI(vmi) { for _, qat := range vmi.Spec.Domain.Devices.QATs { requestResource(&resources, qat.DeviceName) } } |
|---|
PCI passthrough
...
This will call the Kubevirt to mount necessary resources for QAT into the VM, such as /sys/devices/.
PCI passthrough
Fundamentally, Kubevirt create a Kubernetes CRD to hold some resource configurations that fit a libvrit instance. So actually to assign a QAT vf into a VM is to create a libvirt and insert the specific device into hostdev block. (In libvirt, hostdev label means a plain host device assignment with all its limitations)
| <hostdev mode='subsystem' type='pci' managed='yes'> <driver |
|---|
<source>
<address domain='0x0000' bus='0x3d' slot='0x02' function='0x2'/>
</source>
<alias name='hostdev0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</hostdev>
Example
...
| name='vfio'/> <source> <address domain='0x0000' bus='0x3d' slot='0x02' function='0x2'/> </source> <alias name='hostdev0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/> </hostdev> |
|---|
So when a QAT vf resource is required, I call the Kubevirt method to get the device id of the assigned QAT and integrate it into libvirt domain.
// Append HostDev to libvirt DomXML if QAT is required qatPCIAddresses := append([]string{}, c.QATDevices...) ... |
|---|
Example
Based on the Kubevirt official example VMI file, to make use of my QAT version, after building its docker images and uploading to docker, we should open the QAT feature gate through configmap.
| apiVersion: v1 kind: ConfigMap metadata: name: kubevirt-config namespace: kubevirt labels: kubevirt.io: "" data: feature-gates: "QAT" |
|---|
And the key point of QAT VMI:
... spec: ... |
|---|
Conclusion
The patch now has not been merged into the Kubevirt Project and is in review now. (PR: https://github.com/kubevirt/kubevirt/pull/2980)