Introduction
KNI PAE is tested against 3 platforms: libvirt, aws and baremetal. A typical test consist on:
- cleaning environment
- deploying a kubernetes cluster for the desired platform
- apply KNI PAE workloads on it
- destroy cluster
Overall Test Architecture
- Our tests are performed on Akraino Jenkins at https://jenkins.akraino.org/view/kni/ .
Following the different tests per platform are explained
kni-installer-verify-binary, kni-installer-verify-installer
Those tests are executed each time that there is a change in https://gerrit.akraino.org/r/admin/repos/kni/installer . It tests the code in this client tool. The verify-binary one is testing that is possible to construct a binary from the code. The verify-installer one compiles the code of installer, and checks that is valid.
They are executed on a Centos-8g node, provided by Akraino CI
kni-blueprint-pae-verify-deploy-libvirt
Those tests are executed each time that there is a change on https://gerrit.akraino.org/r/admin/repos/kni/blueprint-pae , on the generic and on libvirt profiles.
It executes a deployment of the cluster and applies workloads based on libvirt. All the VMs are run on a baremetal server stored inside Red Hat, connected as a Jenkins slave.
Logs are deployed on https://nexus.akraino.org/.
Following there is the architecture *:
* Green=Akraino dependencies, Orange=system under test, Blue=components created
kni-blueprint-pae-verify-deploy-aws
Those tests are executed each time that there is a change on https://gerrit.akraino.org/r/admin/repos/kni/blueprint-pae , on the generic and on AWS profiles.
It executes a deployment of the cluster and applies workloads based on AWS. The bootstrap VM is run on a baremetal server, connected as a Jenkins Slave. The master and worker nodes are executed inside a testing AWS cluster.
Logs are deployed on https://nexus.akraino.org/.
Following there is the architecture *:
* Green=Akraino dependencies, Orange=system under test, Blue=components created
kni-blueprint-pae-verify-deploy-baremetal
Those tests are executed each time that there is a change on https://gerrit.akraino.org/r/admin/repos/kni/blueprint-pae , on the generic and on baremetal profiles.
It executes a deployment of the cluster and applies workloads based on baremetal. The bootstrap VM is run on a baremetal server, connected as a Jenkins Slave. The master and worker nodes are also baremetal slaves, running on a local lab.
Logs are deployed on https://nexus.akraino.org/.
Following there is the architecture *:
* Green=Akraino dependencies, Orange=system under test, Blue=components created
Test Framework
For the moment, the deployment tests are just consisting on deploying a cluster and running workloads on top of it, checking that they are successfully deployed.
Testings in validation lab are executed manually for the moment
Automated test deploy
The validation of KNI-PAE blueprint is currently based on deploying our cluster and applying workloads on top of it, every time that there is a change on the blueprint repo .
Additionally test can be run manually to verify deployment at a certain stage.
Validation on the blueprint is done for AWS, GCP and virtual baremetal.
At the end of the tests, the deployed cluster is destroyed.
Sample tests can be seen at:
https://jenkins.akraino.org/view/kni/job/kni-blueprint-pae-verify-deploy-aws/
https://jenkins.akraino.org/view/kni/job/kni-blueprint-pae-verify-deploy-baremetal/
https://jenkins.akraino.org/view/kni/job/kni-blueprint-pae-verify-deploy-gcp/
Conformance Test
This is performed through Akraino Blueprint Validation project framework.
However the tests are currently launched manually after a cluster is deployed, integration with the CI is still pending, as we are hitting issues with the framework itself:
https://jira.akraino.org/projects/VAL/issues/VAL-108
https://jira.akraino.org/projects/VAL/issues/VAL-109
https://jira.akraino.org/projects/VAL/issues/VAL-110
As we use OpenShift, we cannot use the standard k8s conformance tests, because they are aimed for upstream Kubernetes and not for OpenShift.
As an alternative, we can run the openshift test validation suite: https://github.com/openshift/origin/blob/master/test/extended/conformance-k8s.sh
This is similar as the sonobuoy one, launching a set of e2e tests to validate that the cluster is deployed and works at a functional level. Those are the collected results:
https://logs.akraino.org/redhat-kni/bluval_results/blueprint-pae/20200505-104443/out.log
Security Test
kube-hunter test was applied on the cluster:
https://logs.akraino.org/redhat-kni/bluval_results/blueprint-pae/20200423-071856/results/k8s/kube-hunter/Kube-Hunter.Kube-Hunter/cluster.log → In OpenShift we expose our version and we do not have control in configuration level for hiding it
https://logs.akraino.org/redhat-kni/bluval_results/blueprint-pae/20200423-071856/results/k8s/kube-hunter/Kube-Hunter.Kube-Hunter/pod.log → CAP_RAW is enabled by default in OpenShift, and same with the other Access Errors. It will need some advanced configuration to bypass these errors but by default the clusters will deploy with these security warnings.
Test Dashboards
https://jenkins.akraino.org/view/kni