Introduction

KNI IE is tested against Google Cloud Platform (GCP). A typical test consist of:

cleaning environment
deploying a management hub cluster via blueprint on the target platform
deploying an industrial edge cluster via blueprint on the target platform
letting the industrial edge cluster auto-register with the management hub and deploy its workloads from the blueprint
destroy cluster

Overall Test Architecture

Our tests are performed by Akraino Jenkins at https://jenkins.akraino.org/view/kni/.
Following the different tests per platform are explained

kni-installer-verify-binary, kni-installer-verify-installer

Those tests are executed each time that there is a change in https://gerrit.akraino.org/r/admin/repos/kni/installer. It tests the code in this client tool. The verify-binary one is testing that is possible to construct a binary from the code. The verify-installer one compiles the code of installer, and checks that is valid.
They are executed on a Centos-8g node, provided by Akraino CI

kni-blueprint-ie-verify-deploy-gcp

Those tests are executed each time that there is a change on https://gerrit.akraino.org/r/admin/repos/kni/blueprint-ie.
It executes a deployment of the cluster and applies workloads based on AWS. The bootstrap VM is run on a baremetal server, connected as a Jenkins Slave. The master and worker nodes are executed inside a testing AWS cluster.
Logs are deployed on https://nexus.akraino.org/.

Following there is the architecture *:

* Green=Akraino dependencies, Orange=system under test, Blue=components created

Test Framework

For the moment, the deployment tests are just consisting on deploying a cluster and running workloads on top of it, checking that they are successfully deployed.
Testings in validation lab are executed manually for the moment

Automated test deploy

The validation of KNI-PAE blueprint is currently based on deploying our cluster and applying workloads on top of it, every time that there is a change on the blueprint repo .
Additionally test can be run manually to verify deployment at a certain stage.

Validation on the blueprint is done for AWS, GCP and virtual baremetal.

At the end of the tests, the deployed cluster is destroyed.

Sample tests can be seen at:
https://jenkins.akraino.org/view/kni/job/kni-blueprint-pae-verify-deploy-aws/
https://jenkins.akraino.org/view/kni/job/kni-blueprint-pae-verify-deploy-baremetal/
https://jenkins.akraino.org/view/kni/job/kni-blueprint-pae-verify-deploy-gcp/

Conformance Test

This is performed through Akraino Blueprint Validation project framework.

However the tests are currently launched manually after a cluster is deployed, integration with the CI is still pending, as we are hitting issues with the framework itself:
https://jira.akraino.org/projects/VAL/issues/VAL-108
https://jira.akraino.org/projects/VAL/issues/VAL-109
https://jira.akraino.org/projects/VAL/issues/VAL-110

As we use OpenShift, we cannot use the standard k8s conformance tests, because they are aimed for upstream Kubernetes and not for OpenShift.
As an alternative, we can run the openshift test validation suite: https://github.com/openshift/origin/blob/master/test/extended/conformance-k8s.sh
This is similar as the sonobuoy one, launching a set of e2e tests to validate that the cluster is deployed and works at a functional level. Those are the collected results:

https://logs.akraino.org/redhat-kni/bluval_results/blueprint-pae/20200505-104443/out.log

Security Test

kube-hunter test was applied on the cluster:

https://logs.akraino.org/redhat-kni/bluval_results/blueprint-pae/20200423-071856/results/k8s/kube-hunter/Kube-Hunter.Kube-Hunter/cluster.log → In OpenShift we expose our version and we do not have control in configuration level for hiding it

https://logs.akraino.org/redhat-kni/bluval_results/blueprint-pae/20200423-071856/results/k8s/kube-hunter/Kube-Hunter.Kube-Hunter/pod.log → CAP_RAW is enabled by default in OpenShift, and same with the other Access Errors. It will need some advanced configuration to bypass these errors but by default the clusters will deploy with these security warnings.

Test Dashboards

https://jenkins.akraino.org/view/kni

Browser not supported