Page Comparison

...

The framework provides tests at different layers of the stack, like hardware, operating system, cloud infrastructure, security, etc. Since the project is constantly evolving, the full list of available tests can be found in the projects repo, where the tests are located under their respective layer. Each layer has its own container image built by the validation project. The full list of images provided can be found in the project’s DockerHub repo.

Getting Start

You can reference how we did bluval testing for the KubeEdge BP in this meeting:

[Akraino TSC] Akraino TSC Meeting (Weekly) - Zoom

Please take a look at the above video starting around 55 minutes.

As a summary, the main reference is:

Bluval User Guide (akraino.org)

There are 2 security related tests: lynis & vuls. And there are 2 k8s related tests: kube-hunter & conformance tests.

The above page shows how to do all the 4 tests in a single framework, i.e Bluval.

I am not sure if you are required to integrate the bluval testing with your Jenkins CI/CD pipeline. I heard from Tina that it’s optional. If you do want to integrate, please refer to this page:

Running bluval in CI

Again we have talked about how we integrated Bluval with CI/CD for the KubeEdge BP in the meeting, you can watch the video recording link.

Here are the steps on a high level:

1. Provision a Jenkins server for CI/CD of your BP
2. Provision a jump server, within which to run all the tests.
3. I suggest you directly download lynis and vuls to run them manually for your SUT (system under test).
4. I also suggest you directly download kube-hunter and sonobuoy to run the tests manually for your k8s cluster, if you have any,
5. Follow the procedure on Bluval User Guide (akraino.org)
6. Upload all your logs to nexus, an example of our uploaded logs are here:

Index of /sites/logs/futurewei/kubeedgees/86 (akraino.org)

The gz files are CI/CD logs from the Jenkins server. All the bluval tests logs are under the results folder.

A few Kube-Hunter fixes can reference here:

KubeEdge BP Test Documents - Akraino - Akraino Confluence

Also, if you ever want to run Vuls directly, you can follow this:

https://vuls.io/docs/en/tutorial-docker.html

Topology

General Requirements

...

ubuntu@jumpserver:~$ ls results/k8s/conformance/
201909110859_sonobuoy_376a4ddc-4498-49fc-af2e-999242c4c245.tar.gz  Conformance.Conformance.log  log.html  output.xml  report.html

Development Environment / Trouble Shooting

These following steps helps you to setup development environment if you want to contribute back to community or trouble shoot the issue for yourself.

...

... On your host eg. for OS layer
ubuntu@jumpserver:~$ docker run --rm -it \ 
  -v /home/ubuntu/validation:/opt/akraino/validation \
  -v /home/ubuntu/results:/opt/akraino/results \
  -v /home/ubuntu/.ssh:/root/.ssh \
  akraino/validation:os-latest /bin/sh

... Within in the container
# cd /opt/akraino/validation && python bluval/bluval.py -l os -o rec

... Its running all the testcases mentioned in bluval-<bluprint_name>.yaml OS layer, and prints commands on screen. One example is here.
Invoking ['robot', '-V', '/opt/akraino/validation/tests/variables_updated.yaml', '-d', '/opt/akraino/results/os/lynis', '-b', 'debug.log', '/opt/akraino/validation/tests/os/lynis']

... You can convert above example as below and run specific test suites
# robot -V /opt/akraino/validation/tests/variables_updated.yaml \
  -d /opt/akraino/results/os/lynis \
  -b debug.log \
  /opt/akraino/validation/tests/os/lynis

... You can add -t "Your testcase name" and run specific testcase, here is the example.
# robot -t "RunLTP syscalls madvise only" -V /opt/akraino/validation/tests/variables_updated.yaml \
  -d /opt/akraino/results/os/ltp \
  -b debug.log \
  -t "RunLTP syscalls madvise only" \
  /opt/akraino/validation/tests/os/ltp

...

Tests are located at /opt/akraino/validation/tests/ and they can be locally modified to print more output.

Common Issues

• FileNotFoundError: [Errno 2] No such file or directory: '/opt/akraino/results/test_info.yaml'

Please take a look at volumes.yaml, results_dir and make sure that entry is correct.

• invalid argument "akraino/validation:blucon-(HEAD" for "-t, --tag" flag: invalid reference format

Please make sure you are not on "detached HEAD". You can use git checkout -b <new-branch-name> to name that branch

The OS layer

TBD

The Hardware layer

...