Contents

Introduction

Akraino uses Tempest for end-to-end testing. Tempest is a group of black box functional/integration tests run against a live OpenStack cluster. Tempest tests cover aspects such as API validation, scenarios, and anything else useful to help validate an OpenStack deployment.

Tempest Flow Diagram

Test Automation

The test_automation project is from Akraino Gerrit. It is to be cloned into /opt directory on the designated test node.  For cloning 'git clone' command can be used and it is shown in the box below.

For the following steps, substitute appropriate values for the OpenStack username, password, region name, and desired timeout ( 1600 Seconds).

$ cd /opt
$ sudo git clone http://gerrit.akraino.org/test_automation.git
$ cd test_automation/openstack-tempest
$ sudo vi tempest/values.yaml # Edit candidate/blacklisted tests as needed
$ OS_USERNAME=test OS_PASSWORD=xyz OS_REGION_NAME=NewRegion ./test_run.sh
$ TIMEOUT=1600 ./test_status.sh # optional flag: --no-delete (use for debugging)

test_run.sh runs the Akraino Tempest Helm package. It performs the following actions:

1. Set default values
2. Validate OpenStack settings
3. Delete any partially deleted instances
4. Clean previous runs, if any, if Tempest is running
5. Install tempest as a pod on a given cluster, also via Helm charts
6. Run Tempest tests
7. Return 0 on success, 1 on error

Prior to running the Helm package, edit values.yaml as needed. Observe that the conf field's script key determines which group of tests are run, as well as any blacklisted tests. Unresolvable failed tests should be added to the blacklist.

test_status.sh checks the status of the Tempest Helm package and collects test results. It performs the following actions:

1. Set default values
2. Check for Tempest pod existence
3. Check pod status frequently
4. Wait for pod to complete, and abort if pod does not complete
5. Save logs, e.g., the default: /tmp/results.txt
6. Capture and display test results
7. Delete Tempest deployment unless --no-delete is specified for debugging purposes
8. Return 0 on success, 1 on error

Additional Test Tools

• Robot CLI: ONAP end-to-end testing
• OpenStack Shaker: Tenant networking performance
• OPNFV Yardstick: Infrastructure verification
• Heptio Sonobuoy: End-to-end K8s conformance-testing, workload debugging, and data collection