I-VICS R4 Test Document

Introduction

<Details about Additional tests required for this Blue Print in addition to the Akraino Validation Feature Project>

Akarino Test Group Information

<The Testing Ecosystem>

Testing Working Group Resources

Overall Test Architecture

A typical integration test has three components:

1. A series of executables with arguments that work together and generate outputs
2. A series of expected outputs that should match the output of the executables
3. A launcher that starts the tests, compares the outputs to the expected outputs, and determines if the test passes

A simple example

The simple example consists of a talker node and a listener node. Code of the talker and listener is found in the ROS 2 demos repository.

The talker sends messages with an incrementing index which, ideally, are consumed by the listener.

The integration test for the talker and listener consist of the three aforementioned components:

1. Executables: talker and listener
2. Expected outputs: two regular expressions that are expected to be found in the stdout of the executables:
   • ‘Publishing: 'Hello World: ...’for thetalker-I heard: [Hello World: ...]for thelistener`
3. Launcher: a launching script in python that invokes executables and checks for the expected output

The launcher starts talker and listener in sequence and periodically checks the outputs. If the regex patterns are found in the output, the launcher exits with a return code of 0 and marks the test as successful. Otherwise, the integration test returns non-zero and is marked as a failure.

The sequence of executables can make a difference during integration testing. If exe_b depends on resources created by exe_a, the launcher must start executables in the correct order.

Some nodes are designed to run indefinitely, so the launcher is able to terminate the executables when all the output patterns are satisfied, or after a certain amount of time. Otherwise, the executables have to use a runtime argument.

Test Bed

Test Framework

This section provides examples for how to use the integration_tests framework. The architecture of integration_tests framework is shown in the diagram below.

integration_test architecture

Traffic Generator

Test API description

<Akraino common tests>

The Test inputs

Test Procedure

Expected output

Test Results

<Blueprint extension tests>

The Test inputs

Test Procedure

Expected output

Test Results

<Feature Project Tests>

The Test inputs

Test Procedure

Expected output

Test Results

Test Dashboards

Single pane view of how the test score looks like for the Blue print.

Additional Testing

Bottlenecks/Errata