Test Units Overview

What are STRIDE Test Units?

STRIDE Test Units is a general term for xUnit-style test modules running within the STRIDE runtime framework. These tests--written in C and C++--are compiled and linked with your embedded software and run in-place on your target hardware. They are suitable for both developer unit testing as well as ongoing regression testing.

An external Test Runner is provided which controls the execution of the tests and publishes test results to the local filesystem and optionally to S2's Internet STRIDE Test Space.

Test Unit Features

In all cases, STRIDE Test Units provide the following capabilities typical of all xUnit-style testing frameworks:

• Specification of a test as a test method
• Aggregation of individual tests into test suites which form execution and reporting units
• Specification of expected results within test methods (typically by using one or more Test Macros)
• Test fixturing (optional setup and teardown)
• Automated execution
• Automated results report generation

Unique Test Unit Features

In addition, STRIDE Test Units offer these unique features:

Remote Execution

Execution and reporting controlled from a remote host, thus making the framework useful for on-target embedded system testing

Dynamic Test and Suite Generation

Test cases and suites can be created and manipulated at runtime

Test Doubles

Dynamic runtime function substitution to implement on-the-fly mocks, stubs, and doubles

Asynchronous Testing Framework (Test Points)

Support for testing of asynchronous activities occurring in multiple threads

Multiprocess Testing Framework

Support for testing across multiple processes running simultaneously on the target

Automatic Timing Data Collection

Automatic "time under test" collection

Automatic Results Publishing to Local Disk and Internet

Automatic publishing of test results to STRIDE Test Space

Test Unit Deployment

Individual Tests

Individual test are implemented as test functions or methods which follow a four-phase testing pattern:

1. Setting up a test fixture (optional)
2. Exercising the System Under Test (SUT)
3. Verifying that the expected outcome has occurred (typically using calls to Assertion or Expectation Macros)
4. Tearing down the test fixture (optional)

Test Units

Individual functions or methods, which typically implement a single test case are grouped into one or more Test Units which are executed as atomic entities.

Grouping of individual tests into a Test Unit can be accomplished in any of three ways:

• A Test Unit can be comprised of the member functions of a C++ class,
• A Test Unit can be comprised of a set of C functions,
• A Test Unit can be comprised of C functions pointed to by members of a C struct

The best choice is usually the C++ class since it offers the best mix of features and ease-of-use. (You can test code written in C or C++ using the C++ class test units.) However, compiling C++ is not always possible, in this case one of the C-based test unit packaging options must be used.

You can freely mix different deployment methods across a project if desired, the format of the results is consistent across all test unit packaging options.

Following are a few short examples.

Simple example using srTest base class

#include <srtest.h>
  
class MyTest : public stride::srTest 
{
public:
  void ExpectPass() 
  {
    srLOG_INFO("this test should pass");
    srEXPECT_EQ(2 + 2, 4); 
  }
  void ExpectFail() 
  {
    srLOG_INFO("this test should fail");
    srEXPECT_GT(2 * 3, 7); 
  }
  int ChangeMyName() 
  {
    srLOG_INFO("this test should have name = MyChangedName");
    testCase.SetName("MyChangedName");
    return 0;
  }
  int ChangeMyDescription() 
  {
    srLOG_INFO("this test should have a description set");
    testCase.SetDescription("this is my new description");
    return 0;
  }
};

#ifdef _SCL
// this pragma identifies MyTest as a test class to the STRIDE compiler
#pragma scl_test_class(MyTest)
#endif

Deployment as C Classes

typdef struct MyTest
{
    void (*ExpectPass)(struct MyTest* self);
    void (*ExpectFail)(struct MyTest* self);
    int (*ChangeMyName)(struct MyTest* self);
    int (*ChangeMyDescription)(struct MyTest* self);
} MyTest;

void MyTest_Init(MyTest* self);

#ifdef _SCL
#pragma scl_test_cclass(MyTest, MyTest_Init)
#endif

static void ExpectPass(MyTest* self)
{
    srLOG_INFO("this test should pass");
    srEXPECT_EQ(2 + 2, 4); 
}
static void ExpectFail(MyTest* self)
{
    srLOG_INFO("this test should fail");
    srEXPECT_GT(2 * 3, 7); 
}
static int ChangeMyName(MyTest* self)
{
    srLOG_INFO("this test should have name = MyChangedName");
    srTestCaseSetName(srTEST_CASE_DEFAULT, "MyChangedName");
    return 0;
}
static int ChangeMyDescription(MyTest* self)
{
    srLOG_INFO("this test should have a description set");
    srTestCaseSetDescription(srTEST_CASE_DEFAULT, "this is my new description");
    return 0;
}
void MyTest_Init(MyTest* self)
{
    self->ExpectPass = ExpectPass;
    self->ExpectFail = ExpectFail;
    self->ChangeMyName = ChangeMyName;
    self->ChangeMyDescription = ChangeMyDescription;
}

Deployment C Functions

void ExpectPass();
void ExpectFail();
int ChangeMyName();
int ChangeMyDescription();

#ifdef _SCL
#pragma scl_test_flist("MyTest", ExpectPass, ExpectFail, ChangeMyName, ChangeMyDescription)
#endif

void ExpectPass()
{
    srLOG_INFO("this test should pass");
    srEXPECT_EQ(2 + 2, 4); 
}
void ExpectFail()
{
    srLOG_INFO("this test should fail");
    srEXPECT_GT(2 * 3, 7); 
}
int ChangeMyName()
{
    srLOG_INFO("this test should have name = MyChangedName");
    srTestCaseSetName(srTEST_CASE_DEFAULT, "MyChangedName");
    return 0;
}
int ChangeMyDescription()
{
    srLOG_INFO("this test should have a description set");
    srTestCaseSetDescription(srTEST_CASE_DEFAULT, "this is my new description");
    return 0;
}

Build and generate the IM code using STRIDE Build Tools:

> s2scompile --c++ testcpp.h
> s2scompile --c testc.h
> s2sbind --output=test.sidb testcpp.h.meta testc.h.meta
> s2sinstrument --im_name=test test.sidb

If using STRIDE Studio, create a new workspace (or open an existing one), add the above source files, adjust your compiler settings, build and generate the IM manually through the UI, or write custom scripts to automate the same sequence.

Build the generate IM code along with the rest of the source to create your application's binary.

Download your application to the Target and start it.

Execute your test units and publish results using the Test Unit Runner.

> perl testunitrun.pl -u -d test.sidb

If using STRIDE Studio, you can execute individual test units interactively by opening the user interface view corresponding to the test unit you would like to execute, then call it. Further more you may write a simple script to automate your test units execution and result publishing.

Requirements

Several variations on typical xUnit-style test units are supported. The additional supported features include:

• Test status can be set using STRIDE Runtime APIs or by specifying simple return types for test methods.
• Integral return types: 0 = PASS; <> 0 = FAIL
• C++ bool return type: true = PASS; false = FAIL
• void return type with no explict status setting is assumed PASS
• Test writers can create additional child suites and tests at runtime by using Runtime APIs.
• We do not rely on exceptions for reporting of status.
• One of the Test Unit pragmas must be applied.

The STRIDE test class framework has the following requirements of each test class:

• The test class must have a suitable default (no-argument) constructor.
• The test class must have one or more public methods suitable as test methods. Allowable test methods always take no arguments (void) and return either void, simple integer types (int, short, long or char) or bool. At this time, we do not allow typedef types or macros for the return values specification.
• The scl_test_class pragma must be applied to the class.

Using a Test Function List

#include <srtest.h>
  
#ifdef __cplusplus
extern "C" {
#endif
  
int tf_Int_ExpectPass(void) {return 0;}
int tf_Int_ExpectFail(void) {return -1;}

#ifdef _SCL
#pragma scl_test_flist("Simple", tf_Int_ExpectPass, tf_Int_ExpectFail)
#endif
   
#ifdef __cplusplus
}
#endif

Using Testpoints

Testpoints are described in the article Using Testpoints.