Test Units

Test Unit Packaging

Individual Tests

Individual test are implemented as test functions or methods which follow a four-phase testing pattern:

1. Setting up a test fixture (optional)
2. Exercising the System Under Test (SUT)
3. Verifying that the expected outcome has occurred (typically using calls to Test Macros or Test Points)
4. Tearing down the test fixture (optional)

Test Units

Individual functions or methods, which typically implement a single test case are grouped into one or more Test Units which are executed as atomic entities.

Grouping of individual tests into a Test Unit can be accomplished in any of three ways:

• A Test Unit can be comprised of the member functions of a C++ class,
• A Test Unit can be comprised of a set of C functions,
• A Test Unit can be comprised of C functions pointed to by members of a C struct

The following table compares the three packaging variants.

Comparison of Test Unit Packaging

Test Unit Type	Advantages	Disadvantages	scl pragma
C++ class Example code Sample tests	Simple and easy to use Can be used with target code that is all C, all C++, or a mix Test utility methods can be encapsulated as test class members or members of a parent class Could be parametrized via constructor arguments Provides convenient syntax for augmenting report annotation (operator <<)	Requires a C++ build environment	scl_test_class
C class Example code Sample tests	Provides encapsulation of test methods Provides encapsulation of state via member variables Could be parametrized via init-function arguments	Requires initialization code for structure function pointer setup Test unit documentation must be associated with the structure function pointer members instead of the actual test method implementations.	scl_test_cclass
C functions Example code Sample tests	Extremely simple syntax	Parametrized test units are not supported No test unit support for constructor/initializer or destructor/de-initializer Changing test unit membership requires editing of the pragma statement Test unit documentation for the FList must be associated with the header file - as such, you can only have one FList per header file if you want to document your test units.	scl_test_flist

The best choice is usually the C++ class since it offers the best mix of features and ease-of-use. (You can test code written in C or C++ using the C++ class test units.) However, compiling C++ is not always possible, in this case one of the C-based test unit packaging options must be used.

You can freely mix different deployment methods across a project if desired, the format of the results is consistent across all test unit packaging options.

Recommendations

STRIDE supports a heterogeneous mix of test unit types, and it's not difficult to refactor the packaging style of a set of tests. However as you plan your testing strategy and tactics, keep these recommendations in mind:

• Where possible, use the C++ class packaging. This provides the most developer benefits and is easiest to use.

• If the C++ packaging is not possible, consider the C class packaging over the function list packaging. The C Class syntax is more involved (you have to explicitly carry around the "this" pointer to the structure), but the benefits of automatic fixturing and state encapsulation outweigh the heavier syntax.

All of the STRIDE pragmas for specifying test units are described here

Results Reporting Model

The STRIDE Framework organizes your testing activity in containers defined by Test Units made up of one or more Test Methods (or Functions). By default, each Test Method corresponds to a single Test Case, although test methods can also be used to create dynamic test cases using the STRIDE Runtime.

An understanding of these entities and the role that each plays is essential to creating and maintaining useful test assets.

Test Methods

Test methods are the test cases that comprise a Test Unit. When using Test Classes (c++), these are just all public functions that are not otherwise declared as fixture methods. For Test C-Class and Test FList units, the methods are similarly bound by membership in the structure (C-Class) or by explicit inclusion in the scl pragma declaration (FList).

Each Test Method by default maps to a single Test Case in the results. When relying on this default behavior of Test Methods, we refer to these methods as static Test Cases.

A less common technique is to create and add a test case dynamically at runtime to an existing Suite via the STRIDE API. We refer to these as dynamic Test Cases.

In the STRIDE Framework, every test case belongs to exactly one Test Unit. Less commonly, a Test Case may belong to a dynamically-created Suite if the Test Case is itself dynamically created. Further, each test case must have a Name and a Status, which are expressed as attributes of the test case. STRIDE assigns default values to these attributes, which may be changed at runtime. Other optional attributes can be set on the test case as well as shown in the table below.

Typically, you will set Test Case attributes via STRIDE Macros that simplify the interaction with the Test Case, the most common of these are Test Code Macros - to set the Test Case status.

Specifying Pass/Fail

There are several ways to indicate pass/fail status in a test method. The most common is to use a Stride test macro, but other options are available. See Indicating Pass/Fail in a Test Method for more details.

Test Units

A Test Unit is a set of Test Cases that is always run together. If fixturing is used, all Test Cases within a Test Unit also share the same fixturing methods. A Test Unit typically holds a set of logically-related tests that target the testing of a specific component or subsystem.

For a C++ class-based or C class-based Test Unit, the Test Unit maps directly to a source code entity. For the free function-based Test Unit, the Test Unit is synthesized by the STRIDE Runtime.

Following are the types of Test Units supported by STRIDE.

C++ Class

If a class is declared a scl_test_class all member functions of the class that meet certain criteria become member Test Cases of the Test Unit.

C Class (using a structure)

If a structure is declared a scl_test_cclass all function pointer members of the structure become member Test Cases of the Test Unit. (The structure must meet certain criteria to be a Test Unit).

Free Function

One or more free functions that meet certain criteria can be grouped into a synthesized Test Unit using the scl_test_flist pragma.

At runtime, STRIDE creates a Suite for each Test Unit that is run, thus providing a Suite context for each member Test Case.

For more on the internal reporting model, see here

Parameterized Test Units

You can pass parameters to Test Units using a command line option of the Runner. This allows you to provide input to a test during runtime.

For more on passing parameters to Test Units, see here

Simple Test Unit Examples

Following are a few short examples. In each example, a single test unit with the name "MyTest" is identified to the STRIDE compiler via a custom special test pragma.

Test Unit as C++ Class

MyTest.h

#include <srtest.h>
  
class MyTest : public stride::srTest 
{
public:
  void ExpectPass() 
  {
    srNOTE_INFO("this test should pass");
    srEXPECT_EQ(2 + 2, 4); 
  }
  void ExpectFail() 
  {
    srNOTE_INFO("this test should fail");
    srEXPECT_GT(2 * 3, 7); 
  }
};

#ifdef _SCL
// this pragma  identifies MyTest as a test class to the STRIDE compiler
#pragma scl_test_class(MyTest)
#endif

Test Unit as C Class

MyTest.h

#include <srtest.h>
  

typedef struct MyTest
{
    void (*ExpectPass)(struct MyTest* self);
    void (*ExpectFail)(struct MyTest* self);
} MyTest;

void MyTest_Init(MyTest* self);

#ifdef _SCL
//  This pragma identifies MyTest as a test c class to the STRIDE compiler.
//  Extra instrumentation code will be generated to call MyTest_Init()  before
// tests are run.
#pragma scl_test_cclass(MyTest, MyTest_Init)
#endif

MyTest.c

#include  "MyTest.h"

static void ExpectPass(MyTest* self)
{
    srNOTE_INFO("this test should pass");
    srEXPECT_EQ(2 + 2, 4); 
}
static void ExpectFail(MyTest* self)
{
    srNOTE_INFO("this test should fail");
    srEXPECT_GT(2 * 3, 7); 
}
void MyTest_Init(MyTest* self)
{
    self->ExpectPass = ExpectPass;
    self->ExpectFail = ExpectFail;
}

Test Unit as Group of Free Functions

MyTest.h

#include <srtest.h>
void ExpectPass();
void ExpectFail();

#ifdef _SCL
//  this pragma identifies MyTest as a test unit to the STRIDE compiler,  specifying
// the four functions as members of the test unit
#pragma scl_test_flist("MyTest", ExpectPass, ExpectFail, ChangeMyName, ChangeMyDescription)
#endif

MyTest.c

#include "MyTest.h"

void ExpectPass()
{
    srNOTE_INFO("this test should pass");
    srEXPECT_EQ(2 + 2, 4); 
}
void ExpectFail()
{
    srNOTE_INFO("this test should fail");
    srEXPECT_GT(2 * 3, 7); 
}

Integrating Test Units Into Your Target Build

STRIDE Test Units are easily integrated into your target build since all required test harnessing code is automatically generated based on header files that include STRIDE Test pragmas.

This harnessing code (referred to as Intercept Module, or IM code) is responsible for

• Communicating with the I/O portion of the STRIDE target runtime
• Instantiating each specified Test Unit
• Running each member test of the Test Unit
• Collecting test output

Harnessing code generation is the responsibility of the STRIDE Build Tools. The useful artifacts created by the build tools comprise the STRIDE database (xx.sidb), and the IM source (strideIM.c/cpp, strideIM.h, and strideIMEntry.h).

To build a fully-instrumented target:

• Several statements are added to your applications main() function (or equivalent) to start and stop the STRIDE I/O and IM threads
• The generated IM source files are compiled and linked with your target application
• The STRIDE library (which provides I/O and common services) is also linked with your application.

Running Target-Based Tests

After configuring TCP/IP or Serial port communication parameters, tests are controlled and run from a remote host computer. See Stride Runner for details.