Test Unit

Test Unit

A Stride Test Unit (also known as a Test Suite) is a general term for xUnit-style test cases running within the Stride runtime framework. These tests are written in C and C++ and are compiled and linked with your software and run, when called, on your target platform. They are suitable for both developer unit testing as well as end-to-end integration testing. An external Test Runner is provided which controls the execution of the tests and publishes test results to the local file system and optionally to Testspace.

A single Test Unit is a set of Test Methods that always run together as an executable unit. Test Methods are the test cases that comprise a Test Unit. Each Test Method by default maps to a single Test Case in the results. When relying on this default behavior of Test Methods, we refer to these methods as static Test Cases. A less common technique is to create and add a Test Case dynamically at runtime to an existing Suite via the Test Services. We refer to these as dynamic Test Cases.

Individual test cases are implemented as test methods or functions which follow a four-phase testing pattern:

1. Setting up a test fixture (optional)
2. Exercising the Software Under Test (SUT)
3. Verifying that the expected outcome has occurred
4. Tearing down the test fixture (optional)

Test fixturing refers to the Setup and Teardown phases of the testing.

In the Setup phase, we put all of the things into place that are required in order to run a our test and expect a particular outcome. This includes things like:

• Acquiring resources such as memory, hardware, etc.
• Setting up required states such as input files in place, memory filled with a pattern, dependencies initialized, etc.

In the Tear down phase, we clean up the fixturing we did in the Setup phase, leaving the system in a state that is ready to be used by the next test.

Packaging

Individual functions or methods, which typically implement a single test case are grouped into one or more Test Units which are executed as atomic entities.

Grouping of individual tests into a Test Unit can be accomplished in any of three ways:

• A Test Unit can be comprised of the member functions of a C++ class,
• A Test Unit can be comprised of a set of C functions,
• A Test Unit can be comprised of C functions pointed to by members of a C struct

The following table compares the three packaging variants.

Comparison of Test Unit Packaging

Test Unit Type	Advantages	Disadvantages	scl pragma
C++ class Example code	Simple and easy to use Can be used with target code that is all C, all C++, or a mix Test utility methods can be encapsulated as test class members or members of a parent class Could be parametrized via constructor arguments Provides convenient syntax for augmenting report annotation (operator <<)	Requires a C++ build environment	scl_test_class
C class Example code	Provides encapsulation of test methods Provides encapsulation of state via member variables Could be parametrized via init-function arguments	Requires initialization code for structure function pointer setup Test unit documentation must be associated with the structure function pointer members instead of the actual test method implementations.	scl_test_cclass
C functions Example code	Extremely simple syntax	Parametrized test units are not supported No test unit support for constructor/initializer or destructor/de-initializer Changing test unit membership requires editing of the pragma statement Test unit documentation for the FList must be associated with the header file - as such, you can only have one FList per header file if you want to document your test units.	scl_test_flist

The best choice is usually the C++ class since it offers the best mix of features and ease-of-use. (You can test code written in C or C++ using the C++ class test units.) However, compiling C++ is not always possible, in this case one of the C-based test unit packaging options must be used.

You can freely mix different deployment methods across a project if desired, the format of the results is consistent across all test unit packaging options.

Test Method Return Values

Test Method return values must conform to certain return types as summarizing in the following table.

Return Type	Description	How Return Value is Interpreted	Default Status
void	Most common return type	Since there is no return, PASS/FAIL status is set in the body of the method using a test macro or a runtime call srTestCaseSetStatus() srTestCaseSetStatusEx() srTestCase::SetStatus()	If the method runs to completion without another status being set, the status is srTEST_PASS If the method does not run to completion (due to crash or hang in software under test), the status is srTEST_INPROGRESS
integer type	Integer types include: int short char long may include signed/unsigned/const qualifiers	If you explicitly set the status in the method (using a test macro or a runtime call), this status will override the status that would otherwise be set by the return value If you don't explicitly set the status in the method, the return value determines the status as follows: Return is zero -> srTEST_PASS Return is non-zero -> srTEST_FAIL	If the method runs to completion, the status is determined by the rules to the left (there is no default) If the method does not run to completion (due to crash or hang in software under test), the status is srTEST_INPROGRESS
bool	c++ only	If you explicitly set the status in the method (using a test macro or a runtime call), this status will override the status that would otherwise be set by the return value If you don't explicitly set the status in the method, the return value determines the status as follows: Return is true -> srTEST_PASS Return is false -> srTEST_FAIL	If the method runs to completion, the status is determined by the rules to the left (there is no default) If the method does not run to completion (due to crash or hang in software under test), the status is srTEST_INPROGRESS

Examples

Following are a few short examples. In each example, a single test unit with the name "MyTest" is identified to the STRIDE compiler via a custom special test pragma.

C++ Class

MyTest.h

#include <srtest.h>
  
class MyTest : public stride::srTest 
{
public:
  void ExpectPass() 
  {
    srNOTE_INFO("this test should pass");
    srEXPECT_EQ(2 + 2, 4); 
  }
  void ExpectFail() 
  {
    srNOTE_INFO("this test should fail");
    srEXPECT_GT(2 * 3, 7); 
  }
};

#ifdef _SCL
// this pragma  identifies MyTest as a test class to the STRIDE compiler
#pragma scl_test_class(MyTest)
#endif

C Class

MyTest.h

#include <srtest.h>
  

typedef struct MyTest
{
    void (*ExpectPass)(struct MyTest* self);
    void (*ExpectFail)(struct MyTest* self);
} MyTest;

void MyTest_Init(MyTest* self);

#ifdef _SCL
//  This pragma identifies MyTest as a test c class to the STRIDE compiler.
//  Extra instrumentation code will be generated to call MyTest_Init()  before
// tests are run.
#pragma scl_test_cclass(MyTest, MyTest_Init)
#endif

MyTest.c

#include  "MyTest.h"

static void ExpectPass(MyTest* self)
{
    srNOTE_INFO("this test should pass");
    srEXPECT_EQ(2 + 2, 4); 
}
static void ExpectFail(MyTest* self)
{
    srNOTE_INFO("this test should fail");
    srEXPECT_GT(2 * 3, 7); 
}
void MyTest_Init(MyTest* self)
{
    self->ExpectPass = ExpectPass;
    self->ExpectFail = ExpectFail;
}

Free Functions

MyTest.h

#include <srtest.h>
void ExpectPass();
void ExpectFail();

#ifdef _SCL
//  this pragma identifies MyTest as a test unit to the STRIDE compiler,  specifying
// the four functions as members of the test unit
#pragma scl_test_flist("MyTest", ExpectPass, ExpectFail, ChangeMyName, ChangeMyDescription)
#endif

MyTest.c

#include "MyTest.h"

void ExpectPass()
{
    srNOTE_INFO("this test should pass");
    srEXPECT_EQ(2 + 2, 4); 
}
void ExpectFail()
{
    srNOTE_INFO("this test should fail");
    srEXPECT_GT(2 * 3, 7); 
}

C++ Only Features

Use Operator << to Augment Test Macros

In C++ test code Test Point, Test Log and Test Code macros support adding to the annotation using the << operator. For example:

srEXPECT_TRUE(a != b) << "My custom message" << " with more data " << 1234;

As delivered, the macros will support stream input annotations for:

• all numeric types,
• C string (char* or wchar_t*), and
• types allowing implicit cast to numeric type or "C" string.

Overloading the << operator

You can also overload the << operator in order to annotate reports using your own custom type. An example is below.

The following will compile and execute successfully given that the << operator is overloaded as shown:

#include <srtest.h>

// MyCustomClass implementation
class MyCustomClass
{
public:
   MyCustomClass(int i) : m_int(i) {}

private: 
   int m_int; 
   friend stride::Message& operator<<(stride::Message& ss, const MyCustomClass& obj);
}; 

stride::Message& operator<<(stride::Message& ss, const MyCustomClass& obj)
{
   ss << obj.m_int;
   return ss;
}

void test()
{
    MyCustomClass custom(34); 

    srEXPECT_FALSE(true) << custom;
}