Test Point

Source instrumentation is the process by which developers and domain experts selectively instrument the source under test for the purpose of writing test scenarios against the executing application. Implementing tests that leverage source instrumentation is called Expectation Testing. This validation technique is very useful for verifying proper code sequencing based on the software's internal design.

#include <srtest.h>
...
/* a test point with no payload */
srTEST_POINT("first test point");

/* a test point with binary payload */
srTEST_POINT_DATA("second test point", myData, sizeofMyData);

/* a test point with simple string payload */
srTEST_POINT_STR("third test point", "payload with simple string");

/* a test point with formatted string payload */
srTEST_POINT_STR("third test point", "payload with format string %d", myVar);

#ifdef __cplusplus
srTEST_POINT_STR("c++ test point", "") << "stream input supported under c++";
#endif

Unlike traditional unit testing that drives testing based on input parameters and isolating functionality, Expectation testing is executed within a fully functional software build running on a real target platform. Test Points are not dependent on input parameters, but often leverage the same types of input/output controls used by functional and black-box testing.

Another unique feature of this type of testing is that domain expertise is not required to implement a test. Developers and domain experts use instrumentation to export design knowledge of the software to the entire team. Furthermore, there is no stubbing required, no special logic to generate input parameters, and no advanced knowledge required of how the application software is coded.

To enable effective test coverage, developers and domain experts are required to insert instrumentation at key locations to gain insight and testability. Here are some general suggested source code areas to consider instrumenting:

• entry/exit points of critical functions
• state transitions
• critical or interesting data transitions (using optional payload to convey data values)
• callback routines
• data persistence
• error conditions

The steps required to implement an Expectation test are the following:

1. Instrumentation
2. Define your Expectations
3. Write the Test Unit

Instrumentation

To make the software testable, the first step in the process is for the experts to selectively insert instrumentation macros into the source code. Test Points themselves have nominal impact on the performance of the application – they are only active during test data collection. Test Points contain names and optional payload data. When Test Points are activated, they are collected in the background, along with timing and any associated data. The set of Test Points hit, their order, timing, and data content can all be used to validate that the software is behaving as expected. Test Logs can also be added to the source code to provide additional information in the context of an executing test.

To specify a test point you should include the srtest.h header file from the Stride Runtime in your compilation unit. The Test Point macros are active only when STRIDE_ENABLED is #defined, therefore it is practical to place these macros in-line in production source. When STRIDE_ENABLED is not #defined, these macros evaluate to nothing.

Test Point Macros
srTEST_POINT(label)	label is a pointer to a null-terminated string
srTEST_POINT_DATA(label, data, size)	label is a pointer to a null-terminated string data is a pointer to a byte sequence size is the size of the data in bytes
srTEST_POINT_STR(label, message)	label is a pointer to a null-terminated string message is a pointer to a null-terminated format string ... variable list matching the format string When used in the context of a c++ compilation unit, this macro also supports the streaming operator to append to the message string (see example below)

Define your Expectations

In addition to instrumenting the source code with Test Points, you must also define the Expectations of the Test Points. This involves defining the list of Test Points expected to be hit during a given test scenario. Expectation can also include any data associated with a Test Point that requires validation.

Write the Test Unit

Once the source under test has been instrumented and the Expectations defined, Stride offers a number of techniques that can be used for implementing expectation tests.

• Tests can be written in C or C++ and executed on the device under test using the Stride framework.
• Tests can be written in Perl.