Test Point Testing in C/C++: Difference between revisions
m (Replace STRIDE with Stride) |
|||
(94 intermediate revisions by 5 users not shown) | |||
Line 1: | Line 1: | ||
__NOTOC__ | |||
Test | Test code to validate the [[Expectations]] of the Test Points often follows this basic pattern: | ||
# Specify an expectation set consisting of expected (i.e. the test points that are expected to be hit) and optionally unexpected (i.e. the test points that are not expected to be hit) test points | |||
# Register the expectation set with the Stride runtime | |||
# Invoke the software under test (causing instrumentation points to be hit). This may not be necessary if the instrumented software under test is constantly running). | |||
.. | |||
# Register the expectation set with the | |||
# | |||
# Wait for the expectation set to be satisfied or a timeout to occur | # Wait for the expectation set to be satisfied or a timeout to occur | ||
Line 38: | Line 14: | ||
void tf_testpoint_wait(void) | void tf_testpoint_wait(void) | ||
{ | { | ||
/* specify | /* specify expected set */ | ||
srTestPointExpect_t expected[]= { | srTestPointExpect_t expected[]= { | ||
{"START"}, | {"START"}, | ||
Line 46: | Line 22: | ||
{0}}; | {0}}; | ||
/* register the expectation set | /* specify unexpected set */ | ||
srTestPointUnexpect_t unexpected[]= { | |||
{"INVALID"}, | |||
{0}}; | |||
/* register the expectation set*/ | |||
srWORD handle; | srWORD handle; | ||
srTestPointSetup(expected, unexpected, srTEST_POINT_EXPECT_UNORDERED, srTEST_CASE_DEFAULT, &handle); | |||
/* start your asynchronous operation */ | /* start your asynchronous operation */ | ||
Line 54: | Line 35: | ||
/* wait for expectation set to be satisfied or a timeout to occur */ | /* wait for expectation set to be satisfied or a timeout to occur */ | ||
srTestPointWait(handle, 1000); | |||
} | } | ||
Line 61: | Line 42: | ||
#endif | #endif | ||
</source> | </source> | ||
== Reference == | == Reference == | ||
=== | === Expectation Set === | ||
An expectation set is specified with an [[Expectations#Expected_List|expected]] array of '''srTestPointExpect_t''' structures and a second optional [[Expectations#Unexpected_List|unexpected]] array of '''srTestPointUnexpect_t''' structures. | |||
==== Expected Array ==== | |||
srTestPointExpect_t is typedef'd as follows: | |||
=== | |||
<source lang='c'> | <source lang='c'> | ||
Line 90: | Line 56: | ||
{ | { | ||
/* the label value is considered the test point's identity */ | /* the label value is considered the test point's identity */ | ||
const srCHAR * | const srCHAR * label; | ||
/* count specifies the number of times the test point is expected to be hit */ | /* optional, count specifies the number of times the test point is expected to be hit */ | ||
srDWORD | srDWORD count; | ||
/* data | /* optional, predicate function to use for payload validation against user data */ | ||
srTestPointPredicate_t predicate; | |||
/* optional, user data to validate the payload against */ | |||
void * user; | |||
} srTestPointExpect_t; | } srTestPointExpect_t; | ||
</source> | </source> | ||
Line 100: | Line 68: | ||
'''NOTES:''' | '''NOTES:''' | ||
* The end of the array has to be marked by a srTestPointExpect_t set to all zero values | * The end of the array has to be marked by a srTestPointExpect_t set to all zero values | ||
* The ''count'' and '' | * The ''count'', ''predicate'' and ''user'' members may be omitted in the array declaration (they will be automatically set to 0 by the compiler) | ||
* A ''count'' value of either 0 or 1 is interpreted as 1 | * A ''count'' value of either 0 or 1 is interpreted as 1 | ||
* The ''count'' could be set as "0 or more" by using the special srTEST_POINT_ANY_COUNT symbolic constant | * The ''count'' could be set as "0 or more" by using the [[Expectations#Special_Processing|special]] srTEST_POINT_ANY_COUNT symbolic constant | ||
* A ''data'' value 0 indicates that there is no | * A ''predicate'' value 0 indicates that any associated data with a test point payload will be ignored. | ||
* A ''user'' value 0 indicates that there is no user data associated with this test point | |||
* The ''label'' could be specified to ''any test point'' within the current expected set of test points by using the [[Expectations#Special_Processing|special]] srTEST_POINT_ANY_IN_SET or srTEST_POINT_ANY_AT_ALL symbolic constants. | |||
=== | ==== Unexpected Array ==== | ||
The | srTestPointUnexpect_t is typedef'd as follows: | ||
<source lang='c'> | |||
typedef struct | |||
{ | |||
/* the label value is considered the test point's identity */ | |||
const srCHAR * label; | |||
} srTestPointUnexpect_t; | |||
</source> | |||
'''NOTES:''' | |||
* The end of the array has to be marked by a srTestPointUnexpect_t set to all zero values | |||
* The ''label'' could be specified to ''everything else'' relative to the '''expected''' array by using the [[Expectations#Special_Processing|special]] srTEST_POINT_EVERYTHING_ELSE symbolic constant. | |||
==== srTestPointPredicate_t ==== | |||
When defining the expectation set per entry a [[Expectations#State_Data_Validation|payload validation]] predicate function could be specified. The signature of it should match the following type: | |||
<source lang="c"> | |||
extern "C" typedef srBYTE (*srTestPointPredicate_t)(const srTestPoint_t* ptTP, void* pvUser); | |||
</source> | |||
{| border="1" cellspacing="0" cellpadding="10" style="align:left;" | |||
| '''Parameters''' | |||
| '''Type''' | |||
| '''Description''' | |||
|- | |||
| ptTP | |||
| Input | |||
| Pointer to the currently processed Test Point. | |||
|- | |||
| pvUser | |||
| Input | |||
| Pointer to opaque user data associated with an entry in the expectation set. | |||
|} | |||
<br> | |||
{| border="1" cellspacing="0" cellpadding="10" style="align:left;" | |||
| '''Return Value''' | |||
| ''' Description''' | |||
|- | |||
| srBYTE | |||
| srTRUE for valid, srFALSE for invalid, srIGNORE otherwise. | |||
|} | |||
'''NOTE:''' | |||
* As part of the standard Stride distribution there are three predefined function predicate helpers: | |||
** srTestPointMemCmp - byte comparison | |||
** srTestPointStrCmp - string case sensitive comparison | |||
** srTestPointStrCaseCmp - string case insensitive comparison | |||
==== srTestPointSetup ==== | |||
The srTestPointSetup() routine is used to register an expectation set. | |||
<source lang="c"> | <source lang="c"> | ||
srBOOL | srBOOL srTestPointSetup(srTestPointExpect_t* ptExpected, | ||
srTestPointUnexpect_t* ptUnexpected, | |||
srBYTE yMode, | |||
srTestCaseHandle_t tTestCase, | |||
srWORD* pwHandle); | |||
</source> | </source> | ||
Line 122: | Line 143: | ||
| ptExpected | | ptExpected | ||
| Input | | Input | ||
| Pointer to an | | Pointer to an expectated array. | ||
|- | |||
| ptUnexpected | |||
| Input | |||
| Pointer to an unexpectated array. This is optional and could be set srNULL. | |||
|- | |- | ||
| | | yMode | ||
| Input | | Input | ||
| | | Bitmask that specifies whether the expectated test points occur in [[Expectations#Sequencing_Properties|order and/or strict]]. Possible values are: <br/> | ||
srTEST_POINT_EXPECT_ORDERED - the test points are expected to be hit exactly in the defined order <br/> | srTEST_POINT_EXPECT_ORDERED - the test points are expected to be hit exactly in the defined order <br/> | ||
srTEST_POINT_EXPECT_UNORDERED - the test points could to be hit in any order | srTEST_POINT_EXPECT_UNORDERED - the test points could to be hit in any order <br/> | ||
srTEST_POINT_EXPECT_STRICT - the test points are expected to be hit exactly as specified (no consecutive duplicate hits)<br/> | |||
srTEST_POINT_EXPECT_NONSTRICT - other test points from the universe could to be hit in between <br/> | |||
srTEST_POINT_EXPECT_CONTINUE - on successful expectation satisfaction continue processing until the wait timeout expires | |||
|- | |- | ||
| | | tTestCase | ||
| Input | | Input | ||
| | | Handle to a test case. srTEST_CASE_DEFAULT can be used for the default test case. | ||
|- | |- | ||
| pwHandle | | pwHandle | ||
| Output | | Output | ||
| Handle that represents the | | Handle that represents the registered expectation set | ||
|} | |} | ||
<br> | <br> | ||
Line 149: | Line 175: | ||
|} | |} | ||
=== srTestPointWait === | ==== srTestPointWait ==== | ||
The srTestPointWait() routine is used to wait for the expectation to be satisfied. | The srTestPointWait() routine is used to wait for the expectation to be satisfied. | ||
<source lang="c"> | <source lang="c"> | ||
srBOOL srTestPointWait(srWORD wHandle, | srBOOL srTestPointWait(srWORD wHandle, | ||
srDWORD dwTimeout); | srDWORD dwTimeout); | ||
</source> | </source> | ||
Line 166: | Line 191: | ||
| Input | | Input | ||
| Handle to a registered expectation set. | | Handle to a registered expectation set. | ||
|- | |- | ||
| dwTimeout | | dwTimeout | ||
Line 184: | Line 205: | ||
|} | |} | ||
'''NOTES:''' | |||
* The test thread blocks until either the expectation set is satisfied, unless srTEST_POINT_EXPECT_CONTINUE is specified on setup, or the timeout elapses. | |||
NOTES: | |||
* The test thread blocks until either the expectation set is satisfied or the timeout elapses. | |||
* All test points hit during the wait (both expected and unexpected) are added to the test report as testcase comments | * All test points hit during the wait (both expected and unexpected) are added to the test report as testcase comments | ||
* Once the wait is over (whether the expectation set has been satisfied or there has been a test failure), the current expectation set is automatically unregistered from the Stride runtime and the handle is released | |||
* Once the wait is over (whether the expectation set has been satisfied or there has been a test failure), the current expectation set is automatically unregistered from the | * If you want to return immediately from a test case if expectation fails then make the check/wait call an argument to the ''srASSERT_TRUE()'' macro. | ||
== | ==== srTestPointCheck ==== | ||
The srTestPointCheck() routine is used to check for the expectation post routine completion. This is useful for verifying a set of expectations events that should have already transpired (thus are waiting to be processed). | |||
<source lang="c"> | <source lang="c"> | ||
srBOOL srTestPointCheck(srWORD wHandle); | |||
</source> | </source> | ||
=== | {| border="1" cellspacing="0" cellpadding="10" style="align:left;" | ||
''' | | '''Parameters''' | ||
| '''Type''' | |||
| '''Description''' | |||
|- | |||
| wHandle | |||
| Input | |||
| Handle to a registered expectation set. | |||
|} | |||
<br> | |||
{| border="1" cellspacing="0" cellpadding="10" style="align:left;" | |||
| '''Return Value''' | |||
| ''' Description''' | |||
|- | |||
| srBOOL | |||
| srTRUE on success, srFALSE otherwise. | |||
|} | |||
'''NOTES:''' | |||
* All test points hit before the check (both expected and unexpected) are added to the test report as testcase comments | |||
* Once the check is done (whether the expectation set has been satisfied or there has been a test failure), the current expectation set is automatically unregistered from the Stride runtime and the handle is released | |||
* If you want to return immediately from a test case if expectation fails then make the check/wait call an argument to the ''srASSERT_TRUE()'' macro. | |||
''' | |||
=== C++ Facade Class === | |||
The ''srtest.h'' file provides a simple [http://en.wikipedia.org/wiki/Facade_pattern facade] class that wraps the test point APIs described above in a simple C++ class called '''srTestPointsHandler'''. If you are writing your unit tests in C++ (using Stride test classes), then this class is available for your use. The class implements the following methods, which correspond exactly to the C API equivalents: | |||
[[ | ; srTestPointsHandler : constructor. Takes four arguments which match exactly the first four parameters of the [[#srTestPointSetup|srTestPointSetup]] function. | ||
[[ | ; Wait : instance method that provides same functionality as [[#srTestPointWait|srTestPointWait]]. | ||
; Check : instance method that provides same functionality as [[#srTestPointCheck|srTestPointCheck]]. |
Latest revision as of 21:14, 8 July 2015
Test code to validate the Expectations of the Test Points often follows this basic pattern:
- Specify an expectation set consisting of expected (i.e. the test points that are expected to be hit) and optionally unexpected (i.e. the test points that are not expected to be hit) test points
- Register the expectation set with the Stride runtime
- Invoke the software under test (causing instrumentation points to be hit). This may not be necessary if the instrumented software under test is constantly running).
- Wait for the expectation set to be satisfied or a timeout to occur
Here is an example:
#include <srtest.h>
void tf_testpoint_wait(void)
{
/* specify expected set */
srTestPointExpect_t expected[]= {
{"START"},
{"ACTIVE"},
{"IDLE"},
{"END"},
{0}};
/* specify unexpected set */
srTestPointUnexpect_t unexpected[]= {
{"INVALID"},
{0}};
/* register the expectation set*/
srWORD handle;
srTestPointSetup(expected, unexpected, srTEST_POINT_EXPECT_UNORDERED, srTEST_CASE_DEFAULT, &handle);
/* start your asynchronous operation */
...
/* wait for expectation set to be satisfied or a timeout to occur */
srTestPointWait(handle, 1000);
}
#ifdef _SCL
#pragma scl_test_flist(“testfunc”, tf_testpoint_wait)
#endif
Reference
Expectation Set
An expectation set is specified with an expected array of srTestPointExpect_t structures and a second optional unexpected array of srTestPointUnexpect_t structures.
Expected Array
srTestPointExpect_t is typedef'd as follows:
typedef struct
{
/* the label value is considered the test point's identity */
const srCHAR * label;
/* optional, count specifies the number of times the test point is expected to be hit */
srDWORD count;
/* optional, predicate function to use for payload validation against user data */
srTestPointPredicate_t predicate;
/* optional, user data to validate the payload against */
void * user;
} srTestPointExpect_t;
NOTES:
- The end of the array has to be marked by a srTestPointExpect_t set to all zero values
- The count, predicate and user members may be omitted in the array declaration (they will be automatically set to 0 by the compiler)
- A count value of either 0 or 1 is interpreted as 1
- The count could be set as "0 or more" by using the special srTEST_POINT_ANY_COUNT symbolic constant
- A predicate value 0 indicates that any associated data with a test point payload will be ignored.
- A user value 0 indicates that there is no user data associated with this test point
- The label could be specified to any test point within the current expected set of test points by using the special srTEST_POINT_ANY_IN_SET or srTEST_POINT_ANY_AT_ALL symbolic constants.
Unexpected Array
srTestPointUnexpect_t is typedef'd as follows:
typedef struct
{
/* the label value is considered the test point's identity */
const srCHAR * label;
} srTestPointUnexpect_t;
NOTES:
- The end of the array has to be marked by a srTestPointUnexpect_t set to all zero values
- The label could be specified to everything else relative to the expected array by using the special srTEST_POINT_EVERYTHING_ELSE symbolic constant.
srTestPointPredicate_t
When defining the expectation set per entry a payload validation predicate function could be specified. The signature of it should match the following type:
extern "C" typedef srBYTE (*srTestPointPredicate_t)(const srTestPoint_t* ptTP, void* pvUser);
Parameters | Type | Description |
ptTP | Input | Pointer to the currently processed Test Point. |
pvUser | Input | Pointer to opaque user data associated with an entry in the expectation set. |
Return Value | Description |
srBYTE | srTRUE for valid, srFALSE for invalid, srIGNORE otherwise. |
NOTE:
- As part of the standard Stride distribution there are three predefined function predicate helpers:
- srTestPointMemCmp - byte comparison
- srTestPointStrCmp - string case sensitive comparison
- srTestPointStrCaseCmp - string case insensitive comparison
srTestPointSetup
The srTestPointSetup() routine is used to register an expectation set.
srBOOL srTestPointSetup(srTestPointExpect_t* ptExpected,
srTestPointUnexpect_t* ptUnexpected,
srBYTE yMode,
srTestCaseHandle_t tTestCase,
srWORD* pwHandle);
Parameters | Type | Description |
ptExpected | Input | Pointer to an expectated array. |
ptUnexpected | Input | Pointer to an unexpectated array. This is optional and could be set srNULL. |
yMode | Input | Bitmask that specifies whether the expectated test points occur in order and/or strict. Possible values are: srTEST_POINT_EXPECT_ORDERED - the test points are expected to be hit exactly in the defined order |
tTestCase | Input | Handle to a test case. srTEST_CASE_DEFAULT can be used for the default test case. |
pwHandle | Output | Handle that represents the registered expectation set |
Return Value | Description |
srBOOL | srTRUE on success, srFALSE otherwise. |
srTestPointWait
The srTestPointWait() routine is used to wait for the expectation to be satisfied.
srBOOL srTestPointWait(srWORD wHandle,
srDWORD dwTimeout);
Parameters | Type | Description |
wHandle | Input | Handle to a registered expectation set. |
dwTimeout | Input | Timeout value in milliseconds; 0 means just check without waiting. |
Return Value | Description |
srBOOL | srTRUE on success, srFALSE otherwise. |
NOTES:
- The test thread blocks until either the expectation set is satisfied, unless srTEST_POINT_EXPECT_CONTINUE is specified on setup, or the timeout elapses.
- All test points hit during the wait (both expected and unexpected) are added to the test report as testcase comments
- Once the wait is over (whether the expectation set has been satisfied or there has been a test failure), the current expectation set is automatically unregistered from the Stride runtime and the handle is released
- If you want to return immediately from a test case if expectation fails then make the check/wait call an argument to the srASSERT_TRUE() macro.
srTestPointCheck
The srTestPointCheck() routine is used to check for the expectation post routine completion. This is useful for verifying a set of expectations events that should have already transpired (thus are waiting to be processed).
srBOOL srTestPointCheck(srWORD wHandle);
Parameters | Type | Description |
wHandle | Input | Handle to a registered expectation set. |
Return Value | Description |
srBOOL | srTRUE on success, srFALSE otherwise. |
NOTES:
- All test points hit before the check (both expected and unexpected) are added to the test report as testcase comments
- Once the check is done (whether the expectation set has been satisfied or there has been a test failure), the current expectation set is automatically unregistered from the Stride runtime and the handle is released
- If you want to return immediately from a test case if expectation fails then make the check/wait call an argument to the srASSERT_TRUE() macro.
C++ Facade Class
The srtest.h file provides a simple facade class that wraps the test point APIs described above in a simple C++ class called srTestPointsHandler. If you are writing your unit tests in C++ (using Stride test classes), then this class is available for your use. The class implements the following methods, which correspond exactly to the C API equivalents:
- srTestPointsHandler
- constructor. Takes four arguments which match exactly the first four parameters of the srTestPointSetup function.
- Wait
- instance method that provides same functionality as srTestPointWait.
- Check
- instance method that provides same functionality as srTestPointCheck.