Runtime Test Services: Difference between revisions
Jump to navigation
Jump to search
(→C) |
(→Notes) |
||
| Line 1,218: | Line 1,218: | ||
#endif | #endif | ||
</source> | </source> | ||
[[Category:Test Units]] | [[Category:Test Units]] | ||
Revision as of 00:12, 2 August 2014
Runtime Test Services
The Runtime Test Services (declared in srtest.h) are a set of APIs in the STRIDE Runtime that facilitate the writing of target based test code. These APIs make up an optional portion of the STRIDE Runtime and can be used to communicate additional information about tests to the host based reporting mechanism. These APIs also allow target test code to create additional test suites and test cases dynamically at runtime.
There are two ways of accessing these APIs: through C-based functions, or through a C++ base class from which you may derive your C++ test class. These are discussed below in C Test Functions, and C++ Test Classes sections below.
Refer to the SCL Reference Guide or the Runtime Developer's Guide for more detailed information about any of these APIs.
C Test Functions
The following C APIs are provided:
- srTestGetParam: returns an input parameters value.
- srTestSuiteAddCase: creates an additional test case at runtime.
- srTestSuiteAddAnnotation: creates a suite annotation at runtime.
- srTestSuiteSetData: associates a custom name|value.
- srTestCaseSetStatus: explicitly sets the status for the specified test case.
- srTestCaseAddAnnotation: creates a test case annotation at runtime.
- srTestSuiteSetData: associates a custom name|value pair with the specified test case.
- srTestAnnotationAddComment: adds a comment to the specified annotation.
C
srTestGetParam
The srTestGetParam() function is used to get the value of a named parameter as a string.
srWORD srTestGetParam(const srCHAR * szName, srCHAR * szValue, srDWORD dwSize, const srCHAR * szDefValue);
| Parameters | Type | Description |
| szName | Input | Pointer to a null-terminated string that represents the name of the parameter. Cannot be null. |
| szValue | Output | Pointer to a block of memory to store the value of the parameter with a maximum size of dwSize chars |
| dwSize | Input | Size in chars of the buffer pointed to by szValue |
| szDefValue | Input (optional) | Pointer to a null-terminated string that represents a default value in case the parameter is not specified. By setting this value to srNULL (or omitting it), you can use srTestGetParam() to test for the existence of a named parameter. If the named parameter doesn't exist, the function will return srERR_HANDLE_INVALID. |
| Return Value | Description |
| srOK | Success |
| srERR | Null or empty szName string passed in. |
| srERR_HANDLE_INVALID | The named parameter is not found and szDefValue is set to srNULL. |
#include <srtest.h>
#define MAX_VALUE_LEN 128
void tfsuite_getParam(void)
{
srCHAR szValue[MAX_VALUE_LEN] = {0};
srWORD wRet = srTestGetParam("ParamName", szValue, MAX_VALUE_LEN, "default value");
}
#ifdef _SCL
#pragma scl_test_flist("testfunc", tfsuite_getParam)
#endif
srTestGetParamLong
The srTestGetParamLong() function is used to get the value of a named parameter as a long.
srLONG srTestGetParamLong(const srCHAR * szName, srLONG lDefValue);
| Parameters | Type | Description |
| szName | Input | Pointer to a null-terminated string that represents the name of the parameter. |
| lDefValue | Input | Default value that is returned if the parameter is not found. |
| Return Value | Description |
| srLONG | Parameter value, or value specified by lDefValue if parameter isn't found. |
#include <srtest.h>
void tfsuite_getParam(void)
{
srLONG lVal = srTestGetParamLong("ParamName", -1);
}
#ifdef _SCL
#pragma scl_test_flist("testfunc", tfsuite_getParam)
#endif
srTestGetParamDouble
The srTestGetParamDouble() function is used to get the value of a named parameter as a double.
double srTestGetParamDouble(const srCHAR * szName, double dbDefValue);
| Parameters | Type | Description |
| szName | Input | Pointer to a null-terminated string that represents the name of the parameter. |
| dbDefValue | Input | Default value that is returned if the parameter is not found. |
| Return Value | Description |
| double | Parameter value, or value specified by dbDefValue if parameter isn't found. |
#include <srtest.h>
void tfsuite_getParam(void)
{
double dbVal = srTestGetParamDouble("ParamName", 0.5772156649);
}
#ifdef _SCL
#pragma scl_test_flist("testfunc", tfsuite_getParam)
#endif
srTestSuiteAddCase
The srTestSuiteAddCase() routine is used to add a new test case to the specified test suite.
srTestCaseHandle_t srTestSuiteAddCase(const srCHAR * szName, const srCHAR * szDescr)
| Parameters | Type | Description |
| szName | Input | Pointer to a null-terminated string that represents the name of the new test case. If null, the default host naming scheme will be used. |
| szDescr | Input | Pointer to a null-terminated string representing the description of the new test case. If null, description will be empty. |
| Return Value | Description |
| srTestCaseHandle_t | Handle of the newly created test case. srTEST_CASE_INVALID indicates failure to create a test case. |
#include <srtest.h>
#include <stdio.h>
void tfsuite_addCase(void)
{
for (int count = 0; count < 5; ++count)
{
char szName[25];
sprintf(szName, "dynamic test case %d", count);
srTestCaseHandle_t test = srTestSuiteAddCase(szName, "this is a dynamic test case");
srTestCaseSetStatus(test, srTEST_PASS, 0);
}
}
#ifdef _SCL
#pragma scl_test_flist("testfunc", tfsuite_addCase)
#endif
srTestSuiteAddAnnotation
The srTestSuiteAddAnnotation() routine is used to add a new annotation to the specified test suite.
srTestAnnotationHandle_t srTestSuiteAddAnnotation(srTestAnnotationLevel_e eLevel,
const srCHAR * szName,
const srCHAR * szFmtDescr, ...)
| Parameters | Type | Description |
| eLevel | Input | Annotation level.
|
| szName | Input | Pointer to a null-terminated string representing the name of the new annotation. If null, the default host naming scheme will be used. |
| szFmtDescr | Input | Pointer to a null-terminated string representing the description of the new annotation. Cannot be null. |
| ... | Input (Optional) | Variable argument list to format szFmtDescr. |
| Return Value | Description |
| srTestAnnotationHandle_t | Handle of the newly created test annotation. srTEST_ANNOTATION_INVALID indicates failure to create a test annotation. |
#include <srtest.h>
void tfsuite_addAnnotation(void)
{
for (int count = 0; count < 5; ++count)
{
char szName[25];
sprintf(szName, "annotation %d", count);
srTestAnnotationHandle_t annot =
srTestSuiteAddAnnotation(srTEST_ANNOTATION_LEVEL_ERROR,
szName,
"description of annotation");
}
}
#ifdef _SCL
#pragma scl_test_flist("testfunc", tfsuite_addAnnotation)
#endif
srTestSuiteSetData
The srTestSuiteSetData() routine is used to associate a custom name|value pair with a test suite.
srWORD srTestSuiteSetData(const srCHAR * szName, const srCHAR * szFmtValue, ...)
| Parameters | Type | Description |
| szName | Input | Pointer to a null-terminated string representing the name of the custom pair. Cannot be null. |
| szFmtValue | Input | Pointer to a null-terminated string representing the value of the custom pair. Cannot be null. |
| ... | Input (Optional) | Variable argument list to format szFmtValue. |
| Return Value | Description |
| srWORD | srOK on success, srERR_HANDLE_INVALID on invalid handle. |
#include <srtest.h>
void tfcase_setData(void)
{
srTestSuiteSetData("MyName", "my value");
}
#ifdef _SCL
#pragma scl_test_flist("testfunc", tfcase_setData)
#endif
srTestCaseSetStatus
The srTestCaseSetStatus() routine is used to set the result of test case.
srWORD srTestCaseSetStatus(srTestCaseHandle_t tTestCase, srTestStatus_e eStatus, srDWORD dwDuration)
| Parameters | Type | Description |
| tTestCase | Input | Handle to a test case. srTEST_CASE_DEFAULT can be used for the default test case. |
| eStatus | Input | Result of the test. Possible values are:
|
| dwDuration | Input | The duration of the test in clock ticks. Using "0" allows the STRIDE Runtime (Intercept Module) to set the time automatically. |
| Return Value | Description |
| srWORD | srOK on success, srERR_HANDLE_INVALID on invalid handle. |
#include <srtest.h>
void tfcase_setStatus(void)
{
srTestCaseSetStatus(srTEST_CASE_DEFAULT, srTEST_PASS, 0);
}
#ifdef _SCL
#pragma scl_test_flist("testfunc", tfcase_setStatus)
#endif
srTestCaseSetStatusEx
The srTestCaseSetStatusEx() routine is used to set the result of test case and allow specification of an extendedFailureCode.
srWORD srTestCaseSetStatus(srTestCaseHandle_t tTestCase, srTestStatus_e eStatus, srDWORD dwDuration, srLONG lExtendedFailureCode)
| Parameters | Type | Description |
| tTestCase | Input | Handle to a test case. srTEST_CASE_DEFAULT can be used for the default test case. |
| eStatus | Input | Result of the test. |
| dwDuration | Input | The duration of the test in clock ticks. Using "0" allows the STRIDE Runtime (Intercept Module) to set the time automatically. |
| lExtendedFailureCode | Input | The Stride framework uses the extendedFailureCode to capture the numeric results of test method when the test method fails via a numeric (non-void, nonbool) return type. |
| Return Value | Description |
| srWORD | srOK on success, srERR_HANDLE_INVALID on invalid handle. |
#include <srtest.h>
void tfcase_setStatusEx(void)
{
srTestCaseSetStatusEx(srTEST_CASE_DEFAULT, srTEST_FAIL, 0, -5);
}
#ifdef _SCL
#pragma scl_test_flist("testfunc", tfcase_setStatusEx)
#endif
srTestCaseAddAnnotation
The srTestCaseAddAnnotation() routine is used to add a new annotation to the specified test case.
srTestAnnotationHandle_t srTestCaseAddAnnotation(rTestCaseHandle_t tTestCase, srTestAnnotationLevel_e eLevel, const srCHAR * szName, const srCHAR * szFmtDescr, ...)
| Parameters | Type | Description |
| tTestCase | Input | Handle to the parent test case to which new test annotation is to be added. srTEST_CASE_DEFAULT can be used for the default test case. |
| eLevel | Input | Annotation level.
|
| szName | Input | Pointer to a null-terminated string representing the name of the new annotation. If null, the default host naming scheme will be used. |
| szFmtDescr | Input | Pointer to a null-terminated string representing the description of the new annotation. Cannot be null. |
| ... | Input (Optional) | Variable argument list to format szFmtDescr. |
| Return Value | Description |
| srTestAnnotationHandle_t | Handle of the newly created test annotation. srTEST_ANNOTATION_INVALID indicates failure to create a test annotation. |
#include <srtest.h>
void tfcase_addAnnotation(void)
{
for (int count = 0; count < 5; ++count)
{
char szName[25];
sprintf(szName, "annotation %d", count);
srTestAnnotationHandle_t annot =
srTestCaseAddAnnotation(srTEST_CASE_DEFAULT,
srTEST_ANNOTATION_LEVEL_ERROR,
szName,
"description of annotation");
}
}
#ifdef _SCL
#pragma scl_test_flist("testfunc", tfcase_addAnnotation)
#endif
srTestCaseSetData
The srTestCaseSetData() routine is used to associate a custom name|value pair with a test case.
srWORD srTestCaseSetData(srTestCaseHandle_t tTestCase, const srCHAR * szName, const srCHAR * szFmtValue, ...);
| Parameters | Type | Description |
| tTestCase | Input | Handle to a test case. srTEST_CASE_DEFAULT can be used for the default test case. |
| szName | Input | Pointer to a null-terminated string representing the name of the custom pair. Cannot be null. |
| szFmtValue | Input | Pointer to a null-terminated string representing the value of the custom pair. Cannot be null. |
| ... | Input (Optional) | Variable argument list to format szFmtValue. |
| Return Value | Description |
| srWORD | srOK on success, srERR_HANDLE_INVALID on invalid handle. |
#include <srtest.h>
void tfcase_setData(void)
{
srTestCaseSetData(srTEST_CASE_DEFAULT, "MyName", "my value);
}
#ifdef _SCL
#pragma scl_test_flist("testfunc", tfcase_setData)
#endif
srTestAnnotationAddComment
The srTestAnnotationAddComment() routine is used to add a comment (aka a log) to a test annotation.
srWORD srTestAnnotationAddComment(srTestAnnotationHandle_t tTestAnnotation, const srCHAR * szLabel, const srCHAR * szFmtComment, ...)
| Parameters | Type | Description |
| tTestAnnotation | Input | Handle to a test annotation. |
| szLabel | Input | Pointer to a null-terminated string for the label. If null, the default label will be applied. |
| szFmtComment | Input | Pointer to a null-terminated string representing the new comment. Cannot be null. |
| ... | Input (Optional) | Variable argument list to format szFmtComment. |
| Return Value | Description |
| srWORD | srOK on success, srERR on null string or invalid formatted string, srERR_HANDLE_INVALID on invalid handle, srERR_STR_TRUNCATED on truncated string. |
#include <srtest.h>
void tfsuiteAnnotation_addComment(void)
{
srTestAnnotationHandle_t annot =
srTestSuiteAddAnnotation(srTEST_ANNOTATION_LEVEL_ERROR,
"annot",
"annot description");
srTestAnnotationAddComment(annot,
srNULL,
"this comment should print %s", "A STRING");
}
#ifdef _SCL
#pragma scl_test_flist("testfunc", tfsuiteAnnotation_addComment)
#endif
C++ Test Classes
The Runtime Test Services C APIs work equally well from C test functions and C++ test classes. If, however, you are using C++ it might be more convinient for you to derive your C++ test classes from the Runtime Test Services C++ base class, srTest. That way you will have access to a set of simpler to use C++ classes.
The following C++ classes are provided:
- class srTest - base test class
- class srTestSuite - represents a test suite
- class srTestCase - represents a test case
- class srTestAnnotation - represents a test annotation
class srTest
The srTest class provides two Member Objects:
- testSuite of class srTestSuite
- testCase of class srTestCase
class srTestSuite
The srTest class provides the folowing methods:
method SetName
The SetName method is used to set the name of test suite.
srWORD SetName(const srCHAR * szName)
| Parameters | Type | Description |
| szName | Input | Pointer to null-terminated string representing the name of test suite. Cannot be empty. |
| Return Value | Description |
| srOK | Success |
| srERR | Null string passed in. |
| srTEST_ERR_HANDLE_INVALID | Invalid handle passed in. |
| srTEST_WARN_STR_TRUNCATED | String passed in was too large and was truncated. |
#include <srtest.h>
class srtest_class : public stride::srTest
{
public:
void suiteSetName()
{
testSuite.SetName("Setting name for suite");
}
};
#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
method SetDescription
The SetDescription method is used to set the description of test suite.
srWORD SetDescription(const srCHAR * szDescr)
| Parameters | Type | Description |
| szDescr | Input | Pointer to null-terminated string representing the description of test suite. Cannot be empty. |
| Return Value | Description |
| srOK | Success |
| srERR | Null string passed in. |
| srTEST_ERR_HANDLE_INVALID | Invalid handle passed in. |
| srTEST_WARN_STR_TRUNCATED | String passed in was too large and was truncated. |
#include <srtest.h>
class srtest_class : public stride::srTest
{
public:
void suiteSetDescription()
{
testSuite.SetDescription("Setting description for suite");
}
};
#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
method AddSuite
The AddSuite[1] method is used to add a new test suite.
srTestSuite AddSuite(const srCHAR * szName = srNULL)
| Parameters | Type | Description |
| szName | Input (Optional) | Pointer to null-terminated string that represents the name of test suite. If empty, the default host naming scheme will be used. |
| Return Value | Description |
| srTestSuite | Newly created test suite class. |
#include <srtest.h>
class srtest_class : public stride::srTest
{
public:
void suiteAddSuite()
{
stride::srTestSuite suite = testSuite.AddSuite("tc Sub Suite");
}
};
#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
method AddCase
The AddCase[1] method is used to add a new test case to test suite.
srTestCase AddCase(const srCHAR * szName = srNULL)
| Parameters | Type | Description |
| szName | Input (Optional) | Pointer to null-terminated string that represents the name of test case. If empty, the default host naming scheme will be used. |
| Return Value | Description |
| srTestCase | Newly created test case class. |
Example
#include <srtest.h>
#include <sstream>
class srtest_class : public stride::srTest
{
public:
void suiteAddSuite()
{
const std::string prefix("dynamic test case ");
for (int count = 0; count < 5; ++count)
{
std::stringstream strm;
strm << prefix << count;
stride::srTestCase tc = testSuite.AddCase(strm.str().c_str());
tc.AddComment("this is a dynamic test case");
tc.SetStatus(srTEST_PASS);
}
}
};
#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
method AddAnnotation
The AddAnnotation method is used to add a new annotation to test suite.
srTestAnnotation AddAnnotation(srTestAnnotationLevel_e eLevel, const srCHAR * szName = srNULL, const srCHAR * szDescr = srNULL)
| Parameters | Type | Description |
| eLevel | Input | Annotation level. Cannot be empty.
|
| szName | Input (Optional) | Pointer to null-terminated string that represents the name of annotation. If empty, the default host naming scheme will be used. |
| szDescr | Input (Optional) | Pointer to null-terminated string that represents the description of annotation. If empty, the description will be blank. |
| Return Value | Description |
| srTestAnnotation | Newly created annotation class. |
#include <srtest.h>
#include <sstream>
class srtest_class : public stride::srTest
{
public:
void suiteAddAnnotation()
{
for (int count = 0; count < 5; ++count)
{
std::stringstream strmName;
std::stringstream strmDescr;
strmName << "annotation " << count;
strmDescr << "description of annotation " << count;
stride::srTestAnnotation ta =
testSuite.AddAnnotation(srTEST_ANNOTATION_LEVEL_INFO,
strmName.str().c_str(),
strmDescr.str().c_str());
}
}
};
#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
class srTestCase
The srTestCase class provides the following methods:
method SetName
The SetName method is used to set the name of test case.
srWORD SetName(const srCHAR * szName)
| Parameters | Type | Description |
| szName | Input | Pointer to null-terminated string representing the name of test case. Cannot be empty. |
| Return Value | Description |
| srOK | Success |
| srERR | Null string passed in. |
| srTEST_ERR_HANDLE_INVALID | Invalid handle passed in. |
| srTEST_WARN_STR_TRUNCATED | String passed in was too large and was truncated. |
#include <srtest.h>
class srtest_class : public stride::srTest
{
public:
void caseSetName()
{
testCase.SetName("Setting name for case");
}
};
#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
method SetDescription
The SetDescription method is used to set the description of test case.
srWORD SetDescription(const srCHAR * szDescr)
| Parameters | Type | Description |
| szDescr | Input | Pointer to null-terminated string representing the description of test case. Cannot be empty. |
| Return Value | Description |
| srOK | Success |
| srERR | Null string passed in. |
| srTEST_ERR_HANDLE_INVALID | Invalid handle passed in. |
| srTEST_WARN_STR_TRUNCATED | String passed in was too large and was truncated. |
#include <srtest.h>
class srtest_class : public stride::srTest
{
public:
void caseSetDescription()
{
testCase.SetDescription("Setting description for case");
}
};
#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
method SetStatus
The SetStatus method is used to set the result of the default test case.
srWORD SetStatus(srTestStatus_e eStatus, srDWORD dwDuration = 0)
| Parameters | Type | Description |
| eStatus | Input | Result of the test. Possible values are:
|
| dwDuration | Input (Optional) | The duration of the test in clock ticks. The default is 0. |
| Return Value | Description |
| srOK | Success |
| srTEST_ERR_HANDLE_INVALID | Invalid handle. |
#include <srtest.h>
class srtest_class : public stride::srTest
{
public:
void caseSetStatus()
{
testCase.SetStatus(srTEST_NOTINUSE, 0);
}
};
#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
method AddComment
The AddComment method is used to add a comment to test case.
srWORD AddComment(const srCHAR * szLabel, const srCHAR * szFmtComment, ...)
| Parameters | Type | Description |
| szLabel | Input | Pointer to a null-terminated string representing the comment label. |
| szFmtComment | Input | Pointer to null-terminated string, which can be formatted, representing the comment. Cannot be empty. |
| ... | Input (Optional) | Variable argument list to format the comment szFmtComment. |
| Return Value | Description |
| srOK | Success |
| srERR | Null string passed in. |
| srTEST_WARN_STR_TRUNCATED | String passed in was too large and was truncated. |
#include <srtest.h>
class srtest_class : public stride::srTest
{
public:
void caseAddComment()
{
testCase.AddComment("this comment should print %s", "A STRING");
}
};
#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
method AddAnnotation
The AddAnnotation method is used to add a new annotation to test case.
srTestAnnotation AddAnnotation(srTestAnnotationLevel_e eLevel, const srCHAR * szName = srNULL, const srCHAR * szDescr = srNULL)
| Parameters | Type | Description |
| eLevel | Input | Annotation level. Cannot be empty.
|
| szName | Input (Optional) | Pointer to null-terminated string that represents the name of annotation. If empty, the default host naming scheme will be used. |
| szDescr | Input (Optional) | Pointer to null-terminated string that represents the description of annotation. If empty, the description will be blank. |
| Return Value | Description |
| srTestAnnotation | Newly created annotation class. |
#include <srtest.h>
#include <sstream>
class srtest_class : public stride::srTest
{
public:
void caseAddAnnotation()
{
for (int count = 0; count < 5; ++count)
{
std::stringstream strmName;
std::stringstream strmDescr;
strmName << "annotation " << count;
strmDescr << "description of annotation " << count;
stride::srTestAnnotation ta =
testCase.AddAnnotation(srTEST_ANNOTATION_LEVEL_INFO,
strmName.str().c_str(),
strmDescr.str().c_str());
}
}
};
#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
class srTestAnnotation
The srTestAnnotation class provides the following methods:
method AddComment
The AddComment method is used to add a comment to an annotation created under a test suite.
srWORD AddComment(const srCHAR * szLabel, const srCHAR * szFmtComment, ...)
| Parameters | Type | Description |
| szLabel | Input | Pointer to null-terminated string for the label. If null, the default label will be applied. |
| szFmtComment | Input | Pointer to null-terminated string, which can be formatted, representing the comment. Cannot be empty. |
| ... | Input (Optional) | Variable argument list to format the comment szFmtComment. |
| Return Value | Description |
| srOK | Success |
| srERR | Null string passed in. |
| srTEST_ERR_HANDLE_INVALID | Invalid handle passed in. |
| srTEST_WARN_STR_TRUNCATED | String passed in was too large and was truncated. |
#include <srtest.h>
class srtest_class : public stride::srTest
{
public:
void suiteAnnotationAddComment()
{
stride::srTestAnnotation ta =
testSuite.AddAnnotation(srTEST_ANNOTATION_LEVEL_INFO,
"annot",
"annot description");
ta.AddComment("this comment on annotation should print %s", "A STRING");
}
};
#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif