Test Units Overview: Difference between revisions

From STRIDE Wiki
Jump to navigation Jump to search
Line 1,664: Line 1,664:
|}
|}


=== Overloading The << operator for report annotation (C++ tests only) ===
=== Use operator << for report annotation (C++ tests only) ===


<nowiki>All the macros support the adding to the report annotations using the << operator. For example:</nowiki>
In C++ test code all macros support the adding to the report annotations using the << operator. For example:


  srEXPECT_TRUE( a != <nowiki>b) << “My custom message”; </nowiki>
<code lang="c++">
srEXPECT_TRUE( a != <nowiki>b) << “My custom message”; </nowiki>
</code>


<nowiki>As delivered, the macros will support stream operator annotations using all the numeric types, char* and s2String. The user may create custom types to be used a report annotation by providing an implementation of the << operator for their custom type. </nowiki>
As delivered, the macros will support stream imput annotations using all the numeric types, and "C" string (char* or wchar_t*). The user may overload the << operator in order to annotate reports using any custom type. An example is below.


<nowiki>The user of the asserting macros package may overload the << operator in order to annotate reports using a custom class. An example is below. </nowiki>
The following will compile and execute successfully given that the << operator is overloaded as shown:


<nowiki>The following will compile and execute successfully given that the << operator is overloaded as shown:</nowiki>
<code lang="c++">
 
<pre>
MyCustomClass custom(34);  
MyCustomClass custom(34);  


Line 1,702: Line 1,702:
   return ss;
   return ss;
}
}
</pre>
</code>


== Scripting a Test Unit ==
== Scripting a Test Unit ==

Revision as of 07:35, 30 January 2009

Introduction

STRIDE enables testing of C/C++ code through the use of xUnit-style test units. Test units can be written by developers, captured using an SCL pragma, and executed from the host. STRIDE facilitates the execution of some or all of the test units by automatically creating entry points for the execution of test units on the target.

Using test units

Prerequisites

see Perl requirements.

How to get started (Overview)

The required steps to get started with writing test units are as follows:

  1. Create a new Studio workspace (or open an existing one).
  2. Set the workspace to compile in C++ mode (In Studio, choose Tools->Settings->Compile as Cpp).
  3. Write a test unit. You may create a C++ test class or a C test function as your test unit. Click here for more information on creating test units.
    4. To implement a test unit as a test class: 
    #include <srtest.h>

class Simple {
public:
    int test1(void) { return  0;} // PASS
    int test2(void) { return 23;} // FAIL <>0
};

#pragma scl_test_class(Simple)

    Or, as a test function: 

    #ifdef __cplusplus
extern "C" {
#endif

int test1(void) 
{
    return 0; // PASS
}

int test1(void) 
{
    return 23; // FAIL <>0
}

#pragma scl_test_flist("Simple", test1, test2)

#ifdef __cplusplus
}
#endif
  4. Compile the workspace & review the Simple interface in the Studio Interface tab
  5. Create a script to generate the Intercept Module(IM) after the compilation step.
    For the simple STUB generation required for test unit execution, you can use the following code
    6. use strict;
use Win32::OLE qw(in);
Win32::OLE->Option(Warn => 3);
my $intercept = $main::studio->Workspace->Intercept;    
$intercept->{Path} = $main::studio->Workspace->Path;
$intercept->{Name} = $main::studio->Workspace->Name;
map {$intercept->Item($_)->{Stub} = 1} (0..($intercept->Count - 1));
$intercept->Create();
  6. Optionally add custom scripts to automate the building and executing your application.
  7. Ensure that the Studio workspace include path contains the location to all of your test unit declaration (header) files.
  8. Once you have created one or more test units, ensure the following:
    • Workspace is compiled and saved
    • Intercept Module is generated (Stubs for all Test Units)
    • Target application re-built
    • Target application downloaded & started
    • STRIDE Connected to Target
  9. If your application is running, you can start executing test units.
    • You can test-execute individual test units interactively using the Studio interface view. To do this, open the user interface view corresponding to the test unit you would like to execute, then call it. The return values will indicate how many tests produced each of four (4) result types. Furthermore, the input to the entry point will allow you to select all methods for execution (the default) or individual methods via a dropdown list of enumerated values.
    • Once you are confident that the test units are behaving as expected, you can generate one or more execution scripts using the Script Wizard. Sample templates for executing test unit entry points are provided in the %STRIDE_DIR%\templates\Script Wizard directory.
    • When writing scripts to execute test units, the AutoScript TestUnits collection is a powerful tool for test unit execution and reporting, and for obtaining test results.
  10. For integration with larger regression test workspaces, we recommend that developers check in their test unit code and, optionally, the template-generated scripts that can be used to execute their test units.

Pragmas for Test Units

STRIDE supports three pragmas for capturing and qualifying test units:

  • scl_test_class ( class-name ): Declares a test class as captured. Once captured, STRIDE will generate the appropriate code for executing the test methods in the class. See the scl_test_class page for more information.
  • scl_test_flist ( test-unit-name , test-function-name { , test-function-name } ): Declares a test unit as captured, by the specified test-unit-name. One or more functions (test methods) are associated with the test unit. Once captured, STRIDE will generate the appropriate code for executing the associated test methods. See the scl_test_flist page for more information.
  • scl_test_cclass ( struct-name , init-function-name { , deinit-function-name } ): Declares a test unit as captured. Once captured, STRIDE will generate the appropriate code for executing the associated test methods. See the scl_test_cclass page for more information.
  • scl_test_setup ( test-unit-name , method-name ): [optional] Declares a member method to be a setup fixture for the test unit. If specified, the setup method will be called before the execution of each test method. See the scl_test_setup page for more information.
  • scl_test_teardown ( test-unit-name , method-name ): [optional] Declares a member method to be a teardown fixture for the test unit. If specified, the teardown method will be called after the execution of each test method. See the scl_test_teardown page for more information.

Test Unit Requirements

Several variations on typical xUnit-style test units are supported. The additional supported features include:

  • Test status can be set using STRIDE Runtime APIs or by specifying simple return types for test methods.
  • Simple return types: 0 = PASS; <> 0 = FAIL
  • void return type with no explict status setting is assumed PASS
  • Test writers can create additional child suites and tests at runtime by using Runtime APIs.
  • We do not rely on exceptions for reporting of status.

The STRIDE test class framework has the following requirements of each test class:

  • The test class must have a suitable default (no-argument) constructor.
  • The test class must have one or more public methods suitable as test methods. Allowable test methods always take no arguments (void) and return either void or simple integer types (int, short, long, char or bool). At this time, we do not allow typedef types or macros for the return values specification.
  • the scl_test_class pragma must be applied to the class.


Simple example using return values for status

Using a Test Class

#include <srtest.h>
  
class Simple {
public:
    int tc_Int_ExpectPass(void) {return 0;}
    int tc_Int_ExpectFail(void) {return -1;}
    bool tc_Bool_ExpectPass(void) {return true;}
    bool tc_Bool_ExpectFail(void) {return false;}
};

#ifdef _SCL
#pragma scl_test_class(Simple)
#endif

Using a Test Function

#include <srtest.h>
  
#ifdef __cplusplus
extern "C" {
#endif
  
int tf_Int_ExpectPass(void) {return 0;}
int tf_Int_ExpectFail(void) {return -1;}
bool tf_Bool_ExpectPass(void) {return true;}
bool tf_Bool_ExpectFail(void) {return false;}

#ifdef _SCL
#pragma scl_test_flist("Simple", tf_Int_ExpectPass, tf_Int_ExpectFail, tf_Bool_ExpectPass, tf_Bool_ExpectFail)
#endif
   
#ifdef __cplusplus
}
#endif

Simple example using runtime test service APIs

Using a Test Class

#include <srtest.h>
  
class RuntimeServices_basic {
public: 
  void tc_ExpectPass(void) 
  {
    srTestCaseAddComment(srTEST_CASE_DEFAULT, "this test should pass");
    srTestCaseSetStatus(srTEST_CASE_DEFAULT, srTEST_PASS, 0); 
  }
  void tc_ExpectFail(void) 
  {
    srTestCaseAddComment(srTEST_CASE_DEFAULT, "this test should fail");
    srTestCaseSetStatus(srTEST_CASE_DEFAULT, srTEST_FAIL, 0); 
  }
  void tc_ExpectInProgress(void) 
  {
    srTestCaseAddComment(srTEST_CASE_DEFAULT, "this test should be in progress");
  }
};

#ifdef _SCL
#pragma scl_test_class(RuntimeServices_basic)
#endif

Using a Test Function

#include <srtest.h>
  
#ifdef __cplusplus
extern "C" {
#endif
  
void tf_ExpectPass(void) 
{
  srTestCaseAddComment(srTEST_CASE_DEFAULT, "this test should pass");
  srTestCaseSetStatus(srTEST_CASE_DEFAULT, srTEST_PASS, 0); 
}
void tf_ExpectFail(void) 
{
  srTestCaseAddComment(srTEST_CASE_DEFAULT, "this test should fail");
  srTestCaseSetStatus(srTEST_CASE_DEFAULT, srTEST_FAIL, 0); 
}
void tf_ExpectInProgress(void) 
{
  srTestCaseAddComment(srTEST_CASE_DEFAULT, "this test should be in progress");
}

#ifdef _SCL
#pragma scl_test_flist("RuntimeServices_basic", tf_ExpectPass, tf_ExpectFail, tf_ExpectInProgress)
#endif
 
#ifdef __cplusplus
}
#endif

Simple example using srTest base class

#include <srtest.h>
  
class MyTest : public stride::srTest {
public:
  void tc_ExpectPass(void) 
  {
    testCase.AddComment("this test should pass");
    testCase.SetStatus(srTEST_PASS, 0); 
  }
  void tc_ExpectFail(void) 
  {
    testCase.AddComment("this test should fail");
    testCase.SetStatus(srTEST_FAIL, 0); 
  }
  void tc_ExpectInProgress(void) 
  {
    testCase.AddComment("this test should be in progress");
  }
  int tc_ChangeMyName(void) 
  {
    testCase.AddComment("this test should have name = MyChangedName");
    testCase.SetName("MyChangedName");
    return 0;
  }
  int tc_ChangeMyDescription(void) 
  {
    testCase.AddComment("this test should have a description set");
    testCase.SetDescription("this is my new description");
    return 0;
  }
};

#ifdef _SCL
#pragma scl_test_class(MyTest)
#endif

Runtime Test Services

The Runtime Test Services (declared in srTest.h) are a set of APIs in the STRIDE Runtime that facilitate the writing of target based test code. These APIs make up an optional portion of the STRIDE Runtime and can be used to communicate additional information about tests to the host based reporting mechanism. These APIs also allow target test code to create additional test suites and test cases dynamically at runtime.

There are 2 alternate ways of accessing these APIs: through C-based functions, or through a C++ base class from which you may derive your C++ test class. These are discussed below in C Test Functions, and C++ Test Classes sections below.

C Test Functions

The following C APIs are provided:


srTestSuiteAddSuite

The srTestSuiteAddSuite() routine is used to add a new test suite to the specified test suite. 

srTestSuiteHandle_t srTestSuiteAddSuite(srTestSuiteHandle_t tParent, const srCHAR * szName)
Parameters Type Description
tParent Input Handle to the parent test suite to which new test suite is to be added. srTEST_SUITE_DEFAULT can be used for the default test suite.
szName Input Pointer to a null-terminated string that represents the name of test suite. If null, the default host naming scheme will be used.


Return Value Description
srTestSuiteHandle_t Handle of the newly created test suite. srTEST_SUITE_INVALID indicates failure to create test suite. 
#include <srtest.h>

void tfsuite_addSuite(void)
{
  srTestSuiteHandle_t subSuite =
  srTestSuiteAddSuite(srTEST_SUITE_DEFAULT, "tf Sub Suite");
}

#ifdef _SCL
#pragma scl_test_flist(“testfunc”, tfsuite_addSuite)
#endif

srTestSuiteSetName

The srTestSuiteSetName() routine is used to set the name of the specified test suite. 

srWORD srTestSuiteSetName(srTestSuiteHandle_t tTestSuite, const srCHAR * szName)
Parameters Type Description
tTestSuite Input Handle to a test suite. srTEST_SUITE_DEFAULT can be used for the default test suite.
szName Input Pointer to a null-terminated string representing the name of test suite. Cannot be null.


Return Value Description
srOK Success
srERR Null string passed in.
srTEST_ERR_HANDLE_INVALID Invalid handle passed in.
srTEST_WARN_STR_TRUNCATED String passed in was too large and was truncated. 
#include <srtest.h>

void tfsuite_setName(void)
{
  srTestSuiteSetName(srTEST_SUITE_DEFAULT, "Setting name for default suite");
}

#ifdef _SCL
#pragma scl_test_flist(“testfunc”, tfsuite_setName)
#endif

srTestSuiteSetDescription

The srTestSuiteSetDescription() routine is used to set the description of the specified test suite. 

srWORD srTestSuiteSetDescription(srTestSuiteHandle_t tTestSuite, const srCHAR * szDescr)
Parameters Type Description
tTestSuite Input Handle to a test suite. srTEST_SUITE_DEFAULT can be used for the default test suite.
szDescr Input Pointer to a null-terminated string representing the description of test suite. Cannot be null.


Return Value Description
srOK Success
srERR Null string passed in.
srTEST_ERR_HANDLE_INVALID Invalid handle passed in.
srTEST_WARN_STR_TRUNCATED String passed in was too large and was truncated. 
#include "srtest.h"
void tfsuite_setDescription(void)
{
  srTestSuiteSetDescription(srTEST_SUITE_DEFAULT,
                            "Setting description for default suite");
}

#ifdef _SCL
#pragma scl_test_flist(“testfunc”, tfsuite_setDescription)
#endif

srTestSuiteAddTest

The srTestSuiteAddTest() routine is used to add a new test case to the specified test suite. 

srTestCaseHandle_t srTestSuiteAddTest(srTestSuiteHandle_t tParent, const srCHAR * szName)
Parameters Type Description
tParent Input Handle to the parent test suite to which new test case is to be added. srTEST_SUITE_DEFAULT can be used for the default test suite.
szName Input Pointer to a null-terminated string that represents the name of test case. If null, the default host naming scheme will be used.


Return Value Description
srTestCaseHandle_t Handle of the newly created test case. srTEST_CASE_INVALID indicates failure to create test case. 
#include <srtest.h>
#include <sstream>

void tfsuite_addTest(void)
{
  for (int count = 0; count < 5; ++count)
  {
    char szName[25];
    sprintf(szName, "dynamic test case %d", count);
    srTestCaseHandle_t test = srTestSuiteAddTest(srTEST_SUITE_DEFAULT, szName);
    srTEST_ADD_COMMENT_WITH_LOCATION(test, "this is a dynamic test case");
  }
}

#ifdef _SCL
#pragma scl_test_flist(“testfunc”, tfsuite_addTest)
#endif

srTestCaseSetName

The srTestCaseSetName() routine is used to set set the name of the specified test case. 

srWORD srTestCaseSetName(srTestCaseHandle_t tTestCase, const srCHAR *szName)
Parameters Type Description
tTestCase Input Handle to a test case. srTEST_CASE_DEFAULT can be used for the default test case.
szName Input Pointer to a null-terminated string representing the name of test case. Cannot be null.


Return Value Description
srOK Success
srERR Null string passed in.
srTEST_ERR_HANDLE_INVALID Invalid handle passed in.
srTEST_WARN_STR_TRUNCATED String passed in was too large and was truncated. 
#include <srtest.h>

void tfcase_setName(void)
{
  srTestCaseSetName(srTEST_CASE_DEFAULT, "Setting name for default case");
}

#ifdef _SCL
#pragma scl_test_flist(“testfunc”, tfcase_setName)
#endif

srTestCaseSetDescription

The srTestCaseSetDescription() routine is used to set the description of the specified test case. 

srWORD srTestCaseSetDescription(srTestCaseHandle_t tTestCase, const srCHAR * szDescr)
Parameters Type Description
tTestCase Input Handle to a test case. srTEST_CASE_DEFAULT can be used for the default test case.
szDescr Input Pointer to a null-terminated string representing the description of test case. Cannot be null.


Return Value Description
srOK Success
srERR Null string passed in.
srTEST_ERR_HANDLE_INVALID Invalid handle passed in.
srTEST_WARN_STR_TRUNCATED String passed in was too large and was truncated. 
#include <srtest.h>

void tfcase_setDescription(void)
{
  srTestCaseSetDescription(srTEST_CASE_DEFAULT,
                           "Setting description for default case");
}

#ifdef _SCL
#pragma scl_test_flist(“testfunc”, tfcase_setDescription)
#endif

srTestCaseAddComment

The srTestCaseAddComment() routine is used to add a comment (aka a log) to be reported with the specified test case. 

srWORD srTestCaseAddComment(srTestCaseHandle_t tTestCase, const srCHAR * szFmtComment, ...)
Parameters Type Description
tTestCase Input Handle to a test case. srTEST_CASE_DEFAULT can be used for the default test case.
szFmtComment Input Pointer to a null-terminated string, which can be formatted, representing the comment. Cannot be null.
... Input (Optional) Variable argument list to format the comment szFmtComment.


Return Value Description
srOK Success
srERR Null string passed in.
srTEST_ERR_HANDLE_INVALID Invalid handle passed in.
srTEST_WARN_STR_TRUNCATED String passed in was too large and was truncated. 
#include <srtest.h>

void tfcase_addComment(void)
{
  srTestCaseAddComment(srTEST_CASE_DEFAULT,
                       "this comment should print %s", "A STRING");
}

#ifdef _SCL
#pragma scl_test_flist(“testfunc”, tfcase_addComment)
#endif

srTestCaseSetStatus

The srTestCaseSetStatus() routine is used to set the result of test case. 

srWORD srTestCaseSetStatus(srTestCaseHandle_t tTestCase, srTestStatus_e eStatus, srDWORD dwDuration)
Parameters Type Description
tTestCase Input Handle to a test case. srTEST_CASE_DEFAULT can be used for the default test case.
eStatus Input Result of the test.
dwDuration Input The duration of the test in clock ticks. Using “0” allows the STRIDE Runtime (Intercept Module) to set the time automatically.


Return Value Description
srOK Success
srTEST_ERR_HANDLE_INVALID Invalid handle passed in. 
#include <srtest.h>

void tfcase_setStatus(void)
{
  srTestCaseSetStatus(srTEST_CASE_DEFAULT, srTEST_PASS, 0);
}

#ifdef _SCL
#pragma scl_test_flist(“testfunc”, tfcase_setStatus)
#endif

srTestCaseSetStatusEx

The srTestCaseSetStatusEx() routine is used to set the result of test case and allow specification of an extendedFailureCode. 

srWORD srTestCaseSetStatus(srTestCaseHandle_t tTestCase, srTestStatus_e eStatus, srDWORD dwDuration, srLONG lExtendedFailureCode)
Parameters Type Description
tTestCase Input Handle to a test case. srTEST_CASE_DEFAULT can be used for the default test case.
eStatus Input Result of the test. dwDuration Input The duration of the test in clock ticks. Using “0” allows the STRIDE Runtime (Intercept Module) to set the time automatically.
lExtendedFailureCode Input The Stride framework uses the extendedFailureCode to capture the numeric results of test method when the test method fails via a numeric (non-void, nonbool) return type.


Return Value Description
srOK Success
srTEST_ERR_HANDLE_INVALID Invalid handle passed in. 
#include <srtest.h>

void tfcase_setStatusEx(void)
{
  srTestCaseSetStatusEx(srTEST_CASE_DEFAULT, srTEST_FAIL, 0, -5);
}

#ifdef _SCL
#pragma scl_test_flist(“testfunc”, tfcase_setStatusEx)
#endif

srTestSuiteAddAnnotation

The srTestSuiteAddAnnotation() routine is used to add a new annotation to the specified test suite. 

srTestAnnotationHandle_t srTestSuiteAddAnnotation(rTestSuiteHandle_t tParent, srTestAnnotationLevel_e eLevel, const srCHAR * szName, const srCHAR * szDescr)
Parameters Type Description
tParent Input Handle to the parent test suite to which new annotation is to be added. srTEST_SUITE_DEFAULT can be used for the default test suite.
eLevel Input Annotation level. Cannot be empty.
szName Input Pointer to a null-terminated string that represents the name of annotation. If null, the default host naming scheme will be used.
szDescr Input Pointer to a null-terminated string representing the description of annotation. If null, description will be empty.


Return Value Description
srTestAnnotationHandle_t HHandle of the newly created annotation. srTEST_ANNOTATION_INVALID indicates failure to create annotation. 
#include <srtest.h>

void tfsuite_addAnnotation(void) 
{
  for (int count = 0; count < 5; ++count)
  {
    char szName[25];
    sprintf(szName, "annotation %d", count);
    srTestAnnotationHandle_t annot = 
                     srTestSuiteAddAnnotation(srTEST_SUITE_DEFAULT,
                                              srTEST_ANNOTATION_LEVEL_ERROR,
                                              szName,
                                              "description of annotation");
  }
}

#ifdef _SCL
#pragma scl_test_flist(“testfunc”, tfsuite_addAnnotation)
#endif

srTestAnnotationAddComment

The srTestAnnotationAddComment() routine is used to add a comment (aka a log) to be reported with the specified annotation. 

srWORD srTestAnnotationAddComment(srTestAnnotationHandle_t tTestAnnotation, const srCHAR * szLabel, const srCHAR * szFmtComment, ...)
Parameters Type Description
tTestAnnotation Input Handle to an annotation created using srTestSuiteAddAnnotation.
szLabel Input Pointer to a null-terminated string for the label. If null, the default label will be applied.
szFmtComment Input Pointer to a null-terminated string, which can be formatted, representing the comment. Cannot be null.
... Input (Optional) Variable argument list to format the comment szFmtComment.


Return Value Description
srOK Success
srERR Null string passed in.
srTEST_ERR_HANDLE_INVALID Invalid handle passed in.
srTEST_WARN_STR_TRUNCATED String passed in was too large and was truncated. 
#include <srtest.h>

void tfsuiteAnnotation_addComment(void) 
{
  srTestAnnotationHandle_t annot = 
                           srTestSuiteAddAnnotation(srTEST_SUITE_DEFAULT,
                                                    srTEST_ANNOTATION_LEVEL_ERROR,
                                                    “annot”,
                                                    “annot description”);
  srTestAnnotationAddComment(annot,
                             srNULL,
                             "this comment should print %s", "A STRING");
}

#ifdef _SCL
#pragma scl_test_flist(“testfunc”, tfsuiteAnnotation_addComment)
#endif

C++ Test Classes

The Runtime Test Services C APIs work equally well from C test functions and C++ test classes. If, however, you are using C++ it might be more convinient for you to derive your C++ test classes from the Runtime Test Services C++ base class, srTest. That way you will have access to a set of simpler to use C++ classes.

The following C++ classes are provided:

class srTest

The srTest class provides two Member Objects:

  • testSuite of class srTestSuite
  • testCaseof class srTestCase

class srTestSuite

The srTest class provides the folowing methods:

method AddSuite

The AddSuite method is used to add a new test suite. 

srTestSuite AddSuite(const srCHAR * szName = srNULL)
Parameters Type Description
szName Input (Optional) Pointer to null-terminated string that represents the name of test suite. If empty, the default host naming scheme will be used.


Return Value Description
srTestSuite Newly created test suite class. 
#include <srtest.h>

class srtest_class : public stride::srTest
{
public:
  void suiteAddSuite(void);
};

void srtest_class::suiteAddSuite(void)
{
  stride::srTestSuite suite = testSuite.AddSuite("tc Sub Suite");
}

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
method SetName

The SetName method is used to set the name of test suite. 

srWORD SetName(const srCHAR * szName)
Parameters Type Description
szName Input Pointer to null-terminated string representing the name of test suite. Cannot be empty.


Return Value Description
srOK Success
srERR Null string passed in.
srTEST_ERR_HANDLE_INVALID Invalid handle passed in.
srTEST_WARN_STR_TRUNCATED String passed in was too large and was truncated. 
#include <srtest.h>

class srtest_class : public stride::srTest
{
public:
  void suiteSetName(void);
};

void srtest_class::suiteSetName(void)
{
  testSuite.SetName("Setting name for suite");
}

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
method SetDescription

The SetDescription method is used to set the description of test suite. 

srWORD SetDescription(const srCHAR * szDescr)
Parameters Type Description
szDescr Input Pointer to null-terminated string representing the description of test suite. Cannot be empty.


Return Value Description
srOK Success
srERR Null string passed in.
srTEST_ERR_HANDLE_INVALID Invalid handle passed in.
srTEST_WARN_STR_TRUNCATED String passed in was too large and was truncated. 
#include <srtest.h>

class srtest_class : public stride::srTest
{
public:
  void suiteSetDescription(void);
};

void srtest_class::suiteSetDescription(void)
{
  testSuite.SetDescription("Setting description for suite");
}

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
method AddTest

The AddTest method is used to add a new test case to test suite. 

srTestCase AddTest(const srCHAR * szName = srNULL)
Parameters Type Description
szName Input (Optional) Pointer to null-terminated string that represents the name of test case. If empty, the default host naming scheme will be used.


Return Value Description
srTestCase Newly created test case class.


Example 

#include <srtest.h>
#include <sstream>

class srtest_class : public stride::srTest
{
public:
  void suiteAddSuite(void);
};

void srtest_class::suiteAddSuite(void)
{
  const std::string prefix("dynamic test case ");
  for (int count = 0; count < 5; ++count)
  {
    std::stringstream strm;
    strm << prefix << count;
    stride::srTestCase tc = testSuite.AddTest(strm.str().c_str());
    tc.AddComment("this is a dynamic test case");
    tc.SetStatus(srTEST_PASS);
  }
}

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
method AddAnnotation

The AddAnnotation method is used to add a new annotation to test suite. 

srTestAnnoation AddAnnotation(srTestAnnotationLevel_e eLevel, const srCHAR * szName = srNULL, const srCHAR * szDescr = srNULL)
Parameters Type Description
eLevel Input Annotation level. Cannot be empty.
szName Input (Optional) Pointer to null-terminated string that represents the name of annotation. If empty, the default host naming scheme will be used.
szDescr Input (Optional) Pointer to null-terminated string that represents the description of annotation. If empty, the description will be blank.


Return Value Description
srTestAnnotation Newly created annotation class. 
#include <srtest.h>
#include <sstream>

class srtest_class : public stride::srTest
{
public:
  void suiteAddAnnotation(void);
};

void srtest_class::suiteAddAnnotation(void) 
{
  const std::string prefixName("annotation ");
  const std::string prefixDescr("description of annotation ");
  for (int count = 0; count < 5; ++count)
  {
    std::stringstream strmName;
    std::stringstream strmDescr;
    strmName << prefixName << count;
    strmDescr << prefixDescr << count;
    stride::srTestAnnotation ta = 
               testSuite.AddAnnotation(srTEST_ANNOTATION_LEVEL_INFO,
                                       strmName.str().c_str(),
                                       strmDescr.str().c_str());
  }
}

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif

class srTestCase

The srTestCase class provides the following methods:

method SetName

The SetName method is used to set the name of test case. 

srWORD SetName(const srCHAR * szName)
Parameters Type Description
szName Input Pointer to null-terminated string representing the name of test case. Cannot be empty.


Return Value Description
srOK Success
srERR Null string passed in.
srTEST_ERR_HANDLE_INVALID Invalid handle passed in.
srTEST_WARN_STR_TRUNCATED String passed in was too large and was truncated. 
#include <srtest.h>

class srtest_class : public stride::srTest
{
public:
  void caseSetName(void);
};

void srtest_class::caseSetName(void)
{
  testCase.SetName("Setting name for case");
}

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
method SetDescription

The SetDescription method is used to set the description of test case. 

srWORD SetDescription(const srCHAR * szDescr)
Parameters Type Description
szDescr Input Pointer to null-terminated string representing the description of test case. Cannot be empty.


Return Value Description
srOK Success
srERR Null string passed in.
srTEST_ERR_HANDLE_INVALID Invalid handle passed in.
srTEST_WARN_STR_TRUNCATED String passed in was too large and was truncated. 
#include <srtest.h>

class srtest_class : public stride::srTest
{
public:
  void caseSetDescription(void);
};

void srtest_class::caseSetDescription(void)
{
  testCase.SetDescription("Setting description for case");
}

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
method AddComment

The AddComment method is used to add a comment to test case. 

srWORD AddComment(const srCHAR * szFmtComment, ...)
Parameters Type Description
szFmtComment Input Pointer to null-terminated string, which can be formatted, representing the comment. Cannot be empty.
... Input (Optional) Variable argument list to format the comment szFmtComment.


Return Value Description
srOK Success
srERR Null string passed in.
srTEST_ERR_HANDLE_INVALID Invalid handle passed in.
srTEST_WARN_STR_TRUNCATED String passed in was too large and was truncated. 
#include <srtest.h>

class srtest_class : public stride::srTest
{
public:
  void caseAddComment(void);
};

void srtest_class::caseAddComment(void)
{
  testCase.AddComment("this comment should print %s", "A STRING");
}

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
method SetStatus

The SetStatus method is used to set the result of the default test case. 

srWORD SetStatus(srTestStatus_e eStatus, srDWORD dwDuration = 0)
Parameters Type Description
eStatus Input Result of the test.
dwDuration Input (Optional) The duration of the test in clock ticks. The default is 0.


Return Value Description
srOK Success
srTEST_ERR_HANDLE_INVALID Invalid handle. 
#include <srtest.h>

class srtest_class : public stride::srTest
{
public:
  void caseSetStatus(void);
};

void srtest_class::caseSetStatus(void)
{
  testCase.SetStatus(srTEST_INPROGRESS, 0);
}

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif

class srTestAnnotation

The srTestAnnotation class provides the following methods:

method AddComment

The AddComment method is used to add a comment to an annotation created under a test suite. 

srWORD AddComment(const srCHAR * szLabel, const srCHAR * szFmtComment, ...)
Parameters Type Description
szLabel Input Pointer to null-terminated string for the label. If null, the default label will be applied.
szFmtComment Input Pointer to null-terminated string, which can be formatted, representing the comment. Cannot be empty.
... Input (Optional) Variable argument list to format the comment szFmtComment.


Return Value Description
srOK Success
srERR Null string passed in.
srTEST_ERR_HANDLE_INVALID Invalid handle passed in.
srTEST_WARN_STR_TRUNCATED String passed in was too large and was truncated. 
#include <srtest.h>

class srtest_class : public stride::srTest
{
public:
  void suiteAnnotationAddComment(void);
};

void srtest_class::suiteAnnotationAddComment(void)
{
  stride::srTestAnnotation ta = 
                 testSuite.AddAnnotation(srTEST_ANNOTATION_LEVEL_INFO,
                                         “annot”,
                                         “annot description”);
  ta.AddComment("this comment on annotation should print %s", "A STRING");
}

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif

Refer to the SCL Reference Guide or the Runtime Developer's Guide for detailed information about any of these APIs.

ASSERT/EXPECT Macros

The STRIDE Test Unit implementation also provides a set of Test Macros available for use within test methods. The macros are optional - you are not required to use them in your test units. They provide shortcuts for testing assertions and automatic report annotation in the case of failures.

The macros can be used in C++ and C test unit code (Note that there is no C version of exceptions test).

General guidelines for all macros

srEXPECT_xx macros will set the current test case to fail (if it hasn’t already been set) and produce an annotation in the report if the expectation fails. If the expectation succeeds, there is no action.

srASSERT_xx macros will set the current test case to fail (if it hasn’t already been set) and insert an annotation into the report if the assertion fails. In addition, a return from the current function will occur. srASSERT_xx macros can only be used in test functions which return void. If the assertion succeeds there is no action.

The report annotation produced by a failed macro always includes the source file and line along with details about the condition that failed and the failing values.

Boolean Macros

The boolean macros take a single condition expression, cond, that evaluates to an integral type or bool. The condition will be evaluated once. if cond evaluates to non-zero the assertion or expectation fails. When a failure occurs the report will be annotated.

Boolean
srEXPECT_TRUE(cond); srASSERT_TRUE(cond); cond is true
srEXPECT_FALSE(cond); srASSERT_FALSE(cond); cond is false

Comparison Macros

Comparison macros take two operands and compare them using the indicated operator. The comparison macros will work for primitive types as well as objects that have the corresponding comparison operator implemented.

Comparison
srEXPECT_EQ(val1, val2); srASSERT_EQ(val1, val2); val1 == val2
srEXPECT_NE(val1, val2); srASSERT_NE(val1, val2); val1 != val2
srEXPECT_LT(val1, val2); srASSERT_LT(val1, val2); val1 < val2
srEXPECT_LE(val1, val2); srASSERT_LE(val1, val2); val1 <= val2
srEXPECT_GT(val1, val2); srASSERT_GT(val1, val2); val1 > val2
srEXPECT_GE(val1, val2); srASSERT_GE(val1, val2); val1 >= val2

C String Comparison Macros

C String Comparison Macros are intended only for use with C-style zero terminated strings. The strings can be char or wchar_t based. In particular, these macros should not be used for object of one or other string class, since such classes have overloaded comparison operators. The standard comparison macros should be used instead.

  • The length of a string displayed in the report annotation message is controllable via the following macro: srCFG_TEST_MACROS_MAX_C_STRING_MESSAGE_DISPLAY (found in srtestmacros.h). If a string value is truncated because of the length limit, the text "...(trunc)" will always appear next to it.
  • An empty string will appear in error message output as “”. A null string will appear as NULL with no surrounding quotes. Otherwise all output strings are quoted.
  • The type of str1 and str2 must be compatible with const char* or const wchar_t*.
C-string comparison
srEXPECT_STREQ(str1, str2); srASSERT_STREQ(str1, str2); str1 and str2 have the same content
srEXPECT_STRNE(str1, str2); srASSERT_STRNE(str1, str2); str1 and str2 have different content
srEXPECT_STRCASEEQ(str1, str2); srASSERT_STRCASEEQ(str1, str2); str1 and str2 have the same content, ignoring case.
srEXPECT_STRCASENE(str1, str2); srASSERT_STRCASENE(str1, str2); str1 and str2 have different content, ignoring case.

Exception Macros

Exception macros are used to ensure that expected exceptions are thrown. They require exception support from the target compiler. If the target compiler does not have exception support the macros cannot be used and must be disabled. The support can be disabled via the macro srCFG_TEST_MACROS_EXCEPTIONS_ON found in srtestmacros.h.

Exceptions
srEXPECT_THROW(statement, ex_type); srASSERT_THROW(statement, ex_type); statement throws an exception of type ex_type
srEXPECT_THROW_ANY(statement); srASSERT_THROW_ANY(statement); statement throws an exception (type not important)
srEXPECT_NO_THROW(statement); srASSERT_NO_THROW(statement); statement does not throw an exception

Predicate Macros

Predicate macros allow user control over the pass/fail decision making in a macro. A predicate is a function returning bool that is implemented by the user but passed to the macro. Other arguments for the predicate are also passed to the macro. The macros allow for predicate functions with up to four parameters.

Predicates
srEXPECT_PRED1(pred, val1) srASSERT_PRED1(pred, val1) pred(val1) returns true
srEXPECT_PRED2(pred, val1, val2) srASSERT_PRED2(pred, val1, val2) pred(val1, val2) returns true
…(up to arity of 4)

All predicate macros require a predicate function function which returns bool. The predicate macros allow functions with one to 4 parameters. Following are the report annotations resulting from expectation or assertion failures.

Floating Point Macros

Floating point macros are for comparing equivalence (or near equivalence) of floating point numbers. These macros are necessary since because equivalence comparisons for floating point numbers will often fail due to round-off errors.

Floating Point Macros
srEXPECT_NEAR(val1, val2, epsilon); srASSERT_NEAR(val1, val2, epsilon); The absolute value of the difference between val1 and val2 is epsilon.

Dynamic Test Case Macros

The macros presented so far are not capable of dealing with dynamic test cases. In order to handle dynamic test cases, each of the macros requires another parameter which is the test case to report against. Other than this, these macros provide exactly equivalent functionality to the non-dynamic peer. The dynamic macros are listed below. All require a test case, value of type srTestCaseHandle_t from srtest.h, to be passed as the first parameter).

Nonfatal assertion Fatal Assertion
Boolean
srEXPECT_TRUE_DYN(tc, cond); srASSERT_TRUE_DYN(tc, cond);
srEXPECT_FALSE_DYN(tc, cond); srASSERT_FALSE_DYN(tc, cond);
Comparison
srEXPECT_EQ_DYN(tc, val1, val2); srASSERT_EQ_DYN(tc, expect, val);
srEXPECT_NE_DYN(tc, val1, val2); srASSERT_NE_DYN(tc, val1, val2);
srEXPECT_LT_DYN(tc, val1, val2); srASSERT_LT_DYN(tc, val1, val2);
srEXPECT_LE_DYN(tc, val1, val2); srASSERT_LE_DYN(tc, val1, val2);
srEXPECT_GT_DYN(tc, val1, val2); srASSERT_GT_DYN(tc, val1, val2);
srEXPECT_GE_DYN(tc, val1, val2); srASSERT_GE_DYN(tc, val1, val2);
C-string comparison
srEXPECT_STREQ_DYN(tc, str1, str2); srASSERT_STREQ_DYN(tc, str1, str2);
srEXPECT_STRNE_DYN(tc, str1, str2); srASSERT_STRNE_DYN(tc, str1, str2);
srEXPECT_STRCASEEQ_DYN(tc, str1, str2); srASSERT_STRCASEEQ_DYN(tc, str1, str2);
srEXPECT_STRCASENE_DYN(tc, str1, str2); srASSERT_STRCASENE_DYN(tc, str1, str2);
Exceptions
srEXPECT_THROW_DYN(statement, ex_type); srASSERT_THROW_DYN(tc, statement, ex_type);
srEXPECT_THROW_ANY_DYN(tc, statement); srASSERT_THROW_ANY_DYN(tc, statement);
srEXPECT_NO_THROW_DYN(tc, statement); srASSERT_NO_THROW_DYN(tc, statement);
Predicates
srEXPECT_PRED1_DYN(tc, pred, val1); srASSERT_PRED1_DYN(tc, pred, val1);
srEXPECT_PRED2_DYN(tc, pred, vall, val2); srASSERT_PRED2_DYN(tc, pred, vall, val2);
…(up to arity of 4)
Floating Point
srEXPECT_NEAR_DYN(tc, val1, val2, epsilon); srASSERT_NEAR_DYN(tc, val1, val2, epsilon);

Use operator << for report annotation (C++ tests only)

In C++ test code all macros support the adding to the report annotations using the << operator. For example:

srEXPECT_TRUE( a != b) << “My custom message”;

As delivered, the macros will support stream imput annotations using all the numeric types, and "C" string (char* or wchar_t*). The user may overload the << operator in order to annotate reports using any custom type. An example is below.

The following will compile and execute successfully given that the << operator is overloaded as shown:

MyCustomClass custom(34);

srEXPECT_FALSE(true) << custom;

// MyCustomClass implementation class MyCustomClass { public: 

  MyCustomClass(int i) : m_int(i) {}

private: 

  int m_int; 
  friend stride::Message& operator<< 
  ( stride::Message& ss, const MyCustomClass& obj);

};

stride::Message& operator<<(stride::Message& ss, const MyCustomClass& obj) { 

  // format the internals as a string and send on
  char buf[20]; 
  sprintf(buf, "%d", obj.m_int); 
  ss << buf;
  return ss;

}

Scripting a Test Unit

To automate the execution and reporting of a Test Unit a script is required. Scripts can be written by hand or automatically generated using the Script Wizard and a corresponding template script. A scripting tool for executing a test unit is the AutoScript TestUnits collection. An Ascript TestUnit object assembles all of the reporting information for the test unit and its corresponding test methods.

  • Require usage of the AutoScript TestUnits collection
  • Can be written by hand (refer below)
  • Can leverage Templates via the Script Wizard
  • Order of multiple test units dictated by SUID assignment


JScript example for a single test unit

The following example script is used to harness a test unit that has been captured using #pragma scl_test_class(Simple). 

// Ensure test unit exists
if (ascript.TestUnits.Item("Simple") != null) 
  ascript.TestUnits.Item("Simple").Run();

Perl example for a single test unit

The following example script is used to harness a test unit that has been captured using #pragma scl_test_class(Simple). 

use strict;
use Win32::OLE;
Win32::OLE->Option(Warn => 3);
   
my $tu = $main::ascript->TestUnits->Item("Simple");
if (defined $tu) {
  $tu->Run();
}

JScript example for multiple test units

The following example script is used to harness two test units that have been captured using #pragma scl_test_class(Simple1) and #pragma scl_test_class(Simple2). 

var Units = ["Simple1","Simple2"];
  
// iterate through each function
for (i in Units)
{
  var tu = ascript.TestUnits.Item(Units[i]);
  if ( tu&nbsp;!= null ) 
    tu.Run();
}

Perl example for multiple test units

The following example script is used to harness two test units that have been captured using #pragma scl_test_class(Simple1) and #pragma scl_test_class(Simple2). 

use strict;
use Win32::OLE;
Win32::OLE->Option(Warn => 3);
   
# initialize an array with all selected function names
my @UnitNames = ("Simple1","Simple2");
foreach (@UnitNames) {   
  my $tu = $main::ascript->TestUnits->Item($_->[1]);
  die "TestUnit not found: $_->[1]\n" unless (defined $tu);
  $tu->Run();
}
Retrieved from "https://www.stridewiki.com/index.php?title=Test_Units_Overview&oldid=9000"