Indicating Pass/Fail in a Test Method

Test Method Return Values

Stride test methods (or free functions in the case of scl_flist test units) must conform to certain criteria. These criteria allow for several choices in return type from the test method.

Return Type	Description	How Return Value is Interpreted	Default Status
void	Most common return type	Since there is no return, PASS/FAIL status is set in the body of the method using a test macro or a runtime call srTestCaseSetStatus() srTestCaseSetStatusEx() srTestCase::SetStatus()	If the method runs to completion without another status being set, the status is srTEST_PASS If the method does not run to completion (due to crash or hang in software under test), the status is srTEST_INPROGRESS
integer type	Integer types include: int short char long may include signed/unsigned/const qualifiers	If you explicitly set the status in the method (using a test macro or a runtime call), this status will override the status that would otherwise be set by the return value If you don't explicitly set the status in the method, the return value determines the status as follows: Return is zero -> srTEST_PASS Return is non-zero -> srTEST_FAIL	If the method runs to completion, the status is determined by the rules to the left (there is no default) If the method does not run to completion (due to crash or hang in software under test), the status is srTEST_INPROGRESS
bool	c++ only	If you explicitly set the status in the method (using a test macro or a runtime call), this status will override the status that would otherwise be set by the return value If you don't explicitly set the status in the method, the return value determines the status as follows: Return is true -> srTEST_PASS Return is false -> srTEST_FAIL	If the method runs to completion, the status is determined by the rules to the left (there is no default) If the method does not run to completion (due to crash or hang in software under test), the status is srTEST_INPROGRESS

Examples

The following example implementations validate the operation of a function named DoAdd() (not shown). Required header file declarations are assumed to be in place.

void return

This is the return type that is most often used as it provides the most convenience.

c++ examples are shown, but techniques are applicable to c as well.

#include <srtest.h>

/////////////////////////////
// shows use of a test macro to set status
/////////////////////////////
void MyTest::Test1()
{
    int sum = DoAdd(2, 2);
    // test macro validates condition and sets test status.
    srEXPECT_EQ(sum, 4);
}

/////////////////////////////
// shows use a a runtime call to set status
/////////////////////////////
void MyTest::Test2()
{
    int sum = DoAdd(3, 3);
    // set the status using a runtime call; much less convenient than
    // using the macro, but same result
    srTestStatus_e eStatus = (sum == 6) ? srTEST_PASS : srTEST_FAIL;
    testCase.SetStatus(eStatus, 0);
}

/////////////////////////////
// shows use of ASSERT macro to "short circuit" the test
/////////////////////////////
void MyTest::Test3()
{
    HANDLE h = GetResource();
    // if the resource is not acquired, there is no point in continuing; the
    // ASSERT macro validates and will set status to srTEST_FAIL and 
    // immediately return if the condition is not met
    srASSERT_NEQ(h, 0);

    int rc = h->DoSomething();
    srEXPECT_EQ(rc, 0);
}

/////////////////////////////
// shows default status
/////////////////////////////
void MyTest::Test4()
{
    // status is not set, so default srTEST_PASS is in force
}

integer return

#include <srtest.h>

int MyTest::Test1()
{
    int sum = DoAdd(2, 2);
    // return 0 -> PASS, non-zero -> FAIL
    return (sum == 4) ? 0 : 1;
}

int MyTest::Test2()
{
    int sum = DoAdd(3, 3);
    // 
    srEXPECT_EQ(sum, 6);
    // return value does not affect test status since the status
    // was set explicitly using a test macro above
    return 0;
}

bool return

#include <srtest.h>

bool MyTest::Test1()
{
    int sum = DoAdd(2, 2);
    // return true -> PASS, false -> FAIL
    return (sum == 4);
}

bool MyTest::Test2()
{
    int sum = DoAdd(3, 3);
    // 
    srEXPECT_EQ(sum, 6);
    // return value does not affect test status since the status
    // was set explicitly using a test macro above
    return true;
}