Test Class Sample

Introduction

The following content relates to the sample files and workspaces installed in %STRIDE_DIR%\Samples\TestUnits\TestClass. This sample consists of a Visual Studio workspace for building an off-target simulator, sample test class source code, and a STRIDE workspace for doing more advanced test class execution.

Getting Started

To begin, open the Visual Studio Solution file in the sample directory. This solution (and corresponding project) were created for Visual Studio 2005. If you have a later version of Visual Studio installed, you should be able to open this solution and it will be automatically upgraded if necessary. If you do not currently have any version of Visual Studio, we recommend that you install the current free (as in beer) version of Visual Studio Express.

Once you have successfully opened the solution, rebuild it. The build process has custom STRIDE build rules integrated and will produce a STRIDE database, intercept module source files, and an off-target simulator application that incorporates the test class source.

Once the build is complete, perform the following steps to run the test classes in the workspace:

1. launch the off-target simulator, TestClass.exe. This will run in a standard console window.
2. open a command prompt window and change to this sample's directory.
3. at the command prompt, run the command TestUnitRun.pl -v. This will execute all of the test units in the workspace and open a browser to display the results.
4. quit the TestClass.exe application by typing 'q' in its console window.

Sample Test Classes

Now that you have built the off-target simulator and executed the test classes it contains, you can take time to peruse the test class source and the corresponding results that each produces. This section provides a brief description for each.

NOTE: each of the example test classes is grouped in namespace corresponding to its top-level category (e.g. Basic or Runtime Services). This is something we have chosen to do for organizational purposes only -- it is not a general requirement that test classes be placed into namespaces.

Basic Examples

These examples cover the simplest, easiest way to code a STRIDE test class. These examples use simple POD return types to indicate status and do not annotate the tests with any rich information (such as comments).

01_01_Basic_Simple

Demonstrates passing and failing tests using the four primary POD types that we can infer status from (int, bool, short, and char).

01_02_Basic_Fixtures

Shows how to use setup and teardown fixtures. The setup and teardown methods are called immediately before and after the execution of each test method, respectively.

01_03_Basic_Exceptions

Demonstrates how exceptions thrown from a test method are caught by our intercept module and noted in the results. Any exception that is caught by the harness is assumed to indicate failure.

01_04_Basic_Constructors

Demonstrates how test classes may have non-trivial constructors. These arguments can be passed using our ascript scripting model for test units.

Runtime Services Examples

These examples cover basic usage of our Runtime Test Services API (as declared in srtest.h).

02_01_RuntimeServices_Simple

Shows how to use srTestCaseSetStatus to set status, srTestCaseAddComment to add a comment, and srTEST_ADD_COMMENT_WITH_LOCATION to add a comment that automatically includes line and file information.

02_02_RuntimeServices_Dynamic

Shows how to use srTestSuiteAddSuite and srTestSuiteAddTest for dynamic suite and test creation in the context of a single test method.

02_03_RuntimeServices_Override

Shows how to use srTestCaseSetStatus to override the status that would otherwise be inferred from the return value.

02_04_RuntimeServices_VarComment

Demonstrates the use of printf style format strings with srTestCaseAddComment.

srTest Examples

These examples show to how to use the stride::srTest base class for your test classes. When you publicly inherit from srTest, you get access to default testCase and testSuite members and their associated methods.

03_01_srTest_Simple

Demonstrates the use of testCase.setStatus to set the status for test cases.

03_02_srTest_Dynamic

Shows how to use testSuite.AddSuite and testSuite.AddTest for dynamic suite and test case creation within the context of one test method.

Test Class Execution

This sample demonstrates two different techniques for executing test classes.

Command Line Execution

Command line execution for test classes is done using the TestUnitRun utility. Here are several examples of specific syntax to execute test classes. All of these commands can be invoked from a standard command shell and the arguments shown assume that the commands are executed with the sample's directory as the starting directory. You must have your TestClass.exe application running in order for the runner to be able to initiate a connection to the target simulator. In addition, you should verify that your %STRIDE_DIR%\bin\transport.cfg file is using the TCP transport to connect to port 8000 (these are the default settings when the product is installed).

Simple execution of all test units

The following command executes all of the test units found in the STRIDE database you have previously generated. For the purpose of this sample, since there is only one database, the -d parameter is not strictly needed, but we show it here for completeness.

 TestUnitRun.pl -d TestClass.sidb

This command executes all Test Units found in the database in descending alpha-numeric sort order. Any Test Class initialization arguments are given default values (typically zero or NULL).

When you run this command, you should see console output like:

 Attempting connection using [Sockets (S2)] transport ...
 Connected to device.
 Initializing STRIDE database objects...
 Done.
 Running Test Basic::Constructors...
 Running Test Basic::Exceptions...
 Running Test Basic::Fixtures...
 Running Test Basic::Simple...
 Running Test RuntimeServices::Dynamic...
 Running Test RuntimeServices::Override... 
 Running Test RuntimeServices::Simple...
 Running Test RuntimeServices::VarComment...
 Running Test srTest::Dynamic...
 Running Test srTest::Simple...
 Disconnected from device.
 Test Results saved to C:\s2\seaside\Samples\TestUnits\TestClass\TestClass.xml
 Test Report saved to C:\s2\seaside\Samples\TestUnits\TestClass\TestClass.html
 ***************************************************************************
 Results Summary
 ***************************************************************************
   Passed:              28
   Failed:              12
   In Progress:          2
   Not Applicable:       2
   ...in 12 suites.
 ***************************************************************************

Execution based on an order file

TestUnitRun can optionally base its execution on simple text file input. We have provided a simple order file, RunSimple.txt, which specifies a subset of all the Test Classes in this sample. This order file also shows how to create subsuites in the final output by using the special {suitepath} syntax, as described in the usage section.

 TestUnitRun.pl -d TestClass.sidb -o RunSimple.txt

...and will produce this output:

 Attempting connection using [Sockets (S2)] transport ...
 Connected to device.
 Initializing STRIDE database objects...
 Done.
 Running Test Basic::Simple...
 Running Test RuntimeServices::Simple...
 Running Test srTest::Simple...
 Disconnected from device.
 Test Results saved to C:\s2\seaside\Samples\TestUnits\TestClass\TestClass.xml
 Test Report saved to C:\s2\seaside\Samples\TestUnits\TestClass\TestClass.html
 ***************************************************************************
 Results Summary
 ***************************************************************************
   Passed:              13
   Failed:               6
   In Progress:          2
   Not Applicable:       2
   ...in 6 suites.
 ***************************************************************************

Execution based on filesystem order

 Attempting connection using [Sockets (S2)] transport ...
 Connected to device.
 Initializing STRIDE database objects...
 Done.
 Running Test Basic::Simple...
 Running Test Basic::Fixtures...
 Running Test Basic::Exceptions...
 Running Test Basic::Constructors...
 Running Test RuntimeServices::Simple...
 Running Test RuntimeServices::Dynamic...
 Running Test RuntimeServices::Override...
 Running Test RuntimeServices::VarComment...
 Running Test srTest::Simple...
 Running Test srTest::Dynamic...
 Disconnected from device.
 Test Results saved to C:\s2\seaside\Samples\TestUnits\TestClass\TestClass.xml
 Test Report saved to C:\s2\seaside\Samples\TestUnits\TestClass\TestClass.html
 ***************************************************************************
 Results Summary
 ***************************************************************************
   Passed:              28
   Failed:              12
   In Progress:          2
   Not Applicable:       2
   ...in 15 suites.
 ***************************************************************************