Test Macros Sample
Introduction
The Test Macros Samples are part of the STRIDE Test Unit Samples. The following content relates to the sample files and workspaces installed in %STRIDE_DIR%\Samples\TestUnits\TestClassMacros. This sample consists of a Visual Studio workspace for building a Windows Off-Target App, sample test class macros source code, and a STRIDE workspace for doing more advanced test class execution.
Getting Started
To begin, open the Visual Studio Solution file in the sample directory. This solution (and corresponding project) were created for Visual Studio 2005. If you have a later version of Visual Studio installed, you should be able to open this solution and it will be automatically upgraded if necessary. If you do not currently have any version of Visual Studio, it is recommended that you install the current free version of Visual Studio Express.
Once you have successfully opened the solution, rebuild it. The build process has custom STRIDE build rules integrated and will produce a STRIDE database, intercept module source files, and a Windows Off-Target App that incorporates the test class source.
Once the build is complete, perform the following steps to run the test classes in the workspace:
- launch the Windows Off-Target App, TestClass.exe. This will run in a standard console window.
- open a command prompt window and change to this sample's directory.
- at the command prompt, run the command TestUnitRun.pl -v. This will execute all of the test units in the workspace and open a browser to display the results.
- quit the TestClass.exe application by typing 'q' in its console window.
Sample Test Class Macros
Now that you have built the Windows Off-Target App and executed the test classes it contains, you can take time to peruse the test class source and the corresponding results that each produces. This section provides a brief description for each.
NOTE: each of the example test classes is grouped in the Basic namespace. This is for organizational purposes only -- it is not a general requirement that test classes be placed into namespaces.
Basic Examples
These examples cover simple uses of each of the macro families. The names of the test methods contain either Pass or Fail. All methods containing Fail illustrates uses of test macros that result in failures. Methods containing Pass illustrate passing uses.
01_01_Basic_ExpectBool
This example demonstrates uses of the srEXPECT_TRUE() and srEXPECT_FALSE() test class macros.
01_02_Basic_Comparison
This example demonstrates uses of the srEXPECT_EQ(), srEXPECT_NE(), srEXPECT_GT(), srEXPECT_GE(), srEXPECT_LT() and srEXPECT_LE() macros.
01_03_Basic_CString
This example demonstrates use of the C-string (zero terminated character sequence) macros srEXPECT_STREQ(), srEXPECT_STRNE(), srEXPECT_STRCASEEQ() and srEXPECT_STRCASENE().
01_04_Basic_Exceptions
This example demonstrates use of the exception verification macros srEXPECT_THROW(), srEXPECT_THROW_ANY() and srEXPECT_NO_THROW().
01_05_Basic_Predicates
This example demonstrates use of the predicate based macros srEXPECT_PRED<n>().
01_06_Basic_FloatingPt
This example demonstrates use of the macro used to test equality (or near equality) of floating point values srEXPECT_NEAR().
01_07_Basic_Assert
This example illustrates the use of the assertion macros (srASSERT_xx) which, in contrast to the expectation macros (srEXPECT_xx), cause the rest of the test case code to by bypassed.
Test Class Execution
This sample demonstrates two different techniques for executing test classes.
Command Line Execution
Command line execution for test classes is done using the TestUnitRun utility. Here are several examples of specific syntax to execute test classes. All of these commands can be invoked from a standard command shell (or other shell of your choosing) and the arguments shown assume that the commands are executed with the sample's directory as the starting directory. You must have your TestClass.exe application running in order for the runner to be able to initiate a connection to the target simulator. In addition, you should verify that your %STRIDE_DIR%\bin\transport.cfg file is using the TCP transport to connect to port 8000 (these are the default settings when the product is installed).
Simple execution of all test units
The following command executes all of the test units found in the STRIDE database you have previously generated. For the purpose of this sample, since there is only one database, the -d parameter is not strictly needed, but it is shown here for completeness.
TestUnitRun.pl -d TestClass.sidb
This command executes all Test Units found in the database in descending alpha-numeric sort order. Any Test Class initialization arguments are given default values (typically zero or NULL).
When you run this command, you should see console output like:
Attempting connection using [Sockets (S2)] transport ... Connected to device. Initializing STRIDE database objects... Done. Running Test Basic::Constructors... Running Test Basic::Exceptions... Running Test Basic::Fixtures... Running Test Basic::Simple... Running Test RuntimeServices::Dynamic... Running Test RuntimeServices::Override... Running Test RuntimeServices::Simple... Running Test RuntimeServices::VarComment... Running Test srTest::Dynamic... Running Test srTest::Simple... Disconnected from device. Test Results saved to C:\STRIDE\Samples\TestUnits\TestClass\TestClass.xml Test Report saved to C:\STRIDE\Samples\TestUnits\TestClass\TestClass.html *************************************************************************** Results Summary *************************************************************************** Passed: 26 Failed: 12 In Progress: 2 Not Applicable: 2 ...in 12 suites. ***************************************************************************
Execution based on an order file
TestUnitRun can optionally base its execution on simple text file input. A simple order file, RunSimple.txt, is provided which specifies a subset of all the Test Classes in this sample. This order file also shows how to create subsuites in the final output by using the special {suitepath} syntax, as described in the usage section.
TestUnitRun.pl -d TestClass.sidb -o RunSimple.txt
...and will produce this output:
Attempting connection using [Sockets (S2)] transport ... Connected to device. Initializing STRIDE database objects... Done. Running Test Basic::Simple... Running Test RuntimeServices::Simple... Running Test srTest::Simple... Disconnected from device. Test Results saved to C:\STRIDE\Samples\TestUnits\TestClass\TestClass.xml Test Report saved to C:\STRIDE\Samples\TestUnits\TestClass\TestClass.html *************************************************************************** Results Summary *************************************************************************** Passed: 13 Failed: 6 In Progress: 2 Not Applicable: 2 ...in 6 suites. ***************************************************************************
Execution based on filesystem order
TestUnitRun can also try to infer execution order and suite hierarchy from filesystem organization. If you have organized your test class source files (only the files that contain the scl_test_class pragma matter here) in a filesystem hierarchy that you want to mimic in your tests, you can specify a root of a directory tree that contains the relevant test class source. TestUnitRun will walk the directory structure and determine order and hierarchy of tests based on the directory structure. To see an example of this in action, you can execute this command with the sample:
TestUnitRun.pl -d TestClass.sidb -f Tests
This will cause the runner to examine the Tests subdirectory structure and build a hierarchy of tests based on that directory tree. Subdirectories will map to suites in the final report. Here is the output for this example:
Attempting connection using [Sockets (S2)] transport ... Connected to device. Initializing STRIDE database objects... Done. Running Test Basic::Simple... Running Test Basic::Fixtures... Running Test Basic::Exceptions... Running Test Basic::Constructors... Running Test RuntimeServices::Simple... Running Test RuntimeServices::Dynamic... Running Test RuntimeServices::Override... Running Test RuntimeServices::VarComment... Running Test srTest::Simple... Running Test srTest::Dynamic... Disconnected from device. Test Results saved to C:\STRIDE\Samples\TestUnits\TestClass\TestClass.xml Test Report saved to C:\STRIDE\Samples\TestUnits\TestClass\TestClass.html *************************************************************************** Results Summary *************************************************************************** Passed: 26 Failed: 12 In Progress: 2 Not Applicable: 2 ...in 15 suites. ***************************************************************************
Workspace-based Execution
TestClass.ssw, a workspace in the TestClass directory, demonstrates the use of script execution with Studio to manage test order and hierarchy. This workspace was created using WorkspaceSetup.pl. The setup and teardown folders provide basic infrastructure scripts that start/stop the simulator application (TestClass.exe) and to manage traceviews used for srPrint message collection. The scripts that drive the testing are in the workspace test folder. What follows is a brief description for each.
RunAll
This folder contains a script, All.js, that iterates through the entire collection of test units and executes them one at a time. The order of execution will be in ascending alpabetical order (by name) since the ArrangeBy collection method was called.
CallConstructor
This folder contains a simple script example that shows how to invoke test classes with constructor arguments. In this example, the Basic::Constructors test class is executed twice with different initialization (constructor) arguments both times.
Run Individual
This folder shows how to use individual scripts to execute test classes. Each script has the following form:
ascript.TestUnits.Item(TEST_CLASS_NAME).Run();
The TEST_Class_NAME is the name of the scl_test_class test to be run. The order and hierarchy of each item may be changed via the Studio tree control by moving the item within the scripts and/or folders. The sample contains individual scripts for a few of the sample scl_test_class tests - you are free to move, add, or delete any items as you experiment with the workspace.