|
|
| (15 intermediate revisions by 4 users not shown) |
| Line 1: |
Line 1: |
| == Introduction == | | == Introduction == |
| The Test CClass Samples are part of the [[Test_Unit_Samples|STRIDE Test Unit Samples]]. The CClass naming convention implies a C struct that is acting like a class object for the purpose of test unit development. The Test CClass Samples pertain to implementing test units in the C language that appear to act like test classes. If you are using C++, you will want to use the [[Test_Class_Sample|Test Class Samples]] for simplicity. The Test CClass functionality is predominantly for legacy systems that have not switched over to C++. However, the Test CClass functionality is not restricted from compiling C++ environment as well.
| |
| <br><br>
| |
| The following content relates to the sample files and workspaces installed in ''%STRIDE_DIR%\Samples\TestUnits\TestCClass''. This sample consists of a [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio Visual Studio] workspace for building a [[Windows_Off-Target_Apps|Windows Off-Target App]], sample [[Test Units|scl_test_cclass]] source code, and a STRIDE workspace for doing more advanced test CClass execution.
| |
|
| |
|
| == Getting Started ==
| | The Test CClass test unit provides a test unit based on a C struct that is acting like a class object for the purpose of test unit containment. The Test CClass functionality is most useful for target environments that do not support C++. However, the Test CClass functionality may be used in C++ environments as well. (If you are using C++, please see the [[Test Class Sample]].) |
| To begin, open the Visual Studio Solution file in the sample directory. This solution (and corresponding project) were created for Visual Studio 2005. If you have a later version installed, you should be able to open this solution (it will be automatically upgraded if necessary). If you do not have any version of Visual Studio, it is recommended that you install the current free version of [http://en.wikipedia.org/wiki/Visual_Studio_Express Visual Studio Express].
| |
|
| |
|
| Once you have successfully opened the solution, rebuild it. The build process has custom STRIDE build rules integrated. The rebuilding will produce a STRIDE database, intercept module source files, and a Windows Off-Target App that incorporates the test class source.
| | == Tests Description == |
|
| |
|
| Once the build is complete, perform the following steps to run the test CClasss in the workspace:
| | === Basic === |
| | These examples cover the simplest, easiest way to code a STRIDE test CClass. These examples use simple [http://en.wikipedia.org/wiki/Plain_Old_Data_Structures POD] return types to indicate status and do not annotate the tests with any rich information (such as comments). |
|
| |
|
| # launch the Windows Off-Target App, TestCClass.exe. This will run in a standard console window.
| | ==== basic_simple ==== |
| # open a command prompt window and change to this sample's directory (%STRIDE_DIR%\Samples\TestUnits\TestCClass).
| | This example demonstrates passing and failing tests using the four primary integer (int, short, and char) types that infer status from. |
| # at the command prompt, run the command <tt>'''TestUnitRun.pl -v'''</tt>. This will execute all of the test units in the workspace and open a browser to display the results.
| |
| # quit the TestCClass.exe application by typing 'q' in its console window.
| |
|
| |
|
| == Sample Test CClass Examples == | | ==== basic_fixtures ==== |
| Now that you have built the Windows Off-Target App and executed the test CClass it contains, you can take time to peruse the test CClass source and the corresponding results that each produces. This section provides a brief description for each.
| | This example demonstrates how to use [[Test Unit Pragmas#Fixturing Pragmas|setup]] and [[Test Unit Pragmas#Fixturing Pragmas|teardown]] fixtures. The setup and teardown methods are called immediately before and after the execution of each test method, respectively. |
|
| |
|
| === Basic Examples === | | ==== basic_parameterized ==== |
| These examples cover the simplest, easiest way to code a STRIDE test CClass. These examples use simple [http://en.wikipedia.org/wiki/Plain_Old_Data_Structures POD] return types to indicate status and do not annotate the tests with any rich information (such as comments).
| |
|
| |
|
| ==== 01_01_Basic_Simple ====
| | This example demonstrates how to pass arguments to the initializer of your test CClass. This is something that is useful when you want to run the same test scenario with different sets of input data, for instance, as described by [http://xunitpatterns.com/Parameterized%20Test.html this pattern]. |
| This example demonstrates passing and failing tests using the four primary POD types that infer status from (int, short, and char). The bool type is accepted only in C++ mode. | |
|
| |
|
| ==== 01_02_Basic_Fixtures ====
| | === Runtime Services === |
| This example demonstrates how to use [[Test_Units#Pragmas_for_Test_Units|setup]] and [[Test_Units#Pragmas_for_Test_Units|teardown]] fixtures. The setup and teardown methods are called immediately before and after the execution of each test method, respectively.
| |
| | |
| ==== 01_03_Basic_Constructors ====
| |
| This example demonstrates how scl_test_cclass may be used with parameterized constructors. These arguments can be passed using our ascript scripting model for test units.
| |
| | |
| === Runtime Services Examples === | |
| These examples cover basic usage of our Runtime Test Services API (as declared in srtest.h). | | These examples cover basic usage of our Runtime Test Services API (as declared in srtest.h). |
|
| |
|
| ==== 02_01_RuntimeServices_Simple ==== | | ==== runtimeservices_simple ==== |
| This example demonstrates how to use [[Test_Units#srTestCaseSetStatus|srTestCaseSetStatus]] to set status, [[Test_Units#srTestCaseAddComment|srTestCaseAddComment]] to add a comment, and srTEST_ADD_COMMENT_WITH_LOCATION to add a comment that automatically includes line and file information. | | This example demonstrates how to use [[Runtime_Test_Services#srTestCaseSetStatus|srTestCaseSetStatus]] to set status and [[Runtime_Test_Services#srTestCaseAddAnnotation|srTestCaseAddAnnotation]] to add a comment. |
| | |
| ==== 02_02_RuntimeServices_Dynamic ====
| |
| This example demonstrates how to use [[Test_Units#srTestSuiteAddSuite|srTestSuiteAddSuite]], [[Test_Units#srTestSuiteAddCase|srTestSuiteAddCase]], [[Test_Units#srTestSuiteAddAnnotation|srTestSuiteAddAnnotation]], and [[Test_Units#srTestAnnotationAddComment|srTestAnnotationAddComment]] for dynamic suite, test case, and annotation creation in the context of a single test method.
| |
| | |
| ==== 02_03_RuntimeServices_Override ====
| |
| This example demonstrates how to use [[Test_Units#srTestCaseSetStatus|srTestCaseSetStatus]] to override the status that would otherwise be inferred from the return value.
| |
| | |
| ==== 02_04_RuntimeServices_VarComment ====
| |
| This example demonstrates the use of [http://en.wikipedia.org/wiki/Printf printf] style format strings with [[Test_Units#srTestCaseAddComment|srTestCaseAddComment]].
| |
| | |
| == Test CClass Execution ==
| |
| This sample demonstrates two different techniques for executing test classes.
| |
| | |
| === Commandline Execution ===
| |
| Command line execution for test CClasss is done using the [[Test_Runners#TestUnitRun.pl|TestUnitRun]] utility. Here are several examples with specific syntax to execute scl_test_cclass tests. All of these commands can be invoked from a standard command shell and the arguments shown assume that the commands are executed with the sample's directory as the starting directory. You must have your TestCClass.exe application running in order for the runner to be able to initiate a connection to the target simulator. In addition, you should verify that your %STRIDE_DIR%\bin\transport.cfg file is using the TCP transport to connect to port 8000 (these are the default settings when the product is installed).
| |
| | |
| ==== Simple execution of all test units ====
| |
| The following command executes all of the test units found in the STRIDE database you have previously generated. For the purpose of this sample, since there is only one database, the -d parameter is not strictly needed, but it is included for completeness.
| |
| | |
| TestUnitRun.pl -d TestCClass.sidb
| |
| | |
| This command executes all Test Units found in the database in descending alpha-numeric sort order. Any Test CClass initialization arguments are given default values (typically zero or NULL).
| |
| | |
| When you run this command, you should see the following:
| |
| | |
| Attempting connection using [Sockets (S2)] transport ...
| |
| Connected to device.
| |
| Initializing STRIDE database objects...
| |
| Done.
| |
| Running Test Basic_Constructors...
| |
| Running Test Basic_Fixtures...
| |
| Running Test Basic_Simple...
| |
| Running Test RuntimeServices_Dynamic...
| |
| Running Test RuntimeServices_Override...
| |
| Running Test RuntimeServices_Simple...
| |
| Running Test RuntimeServices_VarComment...
| |
| Disconnected from device.
| |
| Test Results saved to C:\STRIDE\Samples\TestUnits\TestCClass\TestCClass.xml
| |
| Test Report saved to C:\STRIDE\Samples\TestUnits\TestCClass\TestCClass.html
| |
| ***************************************************************************
| |
| Results Summary
| |
| ***************************************************************************
| |
| Passed: 16
| |
| Failed: 7
| |
| In Progress: 1
| |
| Not Applicable: 1
| |
| ...in 8 suites.
| |
| ***************************************************************************
| |
| | |
| ==== Execution based on an order file ====
| |
| TestUnitRun can optionally base its execution on simple text file input. A simple order file, ''RunSimple.txt'', is provided which specifies a subset of all the scl_test_cclass tests in this sample. This order file also shows how to create subsuites in the final output by using the special '''{suitepath}''' syntax, as described in [[Test_Runners#Usage|the usage section]].
| |
| | |
| TestUnitRun.pl -d TestCClass.sidb -o RunSimple.txt
| |
| | |
| ...and will produce this output:
| |
| | |
| Attempting connection using [Sockets (S2)] transport ...
| |
| Connected to device.
| |
| Initializing STRIDE database objects...
| |
| Done.
| |
| Running Test Basic_Simple...
| |
| Running Test RuntimeServices_Simple...
| |
| Disconnected from device.
| |
| Test Results saved to C:\STRIDE\Samples\TestUnits\TestCClass\TestCClass.xml
| |
| Test Report saved to C:\STRIDE\Samples\TestUnits\TestCClass\TestCClass.html
| |
| ***************************************************************************
| |
| Results Summary
| |
| ***************************************************************************
| |
| Passed: 8
| |
| Failed: 4
| |
| In Progress: 1
| |
| Not Applicable: 1
| |
| ...in 4 suites.
| |
| ***************************************************************************
| |
|
| |
|
| ==== Execution based on file system order ==== | | ==== runtimeservices_dynamic ==== |
| TestUnitRun can also try to infer execution order and suite hierarchy from filesystem organization. If you have organized your test CClass source files (only the files that contain the scl_test_CClass pragma matter here) in a filesystem hierarchy that you want to mimic in your tests, you can specify a root of a directory tree that contains the relevant test CClass source. TestUnitRun will walk the directory structure and determine order and hierarchy of tests based on the directory structure. To see an example of this in action, you can execute this command with the sample:
| | This example demonstrates how to use [[Runtime_Test_Services#srTestSuiteAddCase|srTestSuiteAddCase]], [[Runtime_Test_Services#srTestCaseAddAnnotation|srTestCaseAddAnnotation]], and [[Runtime_Test_Services#srTestAnnotationAddComment|srTestAnnotationAddComment]] for dynamic suite, test case, and annotation creation in the context of a single test method. |
|
| |
|
| TestUnitRun.pl -d TestCClass.sidb -f Tests
| | ==== runtimeservices_override ==== |
| | This example demonstrates how to use [[Runtime_Test_Services#srTestCaseSetStatus|srTestCaseSetStatus]] to override the status that would otherwise be inferred from the return value. |
|
| |
|
| This will cause the runner to examine the Tests subdirectory structure and build a hierarchy of tests based on that directory tree. Subdirectories will map to suites in the final report. Here is the output for this example: | | ==== runtimeservices_varcomment ==== |
|
| | This example demonstrates the use of [http://en.wikipedia.org/wiki/Printf printf] style format strings with [[Runtime_Test_Services#srTestCaseAddAnnotation|srTestCaseAddAnnotation]]. |
| Attempting connection using [Sockets (S2)] transport ...
| |
| Connected to device.
| |
| Initializing STRIDE database objects...
| |
| Done.
| |
| Running Test Basic_Simple...
| |
| Running Test Basic_Fixtures...
| |
| Running Test Basic_Constructors...
| |
| Running Test RuntimeServices_Simple...
| |
| Running Test RuntimeServices_Dynamic...
| |
| Running Test RuntimeServices_Override...
| |
| Running Test RuntimeServices_VarComment...
| |
| Disconnected from device.
| |
| Test Results saved to C:\STRIDE\Samples\TestUnits\TestCClass\TestCClass.xml
| |
| Test Report saved to C:\STRIDE\Samples\TestUnits\TestCClass\TestCClass.html
| |
| ***************************************************************************
| |
| Results Summary
| |
| ***************************************************************************
| |
| Passed: 16
| |
| Failed: 7
| |
| In Progress: 1
| |
| Not Applicable: 1
| |
| ...in 10 suites.
| |
| ***************************************************************************
| |
|
| |
|
| === Workspace-based Execution === | | == Run Tests == |
| TestCClass.ssw, a workspace in the TestCClass directory, demonstrates the use of script execution with Studio to manage test order and hierarchy. This workspace was created using [[WorkspaceSetup.pl]]. The setup and teardown folders provided basic infrastructure scripts that start/stop the simulator application (TestCClass.exe) and to manage traceviews used for [[Runtime_Reference#Logging_Services|srPrint]] message collection. The scripts that drive the testing are in the workspace '''test''' folder. What follows is a brief description for each.
| | Now launch the test app (if you have not already) and execute the runner with the following commands: |
|
| |
|
| ==== RunAll ==== | | ''Test C Class tests'': |
| This folder contains a script, All.js, that iterates through the entire collection of test units and executes them one at a time. The order of execution will be in ascending alpabetical order (by name) since the [[AutoScript#ascript.TestUnits|ArrangeBy]] collection method was called.
| | <pre> |
| | stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run="s2_testcclass_basic_fixtures; s2_testcclass_basic_parameterized(\"mystring\", 8); s2_testcclass_basic_simple" --output=CClass.xml |
| | </pre> |
|
| |
|
| ====CallConstructor====
| |
|
| |
|
| This folder contains a simple script example that shows how to invoke test classes with constructor arguments. In this example, the Basic_Constructors test class is executed twice with different initialization (constructor) arguments both times.
| | Note the command will produce distinct result files for the run (per the ''--output'' command above). Please use the result file to peruse the results by opening each in your browser. |
|
| |
|
| ==== Run Individual ==== | | == Observations == |
| This folder shows how to use individual scripts to execute test classes. Each script has the following form: | | This sample demonstrates a more sophisticated (and complicated) packaging technique for systems that support C compilation only. [[Test_Units#Test_Units|Test C Classes]] are defined by a structure of function pointers (which may also include data) and an initialization function. Review the source code in the directory and follow the sample description. |
|
| |
|
| ascript.TestUnits.Item('''TEST_CClass_NAME''').Run();
| | The following are some test observations: |
| | * the [[Scl_test_cclass|scl_test_cclass pragma]] requires a structure of function pointers as well as an initialization function that is called prior to running the tests. The initialization function must take the c class structure pointer as it's first argument and it will assign values to all the function pointer elements as well as perform any other initialization tasks. The pragma also accepts an optional deinitialization function that will be called after test execution (if provided). |
| | * we've provided documentation using doxygen formatting for these samples. Because the test functions themselves are bound at runtime, the test documentation must be associated with the function pointer elements in the structure - read more [[Test_API#Test_C-Classes|here]]. |
| | * parameterized tests are also supported by test c classes. Arguments to the initialization function that follow the structure pointer argument are considered constructor arguments and can be passed when running the test. |
| | * because the test methods are assigned to the structure members at runtime, it's possible (and recommended) to use statically scoped functions, so as not to pollute the global function space with test functions. That said, you are free to use any functions with matching signatures and linkage, regardless of scope. |
|
| |
|
| The '''TEST_CClass_NAME''' is the name of the scl_test_cclass test to be run. The order and hierarchy of each item may be changed via the Studio tree control by moving the item within the scripts and/or folders. The sample contains individual scripts for a few of the sample scl_test_cclass tests - you are free to move, add, or delete any items as you experiment with the workspace.
| |
|
| |
|
| [[Category: Samples]] | | [[Category: Samples]] |
Introduction
The Test CClass test unit provides a test unit based on a C struct that is acting like a class object for the purpose of test unit containment. The Test CClass functionality is most useful for target environments that do not support C++. However, the Test CClass functionality may be used in C++ environments as well. (If you are using C++, please see the Test Class Sample.)
Tests Description
Basic
These examples cover the simplest, easiest way to code a STRIDE test CClass. These examples use simple POD return types to indicate status and do not annotate the tests with any rich information (such as comments).
basic_simple
This example demonstrates passing and failing tests using the four primary integer (int, short, and char) types that infer status from.
basic_fixtures
This example demonstrates how to use setup and teardown fixtures. The setup and teardown methods are called immediately before and after the execution of each test method, respectively.
basic_parameterized
This example demonstrates how to pass arguments to the initializer of your test CClass. This is something that is useful when you want to run the same test scenario with different sets of input data, for instance, as described by this pattern.
Runtime Services
These examples cover basic usage of our Runtime Test Services API (as declared in srtest.h).
runtimeservices_simple
This example demonstrates how to use srTestCaseSetStatus to set status and srTestCaseAddAnnotation to add a comment.
runtimeservices_dynamic
This example demonstrates how to use srTestSuiteAddCase, srTestCaseAddAnnotation, and srTestAnnotationAddComment for dynamic suite, test case, and annotation creation in the context of a single test method.
runtimeservices_override
This example demonstrates how to use srTestCaseSetStatus to override the status that would otherwise be inferred from the return value.
This example demonstrates the use of printf style format strings with srTestCaseAddAnnotation.
Run Tests
Now launch the test app (if you have not already) and execute the runner with the following commands:
Test C Class tests:
stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run="s2_testcclass_basic_fixtures; s2_testcclass_basic_parameterized(\"mystring\", 8); s2_testcclass_basic_simple" --output=CClass.xml
Note the command will produce distinct result files for the run (per the --output command above). Please use the result file to peruse the results by opening each in your browser.
Observations
This sample demonstrates a more sophisticated (and complicated) packaging technique for systems that support C compilation only. Test C Classes are defined by a structure of function pointers (which may also include data) and an initialization function. Review the source code in the directory and follow the sample description.
The following are some test observations:
- the scl_test_cclass pragma requires a structure of function pointers as well as an initialization function that is called prior to running the tests. The initialization function must take the c class structure pointer as it's first argument and it will assign values to all the function pointer elements as well as perform any other initialization tasks. The pragma also accepts an optional deinitialization function that will be called after test execution (if provided).
- we've provided documentation using doxygen formatting for these samples. Because the test functions themselves are bound at runtime, the test documentation must be associated with the function pointer elements in the structure - read more here.
- parameterized tests are also supported by test c classes. Arguments to the initialization function that follow the structure pointer argument are considered constructor arguments and can be passed when running the test.
- because the test methods are assigned to the structure members at runtime, it's possible (and recommended) to use statically scoped functions, so as not to pollute the global function space with test functions. That said, you are free to use any functions with matching signatures and linkage, regardless of scope.