Training Tests in C/C++ - Revision history

Ivailop: /* test_in_c_cpp/TestClass */

2018-12-28T19:01:59Z

test_in_c_cpp/TestClass

← Older revision		Revision as of 19:01, 28 December 2018
Line 43:		Line 43:

	* all of these example classes have been put into one or more namespaces. This is just for organization purposes, mainly to avoid name collisions when built along with lots of other test classes. Your test classes are '''not''' required to be in namespaces, but it can be helpful in avoiding collisions as the number of tests in your system grows.		* all of these example classes have been put into one or more namespaces. This is just for organization purposes, mainly to avoid name collisions when built along with lots of other test classes. Your test classes are '''not''' required to be in namespaces, but it can be helpful in avoiding collisions as the number of tests in your system grows.
	* we've documented our test classes and methods using [~~http~~://~~www~~.~~stack~~.nl/~~~dimitri/doxygen~~/ doxygen] style comments. This documentation is automatically extracted by our tools and added to the results report - more information about this feature is [[Test_API#Test_Documentation\|here]]		* we've documented our test classes and methods using [https://en.wikipedia.org/wiki/Doxygen doxygen] style comments. This documentation is automatically extracted by our tools and added to the results report - more information about this feature is [[Test_API#Test_Documentation\|here]]
	* you can optionally write test classes that inherit from a base class that we've defined ([[Runtime_Test_Services#class_srTest\|stride::srTest]]). We recommend you start by writing your classes this way so that your classes will inherit some methods and members that make some custom reporting tasks simpler.		* you can optionally write test classes that inherit from a base class that we've defined ([[Runtime_Test_Services#class_srTest\|stride::srTest]]). We recommend you start by writing your classes this way so that your classes will inherit some methods and members that make some custom reporting tasks simpler.
	* exceptions are generally handled by the STRIDE unit test harness, but can be disabled if your compiler does not support them (see ''s2_testclass_basic_exceptions_tests.h/cpp'').		* exceptions are generally handled by the STRIDE unit test harness, but can be disabled if your compiler does not support them (see ''s2_testclass_basic_exceptions_tests.h/cpp'').

Marku at 16:44, 29 August 2011

2011-08-29T16:44:20Z

← Older revision		Revision as of 16:44, 29 August 2011
Line 144:		Line 144:
	* the [[Test_Double_Samples\|TestDouble sample]] shows how to use STRIDE Test Doubles to replace function dependencies at runtime, typically for the purpose of isolating functions under test. This can be a powerful technique, but also requires some up-front work on your part to enable it, as this sample demonstrates.		* the [[Test_Double_Samples\|TestDouble sample]] shows how to use STRIDE Test Doubles to replace function dependencies at runtime, typically for the purpose of isolating functions under test. This can be a powerful technique, but also requires some up-front work on your part to enable it, as this sample demonstrates.

	[[Category: ~~Training~~]]		[[Category: Training_OLD]]

Ivailop at 17:41, 9 June 2011

2011-06-09T17:41:17Z

← Older revision		Revision as of 17:41, 9 June 2011
Line 26:		Line 26:
	Sure. Most of the disadvantages of native on-target tests concern the device build process. If your device build is particularly slow (on the order of hours or days), then adding and running new tests can become a tedious waiting game. Testing is always well served by shorter build cycles, and on-target tests are particularly sensitive to this.		Sure. Most of the disadvantages of native on-target tests concern the device build process. If your device build is particularly slow (on the order of hours or days), then adding and running new tests can become a tedious waiting game. Testing is always well served by shorter build cycles, and on-target tests are particularly sensitive to this.

	In some cases, the additional code space requirements of native tests is a concern, but this is also mitigated by ever-increasing device storage capacities. On platforms that support multiple processes (e.g. ~~embedded linux~~ or ~~WinMobile~~), it's possible to bundle tests into one or more separate test process, which further mitigates the code space concern by isolating the test code in one or more separate applications.		In some cases, the additional code space requirements of native tests is a concern, but this is also mitigated by ever-increasing device storage capacities. On platforms that support multiple processes (e.g. Linux or Windows), it's possible to bundle tests into one or more separate test process, which further mitigates the code space concern by isolating the test code in one or more separate applications.

	== Samples ==		== Samples ==

Marku: /* Run the tests */

2011-01-20T16:13:11Z

Run the tests

← Older revision		Revision as of 16:13, 20 January 2011
Line 102:		Line 102:
	</pre>		</pre>

	''~~C Class~~ tests'':		''Test FList tests'':
	<pre>		<pre>
	stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run="~~s2_testcclass_basic_fixtures~~; ~~s2_testcclass_basic_parameterized(\"mystring\", 8); s2_testcclass_basic_simple~~" --output=~~CClass~~.xml		stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run="s2_testflist_basic_fixtures; s2_testflist_basic_simple" --output=FList.xml
	</pre>		</pre>

	''~~FList~~ tests'':		''Test C Class tests'':
	<pre>		<pre>
	stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run="~~s2_testflist_basic_fixtures~~; ~~s2_testflist_basic_simple~~" --output=~~FList~~.xml		stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run="s2_testcclass_basic_fixtures; s2_testcclass_basic_parameterized(\"mystring\", 8); s2_testcclass_basic_simple" --output=CClass.xml
	</pre>		</pre>

Ivailop: /* test_in_c_cpp/FileServices */

2010-06-10T20:05:25Z

test_in_c_cpp/FileServices

← Older revision		Revision as of 20:05, 10 June 2010
Line 87:		Line 87:
	* most of the functions return an integer status code which should be checked - any nonzero status indicates a failure. We wrote a macro for this sample (''fsASSERT'') that checks return codes and adds any errors to the report. You might choose to do something similar, depending on your needs.		* most of the functions return an integer status code which should be checked - any nonzero status indicates a failure. We wrote a macro for this sample (''fsASSERT'') that checks return codes and adds any errors to the report. You might choose to do something similar, depending on your needs.
	* the file transfer API has both byte and line oriented read/write functions. You can use whichever functions are most appropriate for your needs.		* the file transfer API has both byte and line oriented read/write functions. You can use whichever functions are most appropriate for your needs.
	* this sample uses the local filesystem ([http://en.wikipedia.org/wiki/Stdio stdio]) to write some data to a tempfile - however the ~~stride~~ APIs are themselves buffer/byte oriented and don't require a local filesystem in general. If your device under test ''does'' have a filesystem, the STRIDE APIs can certainly be used to transfer resources to/from the device filesystem.		* this sample uses the local filesystem ([http://en.wikipedia.org/wiki/Stdio stdio]) to write some data to a tempfile - however the STRIDE APIs are themselves buffer/byte oriented and don't require a local filesystem in general. If your device under test ''does'' have a filesystem, the STRIDE APIs can certainly be used to transfer resources to/from the device filesystem.

	=== Build the test app ===		=== Build the test app ===

Ivailop: /* test_in_c_cpp/TestPoint */

2010-06-10T20:03:23Z

test_in_c_cpp/TestPoint

← Older revision		Revision as of 20:03, 10 June 2010
Line 75:		Line 75:
	''observations:''		''observations:''

	* test point tests can be packaged into a harness using any of the [[Test_Units#Test_Units\|three types of test units]] that we support. In this case, we used an FList so that the sample could be used on systems that were not c++ capable.		* test point tests can be packaged into a harness using any of the [[Test_Units#Test_Units\|three types of test units]] that we support. In this case, we used an FList so that the sample could be used on systems that were not C++ capable.
	* one of two methods is used to process the test: [[Test_Point_Testing_in_C/C++#srTestPointWait\|srTestPointWait]] or [[Test_Point_Testing_in_C/C++#srTestPointCheck\|srTestPointCheck]]. The former is used to process test points as they happen (with a specified timeout) and the latter is used to process test points that have already occurred at the time it is called (post completion check).		* one of two methods is used to process the test: [[Test_Point_Testing_in_C/C++#srTestPointWait\|srTestPointWait]] or [[Test_Point_Testing_in_C/C++#srTestPointCheck\|srTestPointCheck]]. The former is used to process test points as they happen (with a specified timeout) and the latter is used to process test points that have already occurred at the time it is called (post completion check).
	* due to the limitations of c syntax, it can be ugly to create the srTestPointExpect_t data, especially where user data validation is concerned (see the ''CheckData'' example, for instance).		* due to the limitations of C syntax, it can be ugly to create the srTestPointExpect_t data, especially where user data validation is concerned (see the ''CheckData'' example, for instance).

	=== test_in_c_cpp/FileServices ===		=== test_in_c_cpp/FileServices ===

Mikee at 21:10, 4 June 2010

2010-06-04T21:10:46Z

← Older revision		Revision as of 21:10, 4 June 2010
Line 32:		Line 32:
	For this training, we will be using some of the samples provided in the [[C/C++_Samples]]. For any sample that we don't cover here explicitly, feel free to explore the sample yourself. All of the samples can be easily built and executed using the STRIDE [[Off-Target Environment]].		For this training, we will be using some of the samples provided in the [[C/C++_Samples]]. For any sample that we don't cover here explicitly, feel free to explore the sample yourself. All of the samples can be easily built and executed using the STRIDE [[Off-Target Environment]].

	The first three samples that we cover are introductions to the different test unit packaging mechanisms that we support in STRIDE. A good overview of the pros and cons for each type is presented [[Test_Units#Test_Units\|here]]. The fourth sample demonstrates test point testing in native code on target (i.e. both the generation and ''validation'' of the test points are done on target). The last ~~two samples cover more advanced topics:~~ [[File_Transfer_Services\|file transfer services~~]] and [[Using_Test_Doubles\|function doubling~~]]. STRIDE file transfer enable reading/writing of files on the host from the device under test and can be useful for implementing data driven test scenarios (for instance, media file playback)~~. The function doubling sample cover an advanced testing technique whereby you can temporarily replace specific functions with another version for the purpose of testing~~.		The first three samples that we cover are introductions to the different test unit packaging mechanisms that we support in STRIDE. A good overview of the pros and cons for each type is presented [[Test_Units#Test_Units\|here]]. The fourth sample demonstrates test point testing in native code on target (i.e. both the generation and ''validation'' of the test points are done on target). The last sample covers [[File_Transfer_Services\|file transfer services]]. The STRIDE file transfer APIs enable reading/writing of files on the host from the device under test and can be useful for implementing data driven test scenarios (for instance, media file playback).

	'''Note:''' each of the three packaging examples include samples that cover ''basic'' usage and more advanced reporting techniques (''runtimeservices''). We recommend for this training that you focus on the ''basic'' samples as they cover the important packaging concepts. The ''runtimeservices'' examples are relevant only if the built-in reporting techniques are not sufficient for your reporting needs.		'''Note:''' each of the three packaging examples include samples that cover ''basic'' usage and more advanced reporting techniques (''runtimeservices''). We recommend for this training that you focus on the ''basic'' samples as they cover the important packaging concepts. The ''runtimeservices'' examples are relevant only if the built-in reporting techniques are not sufficient for your reporting needs.
Line 88:		Line 88:
	* the file transfer API has both byte and line oriented read/write functions. You can use whichever functions are most appropriate for your needs.		* the file transfer API has both byte and line oriented read/write functions. You can use whichever functions are most appropriate for your needs.
	* this sample uses the local filesystem ([http://en.wikipedia.org/wiki/Stdio stdio]) to write some data to a tempfile - however the stride APIs are themselves buffer/byte oriented and don't require a local filesystem in general. If your device under test ''does'' have a filesystem, the STRIDE APIs can certainly be used to transfer resources to/from the device filesystem.		* this sample uses the local filesystem ([http://en.wikipedia.org/wiki/Stdio stdio]) to write some data to a tempfile - however the stride APIs are themselves buffer/byte oriented and don't require a local filesystem in general. If your device under test ''does'' have a filesystem, the STRIDE APIs can certainly be used to transfer resources to/from the device filesystem.

	~~=== test_in_c_cpp/TestDouble ===~~

	~~TBD~~

	=== Build the test app ===		=== Build the test app ===
Line 140:		Line 136:

	Explore all the results and make sure that the results meet your expectations based on the test source that you've previously browsed.		Explore all the results and make sure that the results meet your expectations based on the test source that you've previously browsed.

			=== Other Samples ===

			We've omitted a few samples from this training that cover more advanced and (perhaps) less widely used features. We encourage you to investigate these samples on your own if you are interested - in particular:

			* each of the test packaging samples (''TestClass'', ''TestCClass'', and ''TestFList'') includes examples of using the runtime test APIs to do more advanced reporting (dynamic suite creation, for example). These techniques are applicable for same data driven test scenarios.
			* the [[Test_Double_Samples\|TestDouble sample]] shows how to use STRIDE Test Doubles to replace function dependencies at runtime, typically for the purpose of isolating functions under test. This can be a powerful technique, but also requires some up-front work on your part to enable it, as this sample demonstrates.

	[[Category: Training]]		[[Category: Training]]

Mikee: /* Run the tests */

2010-06-04T20:42:20Z

Run the tests

← Older revision		Revision as of 20:42, 4 June 2010
Line 121:		Line 121:
	</pre>		</pre>

			''File Services tests'':
	''~~Test Point~~ tests'':
	<pre>		<pre>
	stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run="s2_fileservices_basic" --output=FileServices.xml		stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run="s2_fileservices_basic" --output=FileServices.xml

Mikee: /* Samples */

2010-06-04T20:41:58Z

Samples

← Older revision		Revision as of 20:41, 4 June 2010
Line 32:		Line 32:
	For this training, we will be using some of the samples provided in the [[C/C++_Samples]]. For any sample that we don't cover here explicitly, feel free to explore the sample yourself. All of the samples can be easily built and executed using the STRIDE [[Off-Target Environment]].		For this training, we will be using some of the samples provided in the [[C/C++_Samples]]. For any sample that we don't cover here explicitly, feel free to explore the sample yourself. All of the samples can be easily built and executed using the STRIDE [[Off-Target Environment]].

	The first three samples that we cover are introductions to the different test unit packaging mechanisms that we support in STRIDE. A good overview of the pros and cons for each type is presented [[Test_Units#Test_Units\|here]]. The ~~last sample we discuss is the TestPoint~~ sample~~, which~~ demonstrates test point testing in native code on target (i.e. both the generation and ''validation'' of the test points are done on target).		The first three samples that we cover are introductions to the different test unit packaging mechanisms that we support in STRIDE. A good overview of the pros and cons for each type is presented [[Test_Units#Test_Units\|here]]. The fourth sample demonstrates test point testing in native code on target (i.e. both the generation and ''validation'' of the test points are done on target). The last two samples cover more advanced topics: [[File_Transfer_Services\|file transfer services]] and [[Using_Test_Doubles\|function doubling]]. STRIDE file transfer enable reading/writing of files on the host from the device under test and can be useful for implementing data driven test scenarios (for instance, media file playback). The function doubling sample cover an advanced testing technique whereby you can temporarily replace specific functions with another version for the purpose of testing.

	'''Note:''' each of the packaging examples include samples that cover ''basic'' usage and more advanced reporting techniques (''runtimeservices''). We recommend for this training that you focus on the ''basic'' samples as they cover the important packaging concepts. The ''runtimeservices'' examples are relevant only if the built-in reporting techniques are not sufficient for your reporting needs.		'''Note:''' each of the three packaging examples include samples that cover ''basic'' usage and more advanced reporting techniques (''runtimeservices''). We recommend for this training that you focus on the ''basic'' samples as they cover the important packaging concepts. The ''runtimeservices'' examples are relevant only if the built-in reporting techniques are not sufficient for your reporting needs.

	=== test_in_c_cpp/TestClass ===		=== test_in_c_cpp/TestClass ===
Line 71:		Line 71:
	=== test_in_c_cpp/TestPoint ===		=== test_in_c_cpp/TestPoint ===

	~~The last~~ sample ~~we'll consider~~ demonstrates how to do tests that validate [[Test_Point_Testing_in_C/C++\|STRIDE Test Points]] - with native test code. Although test point tests can be written in host-based ~~scriping~~ languages as well, sometimes it's preferable to write (and execute) the test logic in native target code - for instance, when validating large or otherwise complex data payloads. Review the source code in the directory and follow the sample description [[Test Point Samples\|here]].		This sample demonstrates how to do tests that validate [[Test_Point_Testing_in_C/C++\|STRIDE Test Points]] - with native test code. Although test point validation tests can be written in host-based scripting languages as well, sometimes it's preferable to write (and execute) the test logic in native target code - for instance, when validating large or otherwise complex data payloads. Review the source code in the directory and follow the sample description [[Test Point Samples\|here]].

	''observations:''		''observations:''
Line 78:		Line 78:
	* one of two methods is used to process the test: [[Test_Point_Testing_in_C/C++#srTestPointWait\|srTestPointWait]] or [[Test_Point_Testing_in_C/C++#srTestPointCheck\|srTestPointCheck]]. The former is used to process test points as they happen (with a specified timeout) and the latter is used to process test points that have already occurred at the time it is called (post completion check).		* one of two methods is used to process the test: [[Test_Point_Testing_in_C/C++#srTestPointWait\|srTestPointWait]] or [[Test_Point_Testing_in_C/C++#srTestPointCheck\|srTestPointCheck]]. The former is used to process test points as they happen (with a specified timeout) and the latter is used to process test points that have already occurred at the time it is called (post completion check).
	* due to the limitations of c syntax, it can be ugly to create the srTestPointExpect_t data, especially where user data validation is concerned (see the ''CheckData'' example, for instance).		* due to the limitations of c syntax, it can be ugly to create the srTestPointExpect_t data, especially where user data validation is concerned (see the ''CheckData'' example, for instance).

			=== test_in_c_cpp/FileServices ===

			The FileServices sample demonstrates basic usage of the [[File_Transfer_Services\|STRIDE File Transfer APIs]], which provide a way to transfer files to/from the host to the running device. All the data for the file transmission is done using STRIDE messaging between the STRIDE Runner on the host and the STRIDE Runtime on the device, so no additional communication ports are required to use these services. Review the source code in the directory and follow the sample description [[File Services Samples\|here]].

			''observations:''

			* most of the functions return an integer status code which should be checked - any nonzero status indicates a failure. We wrote a macro for this sample (''fsASSERT'') that checks return codes and adds any errors to the report. You might choose to do something similar, depending on your needs.
			* the file transfer API has both byte and line oriented read/write functions. You can use whichever functions are most appropriate for your needs.
			* this sample uses the local filesystem ([http://en.wikipedia.org/wiki/Stdio stdio]) to write some data to a tempfile - however the stride APIs are themselves buffer/byte oriented and don't require a local filesystem in general. If your device under test ''does'' have a filesystem, the STRIDE APIs can certainly be used to transfer resources to/from the device filesystem.

			=== test_in_c_cpp/TestDouble ===

			TBD

	=== Build the test app ===		=== Build the test app ===
Line 106:		Line 120:
	stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run="s2_testpoint_basic" --log_level=all --output=TestPoint.xml		stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run="s2_testpoint_basic" --log_level=all --output=TestPoint.xml
	</pre>		</pre>


			''Test Point tests'':
			<pre>
			stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run="s2_fileservices_basic" --output=FileServices.xml
			</pre>


	These commands will produce distinct result files for each run (per the ''--output'' command above). Please use these result files to peruse the results by opening each in your browser.		These commands will produce distinct result files for each run (per the ''--output'' command above). Please use these result files to peruse the results by opening each in your browser.
Line 111:		Line 132:
	=== Examine the results ===		=== Examine the results ===

	Open the ~~''TestApp.xml'' file~~ and browse the results.		Open the result files created above and browse the results.

	''observations:''		''observations:''

	* there are four top-level suites corresponding to the four samples we discussed above. The command arguments we passed to the runner created these top-level suites.
	* the test documentation has been extracted (at compile time) and is attached to the results when the tests are executed. Most of the test suites/test cases should have documentation in the description field.		* the test documentation has been extracted (at compile time) and is attached to the results when the tests are executed. Most of the test suites/test cases should have documentation in the description field.
	* the test point tests cases show the test points that were encountered, information about failures (if any) and log messages (the latter only because we included the <tt>--log_level=all</tt> option when executing the runner).		* the test point tests cases show the test points that were encountered, information about failures (if any) and log messages (the latter only because we included the <tt>--log_level=all</tt> option when executing the runner).

Mikee: /* Run the tests */

2010-06-04T00:28:37Z

Run the tests

← Older revision		Revision as of 00:28, 4 June 2010
Line 85:		Line 85:
	=== Run the tests ===		=== Run the tests ===

	Now launch the test app (if you have not already) and execute the runner with the following ~~command~~:		Now launch the test app (if you have not already) and execute the runner with the following commands:

			''Test Class tests'':
	<pre>		<pre>
	stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run="/TestPoint{s2_testpoint_basic}" --run="/Test FList{s2_testflist_basic_fixtures; s2_testflist_basic_simple}" --run="/Test C Class{s2_testcclass_basic_fixtures; s2_testcclass_basic_parameterized(\"mystring\", 8); s2_testcclass_basic_simple}" --run="/Test Class{s2_testclass::Basic::Exceptions; s2_testclass::Basic::Fixtures; s2_testclass::Basic::Parameterized; s2_testclass::Basic::Simple}" --~~log_level~~=~~all~~		stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run="s2_testclass::Basic::Exceptions; s2_testclass::Basic::Fixtures; s2_testclass::Basic::Parameterized; s2_testclass::Basic::Simple" --output=TestClass.xml
	</pre>		</pre>

	~~This~~ command ~~line organizes~~ the ~~tests~~ in ~~the four samples above into suites for easier browsing~~.		''C Class tests'':
			<pre>
			stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run="s2_testcclass_basic_fixtures; s2_testcclass_basic_parameterized(\"mystring\", 8); s2_testcclass_basic_simple" --output=CClass.xml
			</pre>

			''FList tests'':
			<pre>
			stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run="s2_testflist_basic_fixtures; s2_testflist_basic_simple" --output=FList.xml
			</pre>

			''Test Point tests'':
			<pre>
			stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run="s2_testpoint_basic" --log_level=all --output=TestPoint.xml
			</pre>

			These commands will produce distinct result files for each run (per the ''--output'' command above). Please use these result files to peruse the results by opening each in your browser.

	=== Examine the results ===		=== Examine the results ===