Training Running Tests - Revision history

Marku: /* Notes */

2011-10-19T22:05:28Z

Notes

← Older revision		Revision as of 22:05, 19 October 2011
Line 149:		Line 149:
	<references/>		<references/>

	[[Category: Training]]		[[Category: Training OLD]]

Marku: /* Using options files */

2011-03-25T22:00:12Z

Using options files

← Older revision		Revision as of 22:00, 25 March 2011
Line 130:		Line 130:

	<pre>		<pre>
	--run=/BasicTests{s2_testclass::Basic::Exceptions;s2_testclass::Basic::Fixtures; s2_testclass::Basic::Parameterized;s2_testclass::Basic::Simple}		--run=/BasicTests{s2_testclass::Basic::Exceptions;s2_testclass::Basic::Fixtures;s2_testclass::Basic::Parameterized;s2_testclass::Basic::Simple}
	--run=/srTest{s2_testclass::srTest::Dynamic;s2_testclass::srTest::Simple}		--run=/srTest{s2_testclass::srTest::Dynamic;s2_testclass::srTest::Simple}
	</pre>		</pre>

Ivailop: /* Background */

2010-06-10T21:16:52Z

Background

← Older revision		Revision as of 21:16, 10 June 2010
Line 1:		Line 1:
	== Background ==		== Background ==

	Device connection and STRIDE test execution is handled by the [[STRIDE Runner]] (aka ''"the runner"''). The runner is a command line tool with a number of options for listing runnable items, executing tests, tracing on test points, and uploading your results. The runner is designed for use in both ad-hoc execution of tests and fully automated CI execution. You've already seen some basic test execution scenarios in the other training sections - now we will look explicitly at several of the most common use-cases for the runner.		Device connection and STRIDE test execution is handled by the [[STRIDE Runner]] (aka ''"the runner"''). The runner is a command line tool with a number of options for listing runnable items, executing tests, tracing on test points, and uploading your results. The runner is designed for use in both ad-hoc execution of tests and fully [[Setting_up_your_CI_Environment\|automated CI]] execution. You've already seen some basic test execution scenarios in the other training sections - now we will look explicitly at several of the most common use-cases for the runner.

	Please review the following reference articles before proceeding:		Please review the following reference articles before proceeding:

Mikee at 22:09, 4 June 2010

2010-06-04T22:09:58Z

← Older revision		Revision as of 22:09, 4 June 2010
Line 122:		Line 122:

	If you plan to use this functionality to organize your tests into sub-suites, we recommend that you create options files to specify test unit groupings. This makes it easier to update and manage the suite hierarchy for your tests.		If you plan to use this functionality to organize your tests into sub-suites, we recommend that you create options files to specify test unit groupings. This makes it easier to update and manage the suite hierarchy for your tests.

			== Using options files ==

			The runner also supports options files which allow you to place commonly used or lengthy command line options in a file. Let's run the same example as above (using suites for organization) - however, this time we'll put the <tt>--run</tt> arguments into a file.

			First, let's create a text file with the <tt>--run</tt> arguments in it - call it ''run_suites.opt'' and add these lines:

			<pre>
			--run=/BasicTests{s2_testclass::Basic::Exceptions;s2_testclass::Basic::Fixtures; s2_testclass::Basic::Parameterized;s2_testclass::Basic::Simple}
			--run=/srTest{s2_testclass::srTest::Dynamic;s2_testclass::srTest::Simple}
			</pre>

			When creating an options file, you can use newlines to separate individual options for easier maintenance and readability. Also, you'll notice we've omitted the quotation marks around the strings we pass to each option since they are generally not needed (they are needed on the command line to allow the command shell to pass them along).

			Once you have created the options file, you can use it with the runner like:

			<pre>
			stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --options_file=run_suites.opt
			</pre>

			When you execute this, you should see the same results as in the previous section.

			Options files are a convenient way to persist and reuse common command line settings to the runner or to organize the way in which your individual test units will be run. Once you have lots of groups adding tests to your system, options files will provide a manageable way to group subsets of tests during execution. Option files also lend themselves to persisting/reusing your test space upload settings (if you are uploading), which typically do not change often.

	== Notes ==		== Notes ==

Marku at 17:15, 3 June 2010

2010-06-03T17:15:38Z

← Older revision		Revision as of 17:15, 3 June 2010
Line 16:		Line 16:
	== Listing items ==		== Listing items ==

	Listing the contents of database is an easy way to see what test units and fixturing functions are available for execution. Open a command line and try the following<ref name="database">These examples assume you are executing the runner from the <tt>~~src~~</tt> directory of your off-target framework. If that's not the case, you will need to adjust the database path accordingly.</ref>:		Listing the contents of database is an easy way to see what test units and fixturing functions are available for execution. Open a command line and try the following<ref name="database">These examples assume you are executing the runner from the <tt>sample_src</tt> directory of your off-target framework. If that's not the case, you will need to adjust the database path accordingly.</ref>:

	<pre>		<pre>

Marku: /* Organizing with suites */

2010-06-03T17:12:59Z

Organizing with suites

← Older revision		Revision as of 17:12, 3 June 2010
Line 105:		Line 105:
	== Organizing with suites ==		== Organizing with suites ==

	Now let's briefly at how you can use the runner to organize subsets of test units into suites. First, let's run our current set of test units without any explicit suite hierarchy:		Now let's briefly describe how you can use the runner to organize subsets of test units into suites. First, let's run our current set of test units without any explicit suite hierarchy:

	<pre>		<pre>
Line 121:		Line 121:
	Now when you view the results, you will see two top-level suites - '''BasicTests''' and '''srTest''' - and within those are the suites for the test units that we specified to be within each suite.		Now when you view the results, you will see two top-level suites - '''BasicTests''' and '''srTest''' - and within those are the suites for the test units that we specified to be within each suite.

	If you plan to use this functionality to organize your tests into ~~subsuites~~, we recommend that you create options files to specify test unit groupings. This makes it easier to update and manage the suite hierarchy for your tests.		If you plan to use this functionality to organize your tests into sub-suites, we recommend that you create options files to specify test unit groupings. This makes it easier to update and manage the suite hierarchy for your tests.

	== Notes ==		== Notes ==

Marku: /* Tracing on test points */

2010-06-03T17:11:27Z

Tracing on test points

← Older revision		Revision as of 17:11, 3 June 2010
Line 101:		Line 101:
	</pre>		</pre>

	The <tt>--trace</tt> argument accepts a filter expression which takes the form of a [http://en.wikipedia.org/wiki/Regular_expression regular expression] that is applied to the test point label. In this case, we've specified a filter that permits any test points that begin with ''ACTIVE''. Filtering gives you a convenient way to inspect ~~quickly inspet~~ specific behavioral aspects of your ~~STRIDE~~ instrumented software.		The <tt>--trace</tt> argument accepts a filter expression which takes the form of a [http://en.wikipedia.org/wiki/Regular_expression regular expression] that is applied to the test point label. In this case, we've specified a filter that permits any test points that begin with ''ACTIVE''. Filtering gives you a convenient way to quickly inspect specific behavioral aspects of your instrumented software.

	== Organizing with suites ==		== Organizing with suites ==

Marku: /* Build a test app */

2010-06-03T17:08:17Z

Build a test app

← Older revision		Revision as of 17:08, 3 June 2010
Line 12:		Line 12:
	== Build a test app ==		== Build a test app ==

	Let's begin by building an off-target test app to use for these examples. The sources we want to include in this app are <tt>test_in_script/Expectations</tt> and <tt>test_in_c_cpp/TestClass</tt>. Copy these source files to your <tt>sample_src</tt> directory and follow [[Off-Target_Test_App#~~Build_Steps~~\|these instructions for building]]		Let's begin by building an off-target test app to use for these examples. The sources we want to include in this app are <tt>test_in_script/Expectations</tt> and <tt>test_in_c_cpp/TestClass</tt>. Copy these source files to your <tt>sample_src</tt> directory and follow [[Off-Target_Test_App#Copy_Source_Sample\|these instructions for building]]

	== Listing items ==		== Listing items ==

Mikee: moved ++WIP++ Training Running Tests to Training Running Tests

2010-06-01T18:36:41Z

moved ++WIP++ Training Running Tests to Training Running Tests

← Older revision	Revision as of 18:36, 1 June 2010
(No difference)

Mikee at 18:36, 1 June 2010

2010-06-01T18:36:19Z

← Older revision		Revision as of 18:36, 1 June 2010
Line 1:		Line 1:
	== Background ==		== Background ==

	Device connection and STRIDE test execution is handled by the [[STRIDE Runner]] (aka ''"the runner"''). The runner is a command line tool with a number of options for listing runnable items, executing tests, tracing on test points, and uploading your results. The runner is designed for use in both ad-hoc execution of tests and fully automated CI execution~~. The command line interface is sufficiently configurable to lend it to a variety of uses~~. You've already seen some basic test execution scenarios in the other training sections - now we will look explicitly at several of the most common use-cases for the runner.		Device connection and STRIDE test execution is handled by the [[STRIDE Runner]] (aka ''"the runner"''). The runner is a command line tool with a number of options for listing runnable items, executing tests, tracing on test points, and uploading your results. The runner is designed for use in both ad-hoc execution of tests and fully automated CI execution. You've already seen some basic test execution scenarios in the other training sections - now we will look explicitly at several of the most common use-cases for the runner.

	Please review the following reference articles before proceeding:		Please review the following reference articles before proceeding:
Line 42:		Line 42:
	A few things to notice:		A few things to notice:
	* The Functions (if any) are listed before the Test Units.		* The Functions (if any) are listed before the Test Units.
	* Function and Test Unit arguments (input params) will be shown, if any. The parameter types are described shown for each~~, absent any typedefs~~.		* Function and Test Unit arguments (input params) will be shown, if any. The parameter types are described shown for each.

	== Tracing on test points ==		== Tracing on test points ==