Training Running Tests: Difference between revisions
No edit summary |
No edit summary |
||
| Line 16: | Line 16: | ||
== List items == | == List items == | ||
Listing the contents of database is an easy way to see what test units and fixturing functions are available for execution. Open a command line and try the following: | Listing the contents of database is an easy way to see what test units and fixturing functions are available for execution. Open a command line and try the following<ref name="database">These examples assume you are executing the runner from the <tt>src</tt> directory of your off-target framework. If that's not the case, you will need to adjust the database path accordingly.</ref>: | ||
<pre> | <pre> | ||
stride --database="../out/TestApp.sidb" --list | stride --database="../out/TestApp.sidb" --list | ||
</pre> | </pre> | ||
| Line 46: | Line 46: | ||
== Trace on test points == | == Trace on test points == | ||
Tracing using the runner will show any STRIDE Test Points that are generated on the device during the window of time that the runner is connected. If you have test points that are continuously being emitted (for instance, in some background thread), then you can just connect to the device with tracing enabled to see them (you'll need to specify a <tt>--trace_timeout</tt> parameter to tell the runner how long to trace for). If your test points require some fixturing to be hit, then you'll need to specify a script to execute that makes the necessary fixture calls. This is precisely what we did in our previous [[Training_Instrumentation#Build, Run, and Trace|Instrumentation training]]. If you recall from that training, we did the following: | Tracing using the runner will show any STRIDE Test Points that are generated on the device during the window of time that the runner is connected. If you have test points that are continuously being emitted (for instance, in some background thread), then you can just connect to the device with tracing enabled to see them (you'll need to specify a <tt>--trace_timeout</tt> parameter to tell the runner how long to trace for). If your test points require some fixturing to be hit, then you'll need to specify a script to execute that makes the necessary fixture calls. This is precisely what we did in our previous [[Training_Instrumentation#Build, Run, and Trace|Instrumentation training]]. If you recall from that training, we did the following<ref name="database"/>: | ||
<pre> | <pre> | ||
stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run=do_state_changes.pl --trace | stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run=do_state_changes.pl --trace | ||
</pre> | </pre> | ||
== Organizing with suites == | == Organizing with suites == | ||
== Notes == | == Notes == | ||
<references/> | <references/> | ||
Revision as of 23:02, 26 May 2010
Background
Device connection and STRIDE test execution is handled by the STRIDE Runner (aka "the runner"). The runner is a command line tool with a number of options for listing runnable items, executing tests, tracing on test points, and uploading your results. The runner is designed for use in both ad-hoc execution of tests and fully automated CI execution. The command line interface is sufficiently configurable to lend it to a variety of uses. You've already seen some basic test execution scenarios in the other training sections - now we will look explicitly at several of the most common use-cases for the runner.
Please review the following reference articles before proceeding:
Build a test app
Let's begin by building an off-target test app to use for these examples. The sources we want to include in this app are test_in_script/Expectations and test_in_c_cpp/TestClass. Copy these source files to your sample_src directory and follow these instructions for building
List items
Listing the contents of database is an easy way to see what test units and fixturing functions are available for execution. Open a command line and try the following[1]:
stride --database="../out/TestApp.sidb" --list
You should see output something like this:
Functions Exp_DoStateChanges() Test Units s2_testclass::Basic::Exceptions() s2_testclass::Basic::Fixtures() s2_testclass::Basic::Parameterized(char const * szString, unsigned int uExpectedLen) s2_testclass::Basic::Simple() s2_testclass::RuntimeServices::Dynamic() s2_testclass::RuntimeServices::Override() s2_testclass::RuntimeServices::Simple() s2_testclass::RuntimeServices::VarComment() s2_testclass::srTest::Dynamic() s2_testclass::srTest::Simple()
A few things to notice:
- The Functions (if any) are listed before the Test Units.
- Function and Test Unit arguments (input params) will be shown, if any. The parameter types are described shown for each, absent any typedefs.
Trace on test points
Tracing using the runner will show any STRIDE Test Points that are generated on the device during the window of time that the runner is connected. If you have test points that are continuously being emitted (for instance, in some background thread), then you can just connect to the device with tracing enabled to see them (you'll need to specify a --trace_timeout parameter to tell the runner how long to trace for). If your test points require some fixturing to be hit, then you'll need to specify a script to execute that makes the necessary fixture calls. This is precisely what we did in our previous Instrumentation training. If you recall from that training, we did the following[1]:
stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run=do_state_changes.pl --trace