Training Running Tests

Background

Device connection and STRIDE test execution is handled by the STRIDE Runner (aka "the runner"). The runner is a command line tool with a number of options for listing runnable items, executing tests, tracing on test points, and uploading your results. The runner is designed for use in both ad-hoc execution of tests and fully automated CI execution. The command line interface is sufficiently configurable to lend it to a variety of uses. You've already seen some basic test execution scenarios in the other training sections - now we will look explicitly at several of the most common use-cases for the runner.

Please review the following reference articles before proceeding:

• STRIDE Runner (reference)
• Running Tests
• Organizing Tests into Suites
• Tracing

Build a test app

Let's begin by building an off-target test app to use for these examples. The sources we want to include in this app are test_in_script/Expectations and test_in_c_cpp/TestClass. Copy these source files to your sample_src directory and follow these instructions for building

List items

Listing the contents of database is an easy way to see what test units and fixturing functions are available for execution. Open a command line and try the following:

stride --database="../out/TestApp.sidb" --list

You should see output something like this:

Functions
  Exp_DoStateChanges()
Test Units
  s2_testclass::Basic::Exceptions()
  s2_testclass::Basic::Fixtures()
  s2_testclass::Basic::Parameterized(char const * szString, unsigned int uExpectedLen)
  s2_testclass::Basic::Simple()
  s2_testclass::RuntimeServices::Dynamic()
  s2_testclass::RuntimeServices::Override()
  s2_testclass::RuntimeServices::Simple()
  s2_testclass::RuntimeServices::VarComment()
  s2_testclass::srTest::Dynamic()
  s2_testclass::srTest::Simple()

A few things to notice:

• The Functions (if any) are listed before the Test Units.
• Function and Test Unit arguments (input params) will be shown, if any. The parameter types are described shown for each, absent any typedefs.

Trace on test points

Tracing using the runner will show any STRIDE Test Points that are generated on the device during the window of time that the runner is connected. If you have test points that are continuously being emitted (for instance, in some background thread), then you can just connect to the device with tracing enabled to see them (you'll need to specify a --trace_timeout parameter to tell the runner how long to trace for). If your test points require some fixturing to be hit, then you'll need to specify a script to execute that makes the necessary fixture calls. This is precisely what we did in our previous Instrumentation training. If you recall from that training, we did the following:

stride --device="TCP:localhost:8000" --database="../out/TestApp.sidb" --run=do_state_changes.pl --trace