Off-Target Environment: Difference between revisions
| Line 133: | Line 133: | ||
Further--as a pre-compilation step--any .h files found in <tt>sample_src</tt> are submitted to the [[s2scompile|S2 compiler]] and subsequent [[Build Tools]]. This will result in | Further--as a pre-compilation step--any .h files found in <tt>sample_src</tt> are submitted to the [[s2scompile|S2 compiler]] and subsequent [[Build Tools]]. This will result in | ||
* the detection of [[ | * the detection of [[Test Unit Pragmas| SCL Pragmas]] which declare Test Units in these .h files | ||
* the inclusion of metadata into the sidb file to describe these Test Units | * the inclusion of metadata into the sidb file to describe these Test Units | ||
* the generation of test harnessing code to run the indicated Test Units and collect results | * the generation of test harnessing code to run the indicated Test Units and collect results | ||
Revision as of 23:20, 24 August 2009
Overview
Two of the largest barriers to embedded developer productivity are long build/test cycles and scarce target hardware. Fortunately, STRIDE's cross-platform capabilities make it possible to run STRIDE in a host-only "sandbox" that emulates your target system for purposes of evaluation. All of the supplied samples and test code run identically under either the sandbox environment or your actual target. In addition, Any test code you write will also run in either environment.
The sandbox utilizes a "target" application that is built and runs on the host system. The test runner application runs on the same system and communicates with the "target" process over a TCP/IP connection. This set up frees you from external hardware dependencies and provides for rapid build/run cycles.
This article guides you through the steps to run and publish a set of sample tests using the sandbox evaluation environment.
Evaluation Steps
To simplify the evaluation, we recommend using a single Windows or Linux (x86) computer for both the target and host systems. Host and target code will run in separate processes and communicate via TCP/IP, thus simulating an embedded target with host computer configuration. All code and techniques used in the sandbox evaluation are directly transferable to a production environment.
The recommended evaluation steps are as follows:
- Review the STRIDE Overview
- Install required packages
- Activate your STRIDE license
- Configure STRIDE Test Space for publishing results
- Build and test the basic TargetApp with built-in S2 diagnostic tests, publishing results to TestSpace
- Build and test the TargetApp with the TestIntro sample, publishing results to TestSpace
- Create your own test, run and publish
- Explore other sample tests
Installing Packages
The following packages must be installed on the computer you will use for your sandbox environment. (Use Windows or Linux packages as appropriate).
See Package Installation for instructions.
- STRIDE SDK
- Host Tools
- Samples
- Build Tools
TestSpace Configuration
Building the TestApp
This step builds the first version of an instrumented target application. This application links with the STRIDE runtime library and includes the S2 diagnostic tests.
This step requires an installation of the STRIDE build tools. If not installed, please see Package Installation for more information.
Linux
- Build the test app
cd ~/stride/SDK/Linux/src make testapp - The file ~/stride/SDK/Linux/out/bin/TestApp will be produced
- Note also that the STRIDE database file ~/stride/SDK/Linux/out/TestApp.sidb is also produced
Windows
- If using Microsoft Visual Studio, open a Visual Studio command prompt to ensure that the compiler and linker are on your PATH.
- Build the test app using the supplied GNU make. (You will get Makefile errors if you use the default make.)
cd stride\SDK\Windows\src ..\bin\make testapp - The file stride\SDK\Windows\out\bin\TestApp.exe will be produced
- Note that the STRIDE database file stride\SDK\Windows\out\TestApp.sidb is also produced
Running the S2 Diagnostic Tests
In this step, we will use the stride executable to run the S2 diagnostic tests.
<dhflashplayer>file=RunningDiagnostics.mp4|width=640|height=535| path=http://www.s2technologies.com/video/wiki/</dhflashplayer>
This step requires an installation of the STRIDE host tools. If not installed, please see Package Installation for more information.
- Invoke the TestApp. In order to see TestApp's output, we recommend that you manually open a new console (or Windows equivalent):
~/stride/SDK/Linux/out/bin/TestApp
- Note TestApp's output upon startup.
starting up... "_srThread" thread started. "stride" thread started. Listening on TCP port 8000
- From a console window, invoke stride as follows, to verify connectivity with the target app (or Windows equivalent):
Note: if this is the first time stride has been run on this computer, you will be prompted for a license activation key. If you access the Internet via a proxy see the information here.stride --diagnostics --database=~/stride/SDK/Linux/out/TestApp.sidb --device=TCP:localhost:8000
As the tests run you will see output in both the TestApp (target) and stride (host) windows.
The host window output is shown here:
Loading database... Connecting to device... runtime version: 4.1.01 Diagnosing target application... link payload fragmentation runtime configuration stub-proxy deadlock target characteristics --------------------------------------------------------------------- Summary: 52 passed, 0 failed, 0 in progress, 12 not in use. Disconnecting from device... Saving result file...
- Note the Summary results shown in the host output; all in use tests should pass.
- To exit TestApp, give the target window focus and enter ctrl-c (or 'q' under Windows).
Viewing Detailed Test Results
If you now list the files in the directory from which you ran stride, you will see the files TestApp.xml and--if you were connected to the Internet when you ran the tests--TestApp.xsl.
By opening TestApp.xml in a web browser, the xsl is automatically applied to create html in the browser.
Details regarding individual diagnostic tests can be found at this link.
Building the TargetApp to Include Test Intro Sample
In this step, will will add a set of sample tests that provide an overview of STRIDE testing techniques. The Test Intro Sample tests are described in the linked article.
This step requires an installation of the STRIDE Samples package. If not installed, please see Package Installation for more information.
The SDK Makefile is set up so that all .c .cpp and .h files found in the directory SDK/Linux/sample_src (or, alternatively SDK\Windows\sample_src) are included in the compile and link of the testapp target.
Further--as a pre-compilation step--any .h files found in sample_src are submitted to the S2 compiler and subsequent Build Tools. This will result in
- the detection of SCL Pragmas which declare Test Units in these .h files
- the inclusion of metadata into the sidb file to describe these Test Units
- the generation of test harnessing code to run the indicated Test Units and collect results
To begin, be sure that TestApp is not running, then copy the .c and .h files found in Samples/TestIntro to SDK/Linux/sample_src (or Windows equivalent). Then build the TestApp as described above.
Listing Test Units in the Database File
In this step, we will use stride to list the Test Units in the TestApp.sidb database.
Run stride as follows (or Windows equivalent):
stride --list --database=~/stride/SDK/Linux/out/TestApp.sidb
You should see output like this:
Loading database... s2_testintro_cclass s2_testintro_flist s2_testintro_testdoubles s2_testintro_testpoints
Note that the S2 diagnostic tests (which are present in this TestApp) are not listed.
Running Test Intro Samples
Running All Tests
Here we will run all tests in the TestApp.sidb database.[1]
- Invoke TestApp in a target console as in earlier steps.
- Invoke stride as shown below and verify Summary results.
stride --database=~/stride/SDK/Linux/out/TestApp.sidb --device=TCP:localhost:8000
Loading database...
Connecting to device...
runtime version: 4.1.01
Executing test units...
s2_testintro_cclass
> 1 passed, 1 failed, 0 in progress, 0 not in use.
s2_testintro_flist
> 2 passed, 1 failed, 0 in progress, 0 not in use.
s2_testintro_testdoubles
> 3 passed, 0 failed, 0 in progress, 0 not in use.
s2_testintro_testpoints
> 3 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 9 passed, 2 failed, 0 in progress, 0 not in use.
Disconnecting from device...
Saving result file...
Running a Subset of Tests
Here we will run only the Test Units named s2_testintro_cclass and s2_testintro_flist against TestApp.
stride -r "s2_testintro_cclass, s2_testintro_flist" --database=~/stride/SDK/Linux/out/TestApp.sidb --device=TCP:localhost:8000
Loading database...
Connecting to device...
runtime version: 4.1.01
Executing test units...
s2_testintro_cclass
> 1 passed, 1 failed, 0 in progress, 0 not in use.
s2_testintro_flist
> 2 passed, 1 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 3 passed, 2 failed, 0 in progress, 0 not in use.
Disconnecting from device...
Saving result file...
Additional Samples
Additional samples are included in the STRIDE Samples package. (See Package Installation for installation information.)
These samples explore each of the techniques outlined in the Test Intro sample in greater depth. See Test Unit Samples for details.
To include any of the samples into your TestApp, just copy the sources into the sample_src and rebuild. To execute the tests, use the stride test runner as before.
Related Links
Notes
- ↑ Note that the S2 diagnostic tests are treated separately, and are not run unless the --diagnostics option is specified to stride.