STRIDE Wiki - User contributions [en]

Main Page

2015-07-09T13:54:39Z

Marku:

Stride™ Wiki 
__NOTOC__
Stride is a test framework for writing C/C++ test cases executing on a device. For an overview of Stride [[Stride Overview| '''click here''']].

== [[Test Implementation]] ==
* [[Test Unit]]
* [[Test Pragmas]]
* [[Test Macros]]
* [[Notes]]
* [[Parameterized_Test_Units | Parameters]]
* [[File Transfer Services | Remoting Files]]
* [[Test_Documentation_in_C/C%2B%2B | Documenting Tests]]
* [[Running Tests]]
* [[Runtime Test Services | Test Services]]

== [[Test Doubles]] ==
* [[Test Double]]
* [[Scl function | Function Pragma]]
* [[Name Mangling]]

== [[Test Points]] ==
* [[Test Point]]
* [[Expectations | Expectations]]
* [[Test_Point_Testing_in_C/C%2B%2B | Writing Tests]]
* [[Test Log | Test Logs]]

== [[Testing with Perl]] ==
* [[Test Script]]
* [[Perl Script APIs | Test API]]
* [[Perl Script Snippets | Examples]]

== [[Installation]] ==

* [[Framework Setup]]
* [[Windows SDK]]
* [[Posix SDK]]
* [[Runtime Integration]]
* [[Build Integration]]
* [[Sample Stride Target Settings | Build Target settings examples]]
* [[STRIDE Extensions for Visual Studio | Build with Visual Studio]]

== [[Training]] ==
* [[Stride Sandbox]]
* [[Training Overview]]
* [[Training Basics]]
* [[Training Advanced]]
* [[Training Perl]]

== [[Reference]] ==
* [[STRIDE Runner| Runner]]
* [[STRIDE Build Tools | Build Tools]]
* [[STRIDE Runtime | Runtime]]
* [[Platform Abstraction Layer | Platform Abstraction Layer (PAL)]]
* [[Intercept Module | Intercept Module (IM)]]
* [[Reporting Model]]
* [[Pragmas]]
* [[Release Notes]]

Main Page(old)

2015-07-09T13:53:45Z

Marku: Created page with " Welcome to the STRIDE™ Wiki STRIDE™ has been designed specifically for '''On-Target White-box Testing'''. S..."

Welcome to the STRIDE™ Wiki 

STRIDE™ has been designed specifically for '''On-Target White-box Testing'''. STRIDE™ is '''test infrastructure''' used by engineers to implement and execute [[Types_of_Testing_Supported_by_STRIDE | '''tests''']] for validating embedded software executing On-Target. The STRIDE™ system consists of a [[STRIDE Overview | '''framework''']] for testing and a [[STRIDE_Test_Space | '''hosted web application''']] for storing and analyzing test results.

<hr/>
'''For an overview of STRIDE™ [[STRIDE Overview| PLEASE CLICK HERE]]'''.

[[Image:screen_icon.png]] [[STRIDE Overview Video | STRIDE Overview Screencast]]
<hr/>

{| class="FCK__ShowTableBorders"
|-
| valign="top" |
{| class="FCK__ShowTableBorders" style="border-right: rgb(187,204,204) 1px solid; border-top: rgb(187,204,204) 1px solid; vertical-align: top; border-left: rgb(187,204,204) 1px solid; border-bottom: rgb(187,204,204) 1px solid" cellspacing="5" cellpadding="2"
|-
! style="border-right: rgb(163,176,191) 1px solid; padding-right: 0.4em; border-top: rgb(163,176,191) 1px solid; padding-left: 0.4em; font-weight: bold; font-size: 120%; background: rgb(206,223,242) 0% 50%; padding-bottom: 0.2em; margin: 0pt; border-left: rgb(163,176,191) 1px solid; color: rgb(0,0,0); padding-top: 0.2em; border-bottom: rgb(163,176,191) 1px solid; text-align: left; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial" | [[:Category:Source Instrumentation | Instrumentation]]

! style="border-right: rgb(163,176,191) 1px solid; padding-right: 0.4em; border-top: rgb(163,176,191) 1px solid; padding-left: 0.4em; font-weight: bold; font-size: 120%; background: rgb(206,223,242) 0% 50%; padding-bottom: 0.2em; margin: 0pt; border-left: rgb(163,176,191) 1px solid; color: rgb(0,0,0); padding-top: 0.2em; border-bottom: rgb(163,176,191) 1px solid; text-align: left; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial" | [[:Category:Tests in Script| Tests in Script]]

! style="border-right: rgb(163,176,191) 1px solid; padding-right: 0.4em; border-top: rgb(163,176,191) 1px solid; padding-left: 0.4em; font-weight: bold; font-size: 120%; background: rgb(206,223,242) 0% 50%; padding-bottom: 0.2em; margin: 0pt; border-left: rgb(163,176,191) 1px solid; color: rgb(0,0,0); padding-top: 0.2em; border-bottom: rgb(163,176,191) 1px solid; text-align: left; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial" | [[:Category:Test Units| Tests in C/C++]]

! style="border-right: rgb(163,176,191) 1px solid; padding-right: 0.4em; border-top: rgb(163,176,191) 1px solid; padding-left: 0.4em; font-weight: bold; font-size: 120%; background: rgb(206,223,242) 0% 50%; padding-bottom: 0.2em; margin: 0pt; border-left: rgb(163,176,191) 1px solid; color: rgb(0,0,0); padding-top: 0.2em; border-bottom: rgb(163,176,191) 1px solid; text-align: left; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial" | [[:Category:Running Tests | Running Tests]]

|-


| valign="top" |
* [[Source Instrumentation Overview | Overview]]
* [[Test Point (old) | Test Points]]
* [[Test Log | Test Logs]]
* [[Function_Capturing | Functions]]
* [[Expectations | Expectations]]


| valign="top" |
* [[Test Modules Overview | Overview]]
* [[Perl Script APIs | Perl Script APIs]]
* [[Perl Script Snippets]]
* [[Script_Samples | Samples]]


| valign="top" |
* [[Test Units Overview | Overview]]
* [[Test Units]]
* [[Test Macros | Test Macros]]
* [[Test API | Test APIs]]
* [[C/C%2B%2B_Samples | Samples]]


| valign="top" |
* [[Running Tests (old) | Running Tests]]
* [[Listing Functions and Test Units|Listing Tests]]
* [[Tracing | Tracing]]
* [[Organizing Tests into Suites | Organizing Tests]]
* [[Setting up your CI Environment | Setting up CI]]

|-
! style="border-right: rgb(163,176,191) 1px solid; padding-right: 0.4em; border-top: rgb(163,176,191) 1px solid; padding-left: 0.4em; font-weight: bold; font-size: 120%; background: rgb(206,223,242) 0% 50%; padding-bottom: 0.2em; margin: 0pt; border-left: rgb(163,176,191) 1px solid; color: rgb(0,0,0); padding-top: 0.2em; border-bottom: rgb(163,176,191) 1px solid; text-align: left; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial" | [[:Category: Test Space|Test Space]]

! style="border-right: rgb(163,176,191) 1px solid; padding-right: 0.4em; border-top: rgb(163,176,191) 1px solid; padding-left: 0.4em; font-weight: bold; font-size: 120%; background: rgb(206,223,242) 0% 50%; padding-bottom: 0.2em; margin: 0pt; border-left: rgb(163,176,191) 1px solid; color: rgb(0,0,0); padding-top: 0.2em; border-bottom: rgb(163,176,191) 1px solid; text-align: left; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial" | [[:Category:Reference | Reference]]

! style="border-right: rgb(163,176,191) 1px solid; padding-right: 0.4em; border-top: rgb(163,176,191) 1px solid; padding-left: 0.4em; font-weight: bold; font-size: 120%; background: rgb(206,223,242) 0% 50%; padding-bottom: 0.2em; margin: 0pt; border-left: rgb(163,176,191) 1px solid; color: rgb(0,0,0); padding-top: 0.2em; border-bottom: rgb(163,176,191) 1px solid; text-align: left; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial" | [[:Category:Installation | Installation]]

! style="border-right: rgb(163,176,191) 1px solid; padding-right: 0.4em; border-top: rgb(163,176,191) 1px solid; padding-left: 0.4em; font-weight: bold; font-size: 120%; background: rgb(206,223,242) 0% 50%; padding-bottom: 0.2em; margin: 0pt; border-left: rgb(163,176,191) 1px solid; color: rgb(0,0,0); padding-top: 0.2em; border-bottom: rgb(163,176,191) 1px solid; text-align: left; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial" | [[:Category:Training | Training]]

|-




| valign="top" |
* [[STRIDE_Test_Space|Overview]]
* [[User Administration]]
* [[Creating Test Spaces]]
* [[Uploading Test Results]]
* [[Notifications]]
* [[Reporting Entities]]
* [[Creating And Using Baselines | Using Baselines]]


| valign="top" |
* [[STRIDE Runner]]
* [[STRIDE Build Tools]]
* [[STRIDE Runtime]]
* [[STRIDE Off-Target Environment]]
* [[Platform Abstraction Layer]]
* [[Intercept Module]]
* [[Test Unit Pragmas]]
* [[Reporting Model]]


| valign="top" |
* [[Installation Overview | Overview]]
* [[Desktop Installation | Framework setup]]
* [[Runtime Integration]]
* [[Build Integration]]
* [[Stride Sandbox]]


| valign="top" |
* [[Training Getting Started | Getting Started]]
* [[Training Test Macros | Test Macros]]
* [[Training File IO | File IO]]
* [[Training Parameters | Parameters]]
* [[Training Fixturing | Fixturing]]
* [[Training Expectations | Test Points]]
* [[Training Doubling | Test Doubles]]
|}

Click here for [[:Category:Release Notes| Release Notes]]

Reference

2015-07-07T16:46:13Z

Marku:

* [[STRIDE Runner| Runner]]
* [[STRIDE Build Tools | Build Tools]]
* [[STRIDE Runtime | Runtime]]
* [[Platform Abstraction Layer | Platform Abstraction Layer (PAL)]]
* [[Intercept Module | Intercept Module (IM)]]
* [[Reporting Model]]
* [[Pragmas]]
* [[Release Notes]]

Release Notes

2015-07-07T16:45:20Z

Marku: Created page with "__NOTOC__ This page documents the changes in STRIDE releases. = 5.0.02x = The code name for this release is ''South Side''. Released on Apr 21, 2015. == Testspace integrati..."

__NOTOC__
This page documents the changes in STRIDE releases.

= 5.0.02x =
The code name for this release is ''South Side''. Released on Apr 21, 2015.

== Testspace integration ==
Stride Runner integration with [http://help.testspace.com Testspace] has been highly improved.

=== Fixes ===
''This section describes defects which have been corrected in STRIDE and the customer tracking number associated with them, if any, in brackets [].''

* [[s2scompile|STRIDE Compiler]] doesn't support Microsoft preprocessor specific "token-pasting".
* [[STRIDE Runner]] fails on execute test modules with Perl 5.18.

=== Tests in Scripts ===
* Added support for Perl version 5.20.
* Support for Perl versions 5.14 and older is not provided by default but is available on demand.

=== Runtime ===
* STRIDE Host Release '''5.0.02x''' is compatible with the Runtime Version '''5.0.02'''
* ''All of the Runtime files have been modified. It is recommended that you update them all.''
* ''All [[Posix SDK]] files have been modified.''
* ''All [[Windows SDK]] files have been modified.''

== Minor and Patch releases ==

=== 5.0.02a ===
''Defects corrected and the customer tracking number associated with them, if any, in brackets []:''
* [[STRIDE Runner]] local generated result file incorrectly renders links in non-IE browsers.
* [[STRIDE Runner]] incorrectly merges result files with BOM.
* [[STRIDE Runner]] may error out in environments with multi-line variable values.
* [[STRIDE Runner]] may not upload results to Testspace in case of HTTP redirect.
* [[STRIDE Runner]] may crash on results upload in case the Testspace server returns an error.
* [[STRIDE Runner]] is not able to access Testspace Project/Space with white space in the name.
* [[STRIDE Runner]] may silently fail or return incorrect error code.
* [[STRIDE Runner]] times out on target connection when using a custom transport.

''The following [[Posix SDK]] files have been modified:''
palIO.c
palOS.c
stride.c

''The following [[Windows SDK]] files have been modified:''
palIO.c

= 5.0.01x =

The code name of this release is''Warm Waters''. Released on Jun 3, 2014.

== Testspace Schema execution ==
Now the [[STRIDE Runner]] allows execution of tests defined in a [http://help.testspace.com Testspace] schema.

=== Input Parameters ===
Passing input parameters per test suite has been implemented. A new Test Class [[ Runtime_Test_Services#method_GetParam | method ]] and C [[ Runtime_Test_Services#srTestGetParam| function]] in the [[STRIDE Runtime]] allow accessing them by name.

=== Interactive Mode ===
The [[File_Transfer_Services | File Transfer]] subsystem has been updated to allow remote (host) interactive prompt and process execution. In addition the [[STRIDE Runner]] now supports single test case execution.

== Change Details ==

=== Fixes ===
''This section describes defects which have been corrected in STRIDE and the customer tracking number associated with them, if any, in brackets [].''

* Handling of input payload in [[STRIDE Runner]] times slower compared to output.
* [[STRIDE Runner]] generates report with incorrect timestamps for kernel [[Test Log | Test Logs]].
* [[STRIDE Runner]] may generate a report xml with invalid file annotation content.

=== Tests in Scripts ===
* Dynamic Test Suites are not supported no more.

=== Tests in C/C++ ===
* Dynamic Test Suites are not supported no more.
* [[Test_Macros#Assertions|Test Assertion macro]] now by default (controlled via compiler define) throw exceptions.
* [[Test_Macros#Notes|Test Annotation macros]] and [[Test_Log|Test Log macros]] now by default (controlled via compiler define) support variadic arguments.
* Error handling has been improved.

=== Runtime ===
* STRIDE Host Release '''5.0.01x''' is compatible with the Runtime Version '''5.0.01'''
* SLAP has been move in the core Runtime.
* <tt>srCOMPLEX_TARGET</tt> is set by default now.
* ''All of the Runtime files have been modified. It is recommended that you update them all.''
* ''All [[Posix SDK]] files have been modified.''
* ''All [[Windows SDK]] files have been modified.''

== Minor and Patch releases ==

=== 5.0.01a ===
''New features and improvements:''
* [[STRIDE Runner]] now can run executables (optionally in the background) and consume directly JUnit xml report.
* [[STRIDE Runner]] now reports overall suite duration.
* [[STRIDE Runner]] has been updated to handle common suite input defined in folder settings.
* [[STRIDE Runner]] now has improved error handling and supports "dry" run (for fast input validation).
''Defects corrected and the customer tracking number associated with them, if any, in brackets []:''
* Incremental "finish" upload via [[STRIDE Runner]] does not complete the result set.

=== 5.0.01b ===
''Defects corrected and the customer tracking number associated with them, if any, in brackets []:''
* [[s2sinstrument|STRIDE Instrumentation Generator]] produces incorrect code for function callbacks in a 64-bit build.
* [[Perl_Script_APIs#Assertions|Perl Assertions]] error out with undefined input.
* [[STRIDE Runner]] does not honor timeout for executables and scripts.
* [[STRIDE Runner]] errors out in "dry" run with no database.
* [[STRIDE Extensions for Visual Studio]] error out when solution is opened from a UNC location.

''The following Runtime files have been modified:''
srcgutil.h

''The following [[Windows SDK]] files have been modified:''
stride.props
stride.targets

=== 5.0.01c ===
''New features and improvements:''
* [[STRIDE Runner]] traversal of a [http://help.testspace.com Testspace] schema has been optimized.
* [[STRIDE Runner]] now shows the path to local report file and the URL to the uploaded Testspace results.

''Defects corrected and the customer tracking number associated with them, if any, in brackets []:''
* [[s2scompile|STRIDE Compiler]] is unable to compile header files with forwardly declared enums.
* [[s2scompile|STRIDE Compiler]] calculates incorrect payload offset for user defined type arguments in a test class constructor.
* [[s2scompile|STRIDE Compiler]] errors out on enums with out of range values.
* [[STRIDE Runner]] is unable to explicitly run "disabled" folder/suite.
* [[STRIDE Runner]] creates empty xsl file on host with no internet connection.
* Static analysis of [[Windows SDK]] source results in a warning.
* [[STRIDE Runtime]] memory pool allocation returns misaligned pointers on 64-bit targets.

''The following Runtime files have been modified:''
srcfg.h
srtest.h
srmem.c

''The following [[Windows SDK]] files have been modified:''
stride.c

=== 5.0.01d ===
''Defects corrected and the customer tracking number associated with them, if any, in brackets []:''
* [[Windows SDK]] may deadlock on timer stop.

''The following [[Windows SDK]] files have been modified:''
palOS.c

=== 5.0.01e ===
''New features and improvements:''
* [[STRIDE Runner]] now generates a result file that can directly open in any major browser.
* [[STRIDE Runner]] now can upload to a sub-folder in [http://help.testspace.com/ Testspace].
* A new set of [[STRIDE Runner]] Testspace folder settings is defined. Some older settings are deprecated.
* New [[Training]] samples are provided.
''Defects corrected and the customer tracking number associated with them, if any, in brackets []:''
* [[Test_Point_Testing_in_C/C%2B%2B#srTestPointCheck|srTestPointCheck]] with more expected than actual may never return
* A <code>[section]</code> in Testspace schema "root" settings hides [[STRIDE Runner]] command line options.
* [[STRIDE Runner]] is unable to init/connect when running a Testspace schema sub-folder/suite
* [[STRIDE Runner]] mangles non-STRIDE test suite input

''The following Runtime files have been modified:''
srtest.c

Stride Sandbox

2015-07-07T16:31:43Z

Marku: /* SDK Makefile */

__NOTOC__
Stride's cross-platform capabilities make it possible to use Stride in a '''host-only''' configuration called the '''Sandbox'''. This environment facilitates ''self-training'', ''evaluations'', and ''trying stuff''. It frees you from external hardware dependencies and provides for a rapid ''edit-build-test'' cycle.

The '''Sandbox''' utilizes the framework's SDK that can be built and executed on the host system. When using the SDK Makefile a simulated target ''native application'' is generated, which we call a '''Test Application (TestApp)'''. The Stride Runner application executes on the same host and communicates with the '''TestApp''' process over a TCP/IP connection.

The '''Sandbox''' requires the Stride framework package to be setup on your desktop. Refer to the [[Framework Setup | Framework Setup]] article for more information. It also requires that your desktop contains one of the following '''compilers''':
* For Windows, Microsoft Visual Studio 2008 or later is required. If you don't already have Visual Studio, the free [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio_Express Visual C++ Express] can be used (download [http://www.microsoft.com/express/download/#webInstall here]). In case you have [http://www.cygwin.com Cygwin] installed, the [http://en.wikipedia.org/wiki/GNU_Compiler_Collection GNU Compiler Collection] could be used as an alternative.
* For Linux and FreeBSD, the [http://en.wikipedia.org/wiki/GNU_Compiler_Collection GNU Compiler Collection] (included by default in almost all distros) is required.

== Building ==

=== SDK Makefile ===
The SDK Makefile is set up by ''default'' so that all <tt>.c</tt> <tt>.cpp</tt> and <tt>.h</tt> files found in the directory <tt>SDK\Windows\sample_src</tt> (or <tt>SDK/Posix/sample_src</tt> for Linux/FreeBSD) are included in the compile and link of the '''testapp''' target.

Further--as a pre-compilation step--any <tt>.h</tt> files found in <tt>sample_src</tt> are submitted to the [[STRIDE Build Tools]]. This will result in
* the detection of [[Test Pragmas| test pragmas]] used to declare Test Suites in these <tt>.h</tt> files
* the generation of a database (<tt>.sidb</tt>) file required for executing tests
* the generation of an [[Intercept Module]] required for executing tests

=== Build Steps ===
To begin, be sure that TestApp is not running then perform the following steps:

====Linux/FreeBSD====
<ol>
<li>Build the test app using GNU make
<source lang="bash">
make -C "$STRIDE_DIR/SDK/Posix/src" testapp
</source>
</li>
<li>Note that the following artifacts are produced by the build:

;<tt>$STRIDE_DIR/SDK/Posix/out/bin/TestApp</tt>
: the test application
;<tt>$STRIDE_DIR/SDK/Posix/out/TestApp.sidb</tt>
: the STRIDE interface database file which contains metadata describing the interfaces remoted by the test app (along with other data)
</li>
</ol>

====Windows====
NOTE: ''In case you have [http://www.cygwin.com Cygwin] and [http://en.wikipedia.org/wiki/GNU_Compiler_Collection GNU Compiler Collection] installed and prefer to use it, please follow the build steps for Linux (see previous section) and ignore the one in here.''
<ol>
<li>If using Microsoft Visual Studio, open a [http://msdn.microsoft.com/en-us/library/ms235639(v=vs.100).aspx Visual Studio Command Prompt] to ensure that the compiler and linker are on your PATH.
</li>
<li>Build the test app using the supplied GNU make. (You will get Makefile errors if you use the default make.)
<source lang="dos">
"%STRIDE_DIR%\SDK\Windows\bin\make" -C "%STRIDE_DIR%\SDK\Windows\src" testapp
</source>
<li>Note that the following artifacts are produced by the build:

;<tt>%STRIDE_DIR%\SDK\Windows\out\bin\TestApp.exe</tt>
: the test application
;<tt>%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb</tt>
: the STRIDE interface database file which contains metadata describing the interfaces remoted by the test app (along with other data)
</li>
</ol>

== Running ==
The test app we just built does not have any user tests in it. At this point it provides a starting point for test that we will subsequently add.

However, a set of diagnostic tests that verify operation of the STRIDE runtime itself are always built into the generated TestApp executable. If desired (we recommend you to do so) you could run them by doing the following:

<ol>
<li>Invoke the TestApp. In order to see TestApp's output, we recommend that you manually run in a console window (or Windows equivalent):
;Linux/FreeBSD
<source lang="bash">
$STRIDE_DIR/SDK/Posix/out/bin/TestApp
</source>
;Windows
<source lang="dos">
%STRIDE_DIR%\SDK\Windows\out\bin\TestApp
</source>
(...or launch from the file explorer)

<li> Note TestApp's output upon startup.
<pre>
--------------------------------------------------
STRIDE Test Console Application.
Enter 'Ctrl+C' to Quit.
--------------------------------------------------
Listening on TCP port 8000
starting up...
"_srThread" thread started.
"stride" thread started.
</pre>
</li>
<li>From a second console window, invoke <tt>[[STRIDE_Runner|stride]]</tt> as follows, to verify connectivity with the test app and STRIDE runtime operation:
 
;Linux/FreeBSD
<source lang="bash">
stride --diagnostics --database="$STRIDE_DIR/SDK/Posix/out/TestApp.sidb" --device=TCP:localhost:8000 --run="*"
</source>
;Windows
<source lang="dos">
stride --diagnostics --database="%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb" --device=TCP:localhost:8000 --run="*"
</source>

As the tests run you will see output in both the TestApp (target) and stride (host) console windows.

The host console window output is shown here:
<pre>
Executing diagnostics...
Connecting to device (TCP:localhost:8000)...
runtime version: 5.0.xy
test suite "/Link"
Loopback ..........
Payload Fragmentation
Stub-Proxy Deadlock
Target Characteristics
> 4 passed, 0 failed, 0 in progress, 0 unknown, 0 not in use, 777.77 ms.
test suite "/Stat"
> 2 passed, 0 failed, 0 in progress, 0 unknown, 0 not in use, 74.98 ms.
test suite "/Time"
> 2 passed, 0 failed, 0 in progress, 0 unknown, 0 not in use, 2559.23 ms.
Disconnecting from device...
---------------------------------------------------------------------
Summary: 8 passed, 0 failed, 0 in progress, 0 unknown, 0 not in use, 3411.98 ms.
</pre>
</li>
<li>Note the Summary results shown in the host output; all in use tests should pass.
</li>
<li>To exit TestApp, give the target window focus and enter <tt>Ctrl-C</tt>.
</ol>

==Build Problems==
This page describes several common problems encountered when building a STRIDE TestApp using the Sandbox and suggested solutions.

=== Make Error 1 ===
;Symptom
On Windows, when attempting to build the testapp from the command line, you encounter an error indicating that:
<pre>
‘cl’ is not recognized as an internal or external command,
operable program or batch file.
</pre>

For example:
<pre>
C:\STRIDE\SDK\Windows\src>..\bin\make.exe testapp
cl -c -nologo -W4 -D_UNICODE -DUNICODE -DWIN32 -D_CONSOLE -DUNDER_NT -I”.” -I”../../Runtime” -I”../../SLAP” -I”../../GRS” -I”../o
ut/src” -I”../sample_src” -GS -Zi -DNDEBUG -MD -O2 -D_LIB -DSTRIDE_STATIC -Fd”../out/desktop-Windows_NT-obj//cl.pdb” -Fo”../out/desktop-Windows_NT-o
bj/srapi.o” ”../../Runtime/srapi.c”
‘cl’ is not recognized as an internal or external command,
operable program or batch file.
make: *** [../out/desktop-Windows_NT-obj/srapi.o] Error 1
</pre>

;Cause
Microsoft Visual Studio 2008 or later is not installed or you are not building from a [http://msdn.microsoft.com/en-us/library/ms235639(v=VS.90).aspx Visual Studio Command Prompt].

;Solution
Make sure you have Microsoft Visual Studio 2008 or later installed.

To ensures that the compiler and linker are on your PATH open a Visual Studio Command prompt:
* Click the Start button, point to All Programs, Microsoft Visual Studio 20XX, Visual Studio Tools, and then click Visual Studio 20XX Command Prompt.

=== Make Error 2 ===
;Symptom
On Windows, when attempting to build the testapp from the command line the following errors are observed:

<pre>
syntax error near unexpected token `('
syntax error near unexpected token `('
syntax error: unexpected end of file
</pre>

For example:
<pre>
C:\stride\SDK\Windows\src>..\bin\make testapp
/bin/sh: -c: line 0: syntax error near unexpected token `('
/bin/sh: -c: line 0: `IF EXIST ../out. (IF NOT EXIST ../out/src mkdir "../out/src") ELSE mkdir "../out" && mkdir "../out/src".'
/bin/sh: -c: line 0: syntax error near unexpected token `('
/bin/sh: -c: line 0: `IF EXIST ../out. (IF NOT EXIST ../out/src mkdir "../out/src") ELSE mkdir "../out" && mkdir "../out/src".'
/bin/sh: -c: line 1: syntax error: unexpected end of file
make: *** [cleanapp] Error 258
</pre>

;Cause
This error occurs because gnu make on Windows will search for an Unix shell (<tt>sh</tt>, <tt>bash</tt> or <tt>csh</tt>) anywhere in your PATH when executing shell commands and only default to DOS shell (<tt>cmd.exe</tt>) when no Unix shell is found. The sandbox Makefile uses DOS shell syntax, so when a Unix shell is found on your PATH, this results to errors like above.

Most commonly, this problem is caused by an installation of [http://en.wikipedia.org/wiki/Cygwin Cygwin], though it can also be caused by an installation of the QNX Software Development Platform.

;Solution

Explicitly specify the DOS shell by invoking make like:
<pre>
..\bin\make SHELL=%ComSpec% testapp
</pre>

'''''Note:''' the value of %ComSpec% should be <tt>C:\Windows\system32\cmd.exe</tt>''

An alternative is to remove any directories from your PATH that contain an Unix shell executable or otherwise prevent such from being found. (e.g. rename its parent directory).

=== Make Error 3 ===
;Symptom
On Linux, when attempting to build the testapp from the command line, the following error is observed:
<pre>
g++: command not found
</pre>

For example:
<pre>
# make testapp
g++ -c -I”.” -I”../../Runtime” -I”../../SLAP” -I”../../GRS” -I”../out/src” -I”../sample_src” -fPIC -D_DEBUG -O0 -g3 -Wall -o ”../out/i386-Linux-obj/srtestpp.obj” ”../../Runtime/srtestpp.cpp”
/bin/sh: g++: command not found
make: *** [../out/i386-Linux-obj/srtestpp.obj] Error 127
</pre>

;Cause
The C++ compiler, <tt>g++</tt> can't be found on your PATH.

Most commonly, this problem is caused by not having a complete installation of [http://en.wikipedia.org/wiki/GNU_Compiler_Collection GNU Compiler Collection].

;Solution
Make sure you have a complete GNU Compiler Collection installed.

=== Make Error 4 ===
;Symptom
When building the testapp from the command line, you see the following compiler errors:

<pre>
../out/src/strideIM.cpp(50) : error C3861: '_srTestResultCountReset': identifier not found
../out/src/strideIM.cpp(54) : error C3861: '_srTestSendFinalStatus': identifier not found
../out/src/strideIM.cpp(56) : error C3861: '_srTestAddToTotal': identifier not found
../out/src/strideIM.cpp(56) : error C3861: '_srTestResultGetTotals': identifier not found
...
...
...
</pre>

;Cause
You are using an outdated version of the STRIDE build tools.

You can see which version of the tools were used to generate your STRIDE sources by looking at the top of the strideIM.cpp source file. The comment block at the top of the file shows this version.

;Solution
Remove the old tools and/or change your path so that the current tools are used.

Runtime Integration

2015-07-07T16:28:18Z

Marku: /* Including the STRIDE Runtime */

==Overview==

This page discusses the first step in an overall STRIDE framework integration. The ultimate goals are to extend your target build process to:
* Incorporate the STRIDE Runtime source files in your target build
* Configure compiler settings
* Add code to target source to start and stop STRIDE runtime threads
* Incorporate the STRIDE Build Tools in your make process as a pre-compilation step

Here we incorporate the [[STRIDE Runtime]] in your target application build without tests. In the second and last step, we add the STRIDE [[Build Tools]] to the mix and hook up the test-running harnessing.

After building the target with the STRIDE Runtime in place, we will run stride from a remote host computer to verify connectivity and runtime operation.

=== Recommendations ===
The STRIDE target environment offers lots of flexibility to accommodate different target scenarios. For an initial target integration, we recommend that you pursue the simplest STRIDE configuration that will yield results. Once this configuration is successfully integrated you can make adjustments as desired.

=== Integrating STRIDE Into Your Target Build ===
To add the STRIDE Runtime to your build process, the following changes must be made:

* Secure a PAL targeted to your target operating system
* Configure the PAL I/O parameters to conform the your target hardware
* Include the STRIDE runtime source files in the software build
* Edit your target application's source code to start the runtime and I/O thread(s)

== Securing a PAL ==
The [[Platform_Abstraction_Layer|Platform Abstraction Layer (PAL)]] provides the glue between the STRIDE Runtime and services offered by your OS. It is the only piece of the runtime that is customized between operating systems.

Fully tested PALs are provided for Posix and Windows, included as <tt>palcfg.h</tt>, <tt>palIO.c</tt> and <tt>palOS.c</tt> in their respective [[Framework_Setup#SDK|SDK]].

== Configuring PAL I/O parameters ==
If your target will communicate with the host computer via the default of TCP/IP port 8000, you can skip this step.

Otherwise, refer to <tt>palcfg.h</tt> that comes with your PAL for a list of configuration parameters.

== Including the STRIDE Runtime ==
The [[STRIDE Runtime]] is distributed in source form. The source files are included with each [[Framework_Setup#SDK|SDK]] under "Runtime" directory. Files in that directories should be included in your target build.

If you are using an S2-supplied SDK, additional required sources can be found in the SDK's <tt>src</tt> directory:

* PAL implementation
** <tt>palcfg.h</tt>
** <tt>palIO.c</tt>
** <tt>palOS.c</tt>

* Convenience functions (these wrap low-level PAL/Runtime calls)
** <tt>stride.c</tt>
** <tt>stride.h</tt>

* Build script
** <tt>Makefile</tt>

The build script (Makefile) supplied with an SDK allows building of the runtime source into a library that could then be linked with the target application.

=== C/C++ Feature Support ===
By default the C source in the Runtime was written with the presumption that the target C compiler and C Runtime (CRT) library supports '''wchar_t''', '''long long''', '''long double''', '''safe snprintf''', and '''variadic macros'''

By default the C++ source in the Runtime was written with the presumption that the target C++ compiler can handle '''namespaces''' and '''exceptions'''.

If desired the user can specify different configuration by using '''-D''' compiler options:

-DsrCRT_HAS_WCHAR_T=<0|1>
-DsrCRT_HAS_LONG_LONG=<0|1>
-DsrCRT_HAS_LONG_DBL=<0|1>
-DsrCRT_HAS_SNPRINTF=<0|1>
-DsrCRT_HAS_VA_ARGS=<0|1>
-DsrCPP_HAS_NAMESPACE=<0|1>
-DsrCPP_HAS_EXCEPTION=<0|1>

where 0=disabled and 1=enabled.

=== STRIDE Feature Control ===
Any application's source file referencing STRIDE source should be compiled with addition ''STRIDE_ENABLED'' switch (preprocessor directive):
<source lang="bash">
cc -DSTRIDE_ENABLED=1 file.c
</source>

The definition of the <tt>STRIDE_ENABLED</tt> macro controls all STRIDE-related sources and macros. If undefined (or defined as zero), STRIDE-related sources are excluded and STRIDE-related macros are left undefined. This provides a simple means of including and excluding test-related entities in your production build.

== Target Source Changes ==

Your target source must be edited to include code that initializes and uninitializes the runtime. The code is typically added to your <tt>main()</tt> function, but the only requirement is that the initialization takes place before the host computer connects and the uninitialization occurs before the process terminates.

Please refer to the standard pre-packaged [[Framework_Setup#SDK|SDK]] for examples of a startup sequence. Briefly it should be something like:

<source lang="c">
/* STRIDE runtime includes */
#include <stride.h>

int main(int argc, char **argv)
{
...

/* initialize the STRIDE subsytem using default I/O */
#ifdef __cplusplus
strideIO_t io = {strideIO_t::strideDEFAULT};
#else
strideIO_t io = {strideDEFAULT};
#endif
if (strideInit(&io) != srTRUE)
return -1;

/* start any IM thread */
...

/* target application code here */
...

/* stop all STRIDE threads and uninitialize STRIDE subsystem */
strideUninit();

return 0;
}
</source>

If you implement a custom PAL then here is an example of the startup code using direct PAL and STRIDE Runtime API calls instead of the stride.c/.h wrappers provided by the SDK:

<source lang="c">
/* STRIDE runtime includes */
#include <palcfg.h>
#include <sr.h>

int main(int argc, char **argv)
{
palDWORD dwRTNID;

...

/* initialize the PAL handler */
if (palOSInit() != palTRUE)
return -1;

/* initialize the PAL IO handler */
if (palIOInit(PAL_DEFAULT_DEVICE_NAME) != palTRUE)
{
palOSUninit();
return -1;
}

/* initialize STRIDE Runtime */
if (srInit() != srOK)
{
palIOShutdown();
srUninit();
palOSUninit();
return -1;
}

/* start the Runtime thread first */
palDWORD dwRTNID = /* create thread with function srThread() */;
if (dwRTNID == 0)
{
palIOShutdown();
srUninit();
palOSUninit();
return -1;
}

/* start any IM thread */
...

/* target application code here */
...

/* stop all IM threads */
...

/* stop Runtime thread */
palNotify(dwRTNID, palSTOP_EVENT);

/* uninitialize STRIDE subsystem */
palIOShutdown();
srUninit();
palOSUninit();

return 0;
}
</source>

==Testing the Runtime Integration==
Once the instrumented target is successfully built, start the target application. On a separate host computer, connected to the target via TCP/IP (or serial port if so configured), run '''[[STRIDE_Runner|stride]]''' as follows (substitute the <tt>--device</tt> option argument as needed):

<source lang="bash">
stride --diagnostics --device=<device_address>
</source>

By specifying <tt>--diagnostics</tt> without also specifying a database, we instruct stride to test for connectivity with the target.

If all is well, the following will be output to the host window (the reported runtime version may be different than that shown):

<pre>
Connecting to device...
runtime version: 5.x.yy
Disconnecting from device...
</pre>

Runtime Integration

2015-07-07T16:27:45Z

Marku: /* Overview */

==Overview==

This page discusses the first step in an overall STRIDE framework integration. The ultimate goals are to extend your target build process to:
* Incorporate the STRIDE Runtime source files in your target build
* Configure compiler settings
* Add code to target source to start and stop STRIDE runtime threads
* Incorporate the STRIDE Build Tools in your make process as a pre-compilation step

Here we incorporate the [[STRIDE Runtime]] in your target application build without tests. In the second and last step, we add the STRIDE [[Build Tools]] to the mix and hook up the test-running harnessing.

After building the target with the STRIDE Runtime in place, we will run stride from a remote host computer to verify connectivity and runtime operation.

=== Recommendations ===
The STRIDE target environment offers lots of flexibility to accommodate different target scenarios. For an initial target integration, we recommend that you pursue the simplest STRIDE configuration that will yield results. Once this configuration is successfully integrated you can make adjustments as desired.

=== Integrating STRIDE Into Your Target Build ===
To add the STRIDE Runtime to your build process, the following changes must be made:

* Secure a PAL targeted to your target operating system
* Configure the PAL I/O parameters to conform the your target hardware
* Include the STRIDE runtime source files in the software build
* Edit your target application's source code to start the runtime and I/O thread(s)

== Securing a PAL ==
The [[Platform_Abstraction_Layer|Platform Abstraction Layer (PAL)]] provides the glue between the STRIDE Runtime and services offered by your OS. It is the only piece of the runtime that is customized between operating systems.

Fully tested PALs are provided for Posix and Windows, included as <tt>palcfg.h</tt>, <tt>palIO.c</tt> and <tt>palOS.c</tt> in their respective [[Framework_Setup#SDK|SDK]].

== Configuring PAL I/O parameters ==
If your target will communicate with the host computer via the default of TCP/IP port 8000, you can skip this step.

Otherwise, refer to <tt>palcfg.h</tt> that comes with your PAL for a list of configuration parameters.

== Including the STRIDE Runtime ==
The [[Runtime_Reference|STRIDE Runtime]] is distributed in source form. The source files are included with each [[Framework_Setup#SDK|SDK]] under "Runtime" directory. Files in that directories should be included in your target build.

If you are using an S2-supplied SDK, additional required sources can be found in the SDK's <tt>src</tt> directory:

* PAL implementation
** <tt>palcfg.h</tt>
** <tt>palIO.c</tt>
** <tt>palOS.c</tt>

* Convenience functions (these wrap low-level PAL/Runtime calls)
** <tt>stride.c</tt>
** <tt>stride.h</tt>

* Build script
** <tt>Makefile</tt>

The build script (Makefile) supplied with an SDK allows building of the runtime source into a library that could then be linked with the target application.

=== C/C++ Feature Support ===
By default the C source in the Runtime was written with the presumption that the target C compiler and C Runtime (CRT) library supports '''wchar_t''', '''long long''', '''long double''', '''safe snprintf''', and '''variadic macros'''

By default the C++ source in the Runtime was written with the presumption that the target C++ compiler can handle '''namespaces''' and '''exceptions'''.

If desired the user can specify different configuration by using '''-D''' compiler options:

-DsrCRT_HAS_WCHAR_T=<0|1>
-DsrCRT_HAS_LONG_LONG=<0|1>
-DsrCRT_HAS_LONG_DBL=<0|1>
-DsrCRT_HAS_SNPRINTF=<0|1>
-DsrCRT_HAS_VA_ARGS=<0|1>
-DsrCPP_HAS_NAMESPACE=<0|1>
-DsrCPP_HAS_EXCEPTION=<0|1>

where 0=disabled and 1=enabled.

=== STRIDE Feature Control ===
Any application's source file referencing STRIDE source should be compiled with addition ''STRIDE_ENABLED'' switch (preprocessor directive):
<source lang="bash">
cc -DSTRIDE_ENABLED=1 file.c
</source>

The definition of the <tt>STRIDE_ENABLED</tt> macro controls all STRIDE-related sources and macros. If undefined (or defined as zero), STRIDE-related sources are excluded and STRIDE-related macros are left undefined. This provides a simple means of including and excluding test-related entities in your production build.

== Target Source Changes ==

Your target source must be edited to include code that initializes and uninitializes the runtime. The code is typically added to your <tt>main()</tt> function, but the only requirement is that the initialization takes place before the host computer connects and the uninitialization occurs before the process terminates.

Please refer to the standard pre-packaged [[Framework_Setup#SDK|SDK]] for examples of a startup sequence. Briefly it should be something like:

<source lang="c">
/* STRIDE runtime includes */
#include <stride.h>

int main(int argc, char **argv)
{
...

/* initialize the STRIDE subsytem using default I/O */
#ifdef __cplusplus
strideIO_t io = {strideIO_t::strideDEFAULT};
#else
strideIO_t io = {strideDEFAULT};
#endif
if (strideInit(&io) != srTRUE)
return -1;

/* start any IM thread */
...

/* target application code here */
...

/* stop all STRIDE threads and uninitialize STRIDE subsystem */
strideUninit();

return 0;
}
</source>

If you implement a custom PAL then here is an example of the startup code using direct PAL and STRIDE Runtime API calls instead of the stride.c/.h wrappers provided by the SDK:

<source lang="c">
/* STRIDE runtime includes */
#include <palcfg.h>
#include <sr.h>

int main(int argc, char **argv)
{
palDWORD dwRTNID;

...

/* initialize the PAL handler */
if (palOSInit() != palTRUE)
return -1;

/* initialize the PAL IO handler */
if (palIOInit(PAL_DEFAULT_DEVICE_NAME) != palTRUE)
{
palOSUninit();
return -1;
}

/* initialize STRIDE Runtime */
if (srInit() != srOK)
{
palIOShutdown();
srUninit();
palOSUninit();
return -1;
}

/* start the Runtime thread first */
palDWORD dwRTNID = /* create thread with function srThread() */;
if (dwRTNID == 0)
{
palIOShutdown();
srUninit();
palOSUninit();
return -1;
}

/* start any IM thread */
...

/* target application code here */
...

/* stop all IM threads */
...

/* stop Runtime thread */
palNotify(dwRTNID, palSTOP_EVENT);

/* uninitialize STRIDE subsystem */
palIOShutdown();
srUninit();
palOSUninit();

return 0;
}
</source>

==Testing the Runtime Integration==
Once the instrumented target is successfully built, start the target application. On a separate host computer, connected to the target via TCP/IP (or serial port if so configured), run '''[[STRIDE_Runner|stride]]''' as follows (substitute the <tt>--device</tt> option argument as needed):

<source lang="bash">
stride --diagnostics --device=<device_address>
</source>

By specifying <tt>--diagnostics</tt> without also specifying a database, we instruct stride to test for connectivity with the target.

If all is well, the following will be output to the host window (the reported runtime version may be different than that shown):

<pre>
Connecting to device...
runtime version: 5.x.yy
Disconnecting from device...
</pre>

Framework Setup

2015-07-07T16:26:34Z

Marku: /* SDK */

= Introduction =
The packages described below contain all of the ''source'' and ''binary'' components required to
* setup a desktop with the '''STRIDE Runner'''
* integrate the '''STRIDE Runtime''' with the target device
* add the '''STRIDE Compiler''' (aka ''Build tools'') to the software build system.

The desktops supported are Windows, Linux, and FreeBSD.

= Packages =
Files are installed by unzipping the provided package to your PC. Packages are available targeting the following operating systems (your version number may be different than that shown):
;Windows (x86)
:<tt>STRIDE_framework-windows_5.x.yy.zip</tt>
;Linux (x86)
:<tt>STRIDE_framework-linux_5.x.yy.tgz</tt>
;FreeBSD (x86)
:<tt>STRIDE_framework-freebsd_5.x.yy.tgz</tt>

Please see the appropriate installation instructions below.

= Windows =

The following installation example assumes the the installation package is located in your root directory and that the directory <tt>\stride</tt> exists. You can choose to install to a different location (all instructions below assume you are installing into <tt>\stride</tt>).

The example uses the open source [http://www.7-zip.org/ 7-Zip] utility to unzip the archive.

cd \stride
"\Program Files\7-Zip\7z" x ..\STRIDE_framework-windows_5.x.yy.zip

Once unzipped, files will have been installed under the <tt>\stride</tt> directory.

=== Updated PATH ===
As a final step, you will need to update your <tt>[http://en.wikipedia.org/wiki/Path_(variable) PATH]</tt> environment variable to include <tt>\stride\bin</tt>.
For instructions on modifying it, please see [http://support.microsoft.com/kb/310519 http://support.microsoft.com/kb/310519].

NOTE: ''Make sure to insert '''no spaces''' before and after the semicolon separators(;).''

=== Create/Update STRIDE_DIR===

Verify that the <tt>STRIDE_DIR</tt> environment variable exists and is set to the root installation directory (<tt>\stride</tt>). If this environment variable does not yet exist, you should create it as a user environment variable.

To confirm installation and display ''help'' run the following command in a console window:

stride -h

=== Uninstalling ===
To uninstall STRIDE simply:
* Remove any reference to <tt>\stride\bin</tt> in your <tt>PATH</tt> environment variable.
* Remove <tt>STRIDE_DIR</tt> environment variable.
* Remove <tt>\stride</tt> directory.

= Linux/FreeBSD =

The following installation example assumes the the installation package is located in your home directory and that the directory <tt>~/stride</tt> exists. You can choose to install to a different location (all instructions below assume you are installing into <tt>~/stride</tt>).

cd ~/stride
tar -zxvf ../STRIDE_framework-linux_5.x.yy.tgz

Once unzipped, files will have been installed under the <tt>~/stride</tt> directory.

=== Updated PATH ===
As a final step, you will need to update your <tt>[http://en.wikipedia.org/wiki/Path_(variable) PATH]</tt> environment variable to include <tt>~/stride/bin</tt>.

If you use the bash shell, enter the following at a command prompt, or to automatically set at each login, add to your <tt>.bashrc</tt>:
export PATH=$PATH:~/stride/bin

For other shells, and more information, please see the following articles:
* [http://www.linuxheadquarters.com/howto/basic/path.shtml http://www.linuxheadquarters.com/howto/basic/path.shtml].
* [http://en.wikipedia.org/wiki/Environment_variable#UNIX http://en.wikipedia.org/wiki/Environment_variable]

=== Create/Update STRIDE_DIR===
Verify that the <tt>STRIDE_DIR</tt> environment variable exists and is set to the root installation directory (<tt>~/stride</tt>). If this environment variable does not yet exist, you should automatically set at each login, add to your <tt>.bashrc</tt>:
export STRIDE_DIR=~/stride

To confirm installation and display ''help'' run the following command in a console window:

stride -h

NOTE: ''In a 64-bit environment the above may fail with errors like: <code>"/lib/ld-linux.so.2: bad ELF interpreter: No such file or directory"</code> or <code>"ELF interpreter /libexec/ld-elf32.so.1 not found"</code>. To resolve this issue install the appropriate 32-bit compatibility libraries for your Linux/FreeBSD distribution:''

* Debian / Ubuntu
sudo apt-get install ia32-libs
sudo apt-get install ia32-libs-multiarch:i386 (for 12.04 or higher)
* Fedora / CentOS / RHEL
sudo yum -y install glibc.i686 libstdc++.i686
* FreeBSD
Make sure to have <code>lib32</code> installed (via [http://www.freebsd.org/cgi/man.cgi?query=sysinstall&apropos=0&sektion=0&manpath=FreeBSD+8.4-RELEASE&arch=default&format=html sysinstall(8)] - Configure|Distributions|lib32) and have your kernel built with:
options COMPAT_FREEBSD32 # Compatible with i386 binaries

=== Uninstalling ===
To uninstall STRIDE simply:
* Remove any reference to <tt>~/stride/bin</tt> in your <tt>PATH</tt> environment variable.
* Remove <tt>STRIDE_DIR</tt> environment variable.
* Remove <tt>~/stride</tt> directory.

= Directories and Files =

To integrate Stride in to your target build system it is required to understand the directories layout and the files inside then. A quick orientation is shown below.

''NOTE:'' ''It's not necessary to understand the workings of Stride to perform evaluation or training. The framework package contains a [[Stride Sandbox]] that utilizes a SDK that is set up with appropriate options and settings to enable "out of the box" functionality.''

==<tt>bin</tt>==
This directory contains the [[Build Tools|Stride Build Tools]] and the [[Stride Runner]].

The build tools are invoked early on in the target software build process to generate special Stride artifacts that are used in subsequent build steps and later when running tests against the target. When using the Stride Sandbox, these files are needed on the host computer since this is where we are building the target application. In a production environment, these files are needed only on the computer that performs the target software build.

The [[Stride Runner]] is the program you use to run tests from the host.

==<tt>lib</tt>==
This directory contains a set of Stride specific core scripting libraries along with prebuild binaries intended to be used for Perl based test scripts.

==<tt>samples</tt>==
The Samples directory contains a number of source files used for training and evaluation.

==<tt>SDK</tt>==
This directory contains the sub-directories <tt>Posix/Windows</tt> and <tt>Runtime</tt>, which contain source code that comprises the [[STRIDE Runtime]]. These sources are built in to a static libary (e.g. STRIDE Runtime library - <tt>stride.a/lib</tt>) as a dependency of your Test Application.

The <tt>Posix</tt> and <tt>Windows</tt> directories contain the target operating system specific source and configuration. If you are interested in the details, consult the articles [[Posix SDK]] and [[Windows SDK]]. Each of them contains the following sub-directories:

*<tt>settings</tt>
: This directory contains a set of <tt>stride.XXX.s2scompile</tt> files, where <tt>XXX</tt> coresponds to the target CPU architecture (i.e. X86, ARM...). These files, used by the [[s2scompile|STRIDE Compiler]], specify target CPU characteristics (endian-ness, data sizes and alignments). On Windows, this directory also contains a set of files for [[STRIDE_Extensions_for_Visual_Studio|building with Visual Studio]].
*<tt>src</tt>
: This directory contains the source of the target [[Platform Abstraction Layer]] PAL. In addition there is a sample Makefile used to produce a sandbox TestApp.

= Perl Installation (Optional) =
If you intend to use '''Test Scripts''' you will need a recent version of Perl (x86 with threads support) installed.

As of this writing, we support only the 32-bit versions 5.8.9, 5.10.x, 5.12.x, 5.14.x, 5.16.x and 5.18.x of Perl.

== Windows ==
It is required to use the standard 32-bit Perl distributions from [http://www.activestate.com/activeperl ActiveState].

The following additional (non-standard) Perl packages are also required for full functionality of STRIDE tests in perl:

* [http://search.cpan.org/perldoc/Class::ISA Class::ISA]
* [http://search.cpan.org/perldoc/Pod::POM Pod::POM]
* [http://search.cpan.org/perldoc/Devel::Symdump Devel::Symdump]
* [http://search.cpan.org/perldoc/Config::Tiny Config::Tiny]

You can easily install these packages using the [http://docs.activestate.com/activeperl/5.16/faq/ActivePerl-faq2.html ppm tool]. If you access the Internet via a proxy make sure to read [http://docs.activestate.com/activeperl/5.16/faq/ActivePerl-faq2.html#ppm_and_proxies this]. Simple command-line installation of PACKAGE_NAME (the package to install) typically just requires typing:

ppm install PACKAGE_NAME

== Linux/FreeBSD ==
We recommend you to use the standard 32-bit Perl distribution that comes with your OS version. In case you need to manually build from source make sure to configure "shared library" (<tt>-Duseshrplib</tt>), "thread support" (<tt>-Duseithreads</tt>) and no "64-bit support" (<tt>-Uuse64bitint -Uuse64bitall</tt>).

The following additional (non-standard) Perl packages are also required for full functionality of Stride tests in perl:

* [http://search.cpan.org/perldoc/YAML::XS YAML::XS]
* [http://search.cpan.org/perldoc/Class::ISA Class::ISA]
* [http://search.cpan.org/perldoc/Pod::POM Pod::POM]
* [http://search.cpan.org/perldoc/Devel::Symdump Devel::Symdump]
* [http://search.cpan.org/perldoc/Config::Tiny Config::Tiny]

If your perl is installed in a system directory (<tt>/usr/bin/perl</tt>, for instance), you will need root access to install shared modules. The simplest method for installing packages is via the [http://www.perl.com/doc/manual/html/lib/CPAN.html CPAN shell]. If you access the Internet via a proxy make sure to set the appropriate [http://search.cpan.org/dist/CPAN/lib/CPAN.pm#Config_Variables CPAN config variables]. To start the shell in interactive mode:

sudo perl -MCPAN -eshell

Once in the shell, search for and install the latest stable version of PACKAGE_NAME (the package to install):

install PACKAGE_NAME

The STRIDE perl packages also need to load your system's '''libperl.so''' (shared object file) at runtime. Depending on your system, this file should be loadable from a perl CORE directory or from one of the shared system directories. If you '''DO NOT''' have this shared library on your system, you might need to install a ''libperl-dev'', ''perl-devel'' or ''perl-libs'' package in order to get it. Here is how you can do that on the console of some Linux distributions:

* Debian / Ubuntu
sudo apt-get install libperl-dev
* Fedora / CentOS / RHEL
sudo yum -y install perl-devel

== Validation ==
Once you have installed Perl we recommend you to run the following command in a console window:

<source lang="bash">
stride --diagnostics Perl --output PerlCheck
</source>

If everything was properly set up you should get the following output:

<pre>
Executing diagnostics...
script "/Perl"
> 2 passed, 0 failed, 0 in progress, 0 not in use, 486.95 ms.
---------------------------------------------------------------------
Summary: 2 passed, 0 failed, 0 in progress, 0 not in use, 486.95 ms

Saving result file...
</pre>

In addition a report file with name <tt>PerlCheck.xml</tt> will be created in the current directory. If interested in the details you could open that report file in a browser of your choice.

Test Point

2015-07-07T16:04:30Z

Marku: /* Write the Test Unit */

__NOTOC__
Source instrumentation is the process by which developers and domain experts selectively instrument the source under test for the purpose of writing test scenarios against the executing application. Implementing tests that leverage source instrumentation is called '''Expectation Testing'''. This validation technique is very useful for verifying proper code sequencing based on the software's internal design.

<source lang='c'>
#include <srtest.h>
...
/* a test point with no payload */
srTEST_POINT("first test point");

/* a test point with binary payload */
srTEST_POINT_DATA("second test point", myData, sizeofMyData);

/* a test point with simple string payload */
srTEST_POINT_STR("third test point", "payload with simple string");

/* a test point with formatted string payload */
srTEST_POINT_STR("third test point", "payload with format string %d", myVar);

#ifdef __cplusplus
srTEST_POINT_STR("c++ test point", "") << "stream input supported under c++";
#endif
</source>

Unlike traditional unit testing that drives testing based on input parameters and isolating functionality, ''Expectation'' testing is executed within a fully functional software build running on a real target platform. Test Points are not dependent on input parameters, but often leverage the same types of input/output controls used by functional and black-box testing.

Another unique feature of this type of testing is that domain expertise is not required to implement a test. Developers and domain experts use instrumentation to export design knowledge of the software to the entire team. Furthermore, there is no stubbing required, no special logic to generate input parameters, and no advanced knowledge required of how the application software is coded.

To enable effective test coverage, developers and domain experts are required to insert instrumentation at key locations to gain insight and testability. Here are some general suggested source code areas to consider instrumenting:
* critical function entry/exit points
* state transitions
* critical or interesting data transitions (using optional payload to convey data values)
* callback routines
* data persistence
* error conditions

The steps required to implement an Expectation test are the following:
#[[#Instrumentation | Instrumentation]]
#[[#Define your Expectations | Define your Expectations]]
#[[#Write the Test Unit | Write the Test Unit]]

== Instrumentation ==
To make the software ''testable'', the first step in the process is for the experts to selectively insert instrumentation macros into the source code. Test Points themselves have nominal impact on the performance of the application – they are only active during test data collection<ref name="n1"> Test data collection is typically implemented in a low priority background thread. The data is captured in the calling routine's thread context (no context switch) but processed in the background or on the host. Instrumentation macros return immediately to the caller (i.e. no-op) when testing is not active.</ref>. Test Points contain names and optional payload data. When Test Points are activated, they are collected in the background, along with timing and any associated data. The set of Test Points hit, their order, timing, and data content can all be used to validate that the software is behaving as expected. [[Test_Log | Test Logs]] can also be added to the source code to provide additional information in the context of an executing test.

To specify a test point you should include the '''srtest.h''' header file from the STRIDE Runtime in your compilation unit. The Test Point macros are active only when <tt>STRIDE_ENABLED</tt> is <tt>#define</tt>d, therefore it is practical to place these macros in-line in production source. When <tt>STRIDE_ENABLED</tt> is not <tt>#define</tt>d, these macros evaluate to nothing.

{| class="prettytable"
| colspan="2" | '''Test Point Macros'''
|-valign="top"
| '''srTEST_POINT'''(''label'')
| ''label'' is a pointer to a null-terminated string

|-valign="top"
| '''srTEST_POINT_DATA'''(''label'', ''data'', ''size'')
| ''label'' is a pointer to a null-terminated string 
''data'' is a pointer to a byte sequence 
''size'' is the size of the ''data'' in bytes

|-valign="top"
| '''srTEST_POINT_STR'''(''label'', ''message'')
| ''label'' is a pointer to a null-terminated string 
''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string 
When used in the context of a c++ compilation unit, this macro also supports the streaming operator to append to the message string (see example below)
|}

== Define your Expectations ==

In addition to instrumenting the source code with Test Points, you must also define the [[Expectations]] of the Test Points. This involves defining the list of Test Points expected to be hit during a given test scenario. Expectation can also include any '''data''' associated with a Test Point that requires validation.

== Write the Test Unit ==
Once the source under test has been instrumented and the [[Expectations]] defined, Stride offers a number of techniques that can be used for implementing '''expectation tests'''.

* Tests can be written in [[Test_Point_Testing_in_C/C%2B%2B | C or C++]] and executed on the device under test using the Stride framework.
* Tests can be written in [[Perl_Script_APIs | Perl]].

== Notes ==
<references/>

Test Double

2015-07-07T15:21:29Z

Marku: Created page with "__NOTOC__ The ''Test Double'' feature provides a means for intercepting C/C++ language '''global functions''' on the target, and directing the call to a substitute function wi..."

__NOTOC__
The ''Test Double'' feature provides a means for intercepting C/C++ language '''global functions''' on the target, and directing the call to a substitute function with identical parameters and return value. The use case is a unit test where the function under test uses a function during its execution, and this dependency is simulated by a substitute or double function during testing. The unit test is able to control the substitution of the dependency during run time, and thereby verify the behavior of the function under test.
The following sample illustrates the relationship of the function under test and a dependency:

<source lang="c">
// depend.h
int depend(int x);
</source>

<source lang="c">
// depend.c
#include "depend.h"

int depend(int x)
{
return x + x;
}
</source>

<source lang="c">
// test.h
int test(int x, int y);
</source>

<source lang="c">
// test.c
#include "test.h"
#include "depend.h"

int test(int x, int y)
{
return depend(x) * y;
}
</source>

In the above sample, ''test()'' is the function under test and ''depend()'' is a dependency candidate for doubling.

The steps required to achieve doubling of a dependency function are as follows:
#[[#Configuring the Double Using Function Pragma | Configuring the Double Using Function Pragma]]
#[[#Creating_Double_Intercepts_in_the_IM|Create Double Intercepts in the IM]]
#[[#Switching_the_Double_Function_During_Runtime|Switch Double Function During Runtime]]

==Configuring the Double Using Function Pragma==

The syntax of the [[Scl_function|function pragma]] supports a set of '''''optional''''' attributes that allow the specification of function interception parameters. When captured for the purpose of interception ('''intercept-able''') the optional arguments are '''required'''.

#pragma scl_function(function-name [,context, name-mangling, group-id])

Refer to the actual [[Scl_function | function pragma]] definition for an explanation of
* '''context''' - set to ''DEFINITION'' or ''REFERENCE''
* '''name-mangling''' - set to ''EXPLICIT'' or ''IMPLICIT''. Note if set to explicit requires usage of the ''srTEST_INTERCEPT'' macro
* '''group-id''' - user defined to enable or disable interception

In the above sample, ''depend()'' needs to be captured via the pragma and enabled for interception in the following manner.

<source lang="c">
// depend_scl.h
#include "depend.h"

#pragma scl_function(depend, "DEFINITION", "IMPLICIT", "TEST_GROUP")
</source>

==Creating Double Intercepts in the IM==
If a function has been configured as a double candidate using the function pragma as outlined in the above step, the [[Build_Tools|Stride Build Tools]] will automatically create the [[Intercept Module]], aka IM, that contains the intercept for the double function. This all happens automatically during the build process.

But to enable the interception the source file containing the <tt>depend()</tt> implementation '''must be altered''' to [[Name_Mangling|mangle the function's name]] by defining the Group ID and including the generated Intercept Module header file (xxxIM.h):

<source lang="c">
// depend.c
#include "depend.h"
/* define the Group ID before including the IM header */
#define TEST_GROUP
#include "strideIM.h"

int depend(int x)
{
return x + x;
}
</source>

==Switching the Double Function During Runtime==
In the context of the test unit code the following STRIDE runtime macros, defined in the STRIDE runtime header file '''srtest.h''', could be used for substituting a stub function for a double candidate.

<source lang="c">
srDOUBLE_SET(fn, fnDbl)
srDOUBLE_RESET(fn)

srDOUBLE_GET(fn, pfnDbl) // used for more advanced scenarios
</source>

Where:
*'''fn''' is the function qualified by <tt>scl_function</tt> or <tt>scl_func</tt> as a dependency candidate, as above.
*'''pfnDbl''' is a pointer to a object of type <tt>srFunctionDouble_t</tt>, declared to hold the current value of the active double function.
*'''fnDbl''' is a function that is to be the current active double. ''The function passed in should '''always''' match the signature of the dependency candidate specified by '''fn'''.''
'''''Note:''''' the initial value of the current active double is always the dependency candidate function.

=== Example 1 ===
The following example shows how to double a routine for the lifetime of a C++ Test Unit:

<source lang="cpp">
#include <srtest.h>
#include "depend.h"

extern "C" int depend_dbl(int a) { return a * a; }

class Test: public stride::srTest
{
public:

Test()
{
srDOUBLE_SET(depend, depend_dbl);
}

~Test()
{
srDOUBLE_RESET(depend);
}

int test1(void) { return test(1, 2); }
int test2(void) { return test(5, 6); }
};

#ifdef _SCL
#pragma scl_test_class(Test)
#pragma scl_function(depend, "DEFINITION", "IMPLICIT", "TEST_GROUP")
#endif
</source>

=== Example 2 ===
The following example shows how to make a call to the original routine in the context of a double by safely handling any nested doubling:

<source lang="c">
#include <srtest.h>
#include "depend.h"

extern "C" int depend_dbl(int a)
{
int ret;

srFunctionDouble_t fn;
srDOUBLE_GET(depend, &fn);
srDOUBLE_RESET(depend);
ret = depend(a);
srDOUBLE_SET(depend, fn);

return ret;
}
</source>

What is Unique About STRIDE

2015-07-07T15:17:21Z

Marku: /* Test Doubles */

STRIDE was designed by embedded engineers to facilitate implementing and executing '''On-Target White-box Testing'''. The various distinctive features describe here make it easy for both developers and testers to participate in validating the software during the development phase. The following is a high-level list of some of the unique features of STRIDE:
 
 

== STRIDE works on virtually any target platform ==
STRIDE's cross-platform framework facilitates a ''unified testing approach'' for all team members, enabling organizations to standardize on a test workflow that is independent of the target platform being used or what branch of software is being changed.

=== Runtime written in standard C ===
The [[Runtime_Reference | '''STRIDE Runtime''']] is written in standard C on top of a simple [[Platform_Abstraction_Layer | '''platform abstraction layer''']] that enables it to work across platforms. It can be configured for a single process multi-threading environment or multiple process environments (i.e. Linux, Windows, Embedded RTOS, etc.). It is delivered as source code to be included in the application's build system. The transport between the host and target is configurable and supports Serial and TCP/IP by default. Custom transports are also available. The runtime has also been tailored specifically to embedded applications, overhead is minimal. It consumes very little memory for table and control block storage. [[Frequently_Asked_Questions_About_STRIDE#What_is_the_size_of_the_STRIDE_Runtime.3F | Resource usage]] is configurable and can be tailored to the limitations of the target platform.

===Integrates With Your Existing Build System ===
STRIDE also auto-generates [[Intercept_Module | '''harnessing and remoting logic''']] as source code during the make process, removing any dependencies on specific compilers and / or processors.

===Supports Off-Target Testing ===
Testing can also be conducted using an [[Off-Target_Environment | '''Off-Target Environment''']] which is provided for desktop (Windows, Linux or FreeBSD) host machines. The [[Posix SDK | '''Posix''']] and [[Windows SDK | '''Windows''']] ''SDKs'' allow for a seamless transition between the real target and an Off-Target host environment.

== STRIDE provides built-in automation and reporting ==
STRIDE enables software builds to be both fully ''functional'' and ''testable'' at the same time. Built-in ''automation and reporting'' is similar in concept to a ''debug build'' except the focus is on testing.
===Testable Builds===
The ''testable build'' leverages the existing software build process by integrating into the same ''make system'' used by the development team (no one-off or special builds). Automatically included in the build is test automation controllable from the host. When tests are executed, an xml is generated on the host (and optionally uploaded to Test Space) which includes detailed test results and timing analysis. This xml file uses a custom schema for representing the [[Reporting_Model | '''hierarchy of results''']] (suites, cases, etc.). Also included is a stylsheet specification (which will be written to the same directory as the xml file) that allows the results to be viewed as HTML in a browser.

Developers can easily pre-flight test their source code changes before ''committing'' them to a baseline. Builds can be [[Setting_up_your_CI_Environment | '''automatically regression tested''']] as part of the daily build process.

===Functional Builds===
The ''testable software'' is still fully functional and works exactly the same as before. Whatever the software image was used for in the past -- system testing, developer debugging, etc. -- is still applicable. The STRIDE [[Test_Units | '''native test logic''']] is separated from the application source code and is NOT executed unless invoked via the [[Stride_Runner | '''runner''']]. The [[Source_Instrumentation_Overview | '''source instrumentation''']] is only active when executing tests. The impact of built-in testability to the software application is nominal.

The application can easily be switched back to a ''non-testable'' build by simply removing the [[Runtime_Integration#STRIDE_Feature_Control | '''STRIDE_ENABLED''']] preprocessor directive and rebuilding. This flag controls all STRIDE related source code and macros; there are no changes required to the build process to enable or disable this functionality.

== STRIDE offers testing techniques for deeper coverage ==
STRIDE offers numerous testing techniques that enable deeper and more effective testing with less effort.
===Test Macros===
[[Test_Macros | Test Macros in native code]] provide one-line shortcuts for validating assertions and automatic report annotation in the case of failures. Test Macros are supported in both C/C++ and [[Perl_Script_APIs#Assertions | our scripting solution]].

===Fixturing===
Setting up your ''Software Under Test'' to be in the right state for a test is critical for repeatability. STRIDE supports a set of [http://en.wikipedia.org/wiki/Test_fixture '''Test fixture techniques'''] that include
* [[Test_Fixturing_in_C/C%2B%2B#Specifying___Fixturing_Methods | '''Setup/Teardown''']]
* [[Parameterized_Test_Units | '''Parameter passing to tests''']]
* [[File_Transfer_Services | '''Host/Target file transfer''']]
* [[Function_Capturing | '''Function remoting''']]
* etc.

These techniques encourage best-practices such as partitioning of fixturing code and actual test code, and--in addition--these same techniques can be leveraged to dynamically configure your tests at run-time.

===Expectations===
STRIDE leverages [[Source_Instrumentation_Overview | '''source instrumentation''']] to provide '' [[Expectations | '''Expectations''']] ''as an additional validation technique that can be applied to the executing software application. The execution sequencing of the code, along with state data, can be automatically validated based on ''what is expected''. This validation technique does not focus on calling functions / methods but rather verifies ''code sequencing''. This can span threads, process boundaries, and even multiple targets, as the application(s) is running. Leveraging simple macros -- called [[Test_Point | '''Test Points''']] -- developers strategically instrument the source under test.

The test validation can be implemented in both [[Expectation_Tests_in_C/C%2B%2B | '''C/C++ on the target''']] and [[Perl_Script_APIs#STRIDE::Test | '''Perl script on the host''']]. In either case, the validation is done without impacting the application's performance (the on-target test code is executed in a background thread and scripts are executed on the host machine). When failures do occur, context is provided with the file name and associated line number of the failed expectations. This type of validation can be applied to a wide-range of testing scenarios:
* State Machines
* Data flow through system components
* Sequencing between threads
* Drivers that don't return values
* and much more ...

''Software Quality Assurance (SQA)'' can also leverage [[Expectations | '''Expectations''']] as part of their existing functional / system testing. Because the [[Stride_Runner | '''runner''']] is a command line utility, it is easily controlled from existing test infrastructure. SQA can use the [[Source_Instrumentation_Overview | '''instrumentation''']] to create their own [[Reporting_Model#Suites | '''test suite''']] using scripting that executes in concert with existing test automation.

===Test Doubles===
For more advanced testing scenarios, dependencies can be [[Test_Double | '''doubled''']]. This feature provides a means for intercepting C/C++ global functions on the target and substituting a stub, fake, or mock. The substitution is all controllable via the runtime, allowing the software to continue executing normally when not running a test.

===Other Features===
There are numerous other features that can be leveraged to facilitate deeper test coverage:
* Remoting global functions for script based API testing
* Built-in logging on test execution
* Dynamic test / suite creation
* Seamless publishing to [[STRIDE_Test_Space | '''Test Space''']]
* and much [[Test_API | ''' more ... ''']]

== STRIDE supports test implementation in C/C++ and Script ==
The test validation can be implemented in both ''native code'' on the target and ''script'' on the host.

===Tests in C/C++===
Writing API/Unit tests in [[Test_Units_Overview | '''native code''']] is the simplest way to begin validating the software. There is no new language to learn, no proprietary editor, and your normal programming workflow is not interrupted. Also there is no special APIs required to register tests, suites, etc. Just write your test in any combination of C/C++ and the auto-generated [[Intercept_Module | '''intercept module''']] via the [[Build_Tools | '''STRIDE build tools''']] takes care of everything. Tests from separate teams are automatically aggregated by the system -- no coordination is required.

This type of testing works well for:

* Calling APIs directly
* Validating C++ classes
* Isolating modules
* Critical processing / timing
* and much more...
===Tests in Script===
STRIDE also supports writing tests in [[Test_Modules_Overview | '''Perl script''']]. When writing test scripts there are minimal dependencies on the software build process. Tests can also validate global functions, setup conditions from the host, etc.

Scripts are well suited for Integration Testing focusing on:

* State Machines
* Data flow through system components
* Sequencing between threads
* and much more...

In either case, the validation is done without impacting the application's performance.

== STRIDE includes Web-based test results management ==
When executing '''tests''', results can be uploaded to [[STRIDE_Test_Space | '''STRIDE Test Space''']] for persistence and analysis. Test result data can be uploaded manually (using the web interface) or automatically using the [[Stride_Runner | '''Runner''']].
===Centralized Hosting===
Hosting test results in a ''central location'' with easy access via a browser enables the entire team to better participate in ensuring quality during the ongoing development process. [[STRIDE_Test_Space#Results_View | '''Reports''']] containing test results, timing, [[Tracing | '''tracing logs''']], and built-in test documentation (supported in both [[Test_API#Test_Documentation | '''C/C++''']] and [[Perl_Script_APIs#Documentation | '''Perl''']]) all correlated together enhances the context for all team members to manage testing and optimize resolving failures.
===Team Collaboration===
Team collaboration on testing activities is also facilitated with auto-generated email [[Notifications | '''notifications''']] and [[STRIDE_Test_Space#Messages | '''messaging''']]. Emails contain links to test failures where file names and line numbers are provided to aid in resolution. Messaging allows team members to more effectively communicate on the specifics of test results while minimizing information required in traditional emails, status reports, etc.

[[Category: Overview]]

Test API

2015-07-07T15:16:08Z

Marku:

The APIs for testing in C/C++ are documented in the following articles:

* [[Test Point Testing in C/C++|Test Point Testing API]] (for writing native test point validation tests)
* [[Runtime Test Services|Runtime Test Services API]] (for dynamic test result manipulation)
* [[File Transfer Services|File Transfer API]] (for host file manipulation from the device under test)
* [[Test Fixturing in C/C++|Test Fixturing]] (for test unit fixturing in native code)
* [[Test Documentation in C/C++|Test Documentation]] (how to document your native test units)
* [[Test Double|Test Doubles]] (for advanced dependency interception/function mocking in native code)

[[Category:Test Units]]

Stride Overview

2015-07-07T14:26:19Z

Marku:

__NOTOC__
'''Stride™''' is a [http://en.wikipedia.org/wiki/XUnit xUnit-style] test framework for C/C++ software comprised of an external test ''Runner'' , ''Runtime'' source code library, and a set of ''Build Tools''. It was designed with a specific focus on enabling tests to be executed on a device using a fully functional build. This means that the software works the same as before, but also has built-in tests that can be invoked on-demand by the test ''Runner''. Stride has also been integrated with [http://www.testspace.com Testspace] for test content management.

Stride's unique host-target architecture allows easier and better On-Target/Device White-box testing. Software builds automatically become more testable; facilitating deeper code coverage opportunities.

[[image:Stride Overview.jpg | 600px | center | Stride block diagram]]

==== Runner ====
Stride includes a lightweight [[Stride_Runner | '''Runner''']] application that is a host-based command-line utility for interactive and automated test execution. It also integrates seamlessly with [http://www.testspace.com Testspace] for ''test content management''.
 

====Runtime====
Stride also contains a [[STRIDE Runtime | '''Runtime''']] software source package that supports connectivity with the host system. It is integrated with the software application to enable ''testability'' to the software with minimal impact on performance or the size of the application. The software is agnostic to the RTOS, CPU, and transport. It can be configured to support multi-processes, multi-threads, user/kernel spaces, or a simpler single application process. Typically the runtime is configured as a background thread and only executes when running tests.

==== Build Tools ====
[[image:STRIDE 4.2 Framework-b.jpg |right|300px | border]]

The Stride [[STRIDE_Build_Tools | '''Build Tools''']] are a set of command-line utilities that integrate with your target software build process. They preprocess standard C/C++ header files that contain Stride test declarations to auto-generate source code comprising a custom [[Intercept_Module | '''Intercept Module''']]. The intercept module code is then built with the Stride Runtime to provide ''fixturing'' and ''harnessing'' test logic as part of your build image.

= Cross Platform =
Stride works on virtually any target platform. Stride's cross-platform framework facilitates a ''unified testing approach'' for all team members, enabling organizations to standardize on a test workflow that is independent of the target platform being used or what branch of software is being changed.

=== Runtime written in standard C ===
The Stride Runtime is written in standard C on top of a simple [[Platform_Abstraction_Layer | '''platform abstraction layer''']] that enables it to work across platforms. It can be configured for a single process multi-threading environment or multiple process environments (i.e. Linux, Windows, Embedded RTOS, etc.). It is delivered as source code to be included in the application's build system. The transport between the host and target is configurable and supports Serial and TCP/IP by default. Custom transports are also available. The runtime has also been tailored specifically to embedded applications, overhead is minimal. It consumes very little memory for table and control block storage. Resource usage is configurable and can be tailored to the limitations of the target platform.

===Integrates With Your Existing Build System ===
Stride also auto-generates ''intercept module'' used for harnessing and remote logic as source code during the make process, removing any dependencies on specific compilers and / or processors.

===Supports Off-Target Testing ===
Testing can also be conducted using a standard desktop machine (Windows, Linux, or FreeBSD). The Stride ''SDKs'' allow for a seamless transition between the real target and an Off-Target host environment.

= Rich set of Assertions =
Test cases are typically constructed leveraging a large set of available assertion macros. On failures macros provide such details as file name, line number, and why the condition failed. Stride provides are large set of optional macros available for automatic report annotation in the case of failures.

<source lang=cpp>
#include <mytest.h>

void MyTest::CheckFoo() {
..
srEXPECT_EQ(foo(2 + 2), 4);
}
void MyTest::CheckBoo() {
..
srEXPECT_GT(boo(3 * 3), 7);
}
</source>

= Extensive Parameter Support =

Parameterized testing is key to reducing test code duplication. Constructor arguments, name-value collections, and host file remoting are all supported. Stride provides an easy way for passing and accessing these types of parameters, allowing customization of test behavior at runtime.

Create an INI-type of file (i.e. myinput.ini)

Loopsize = 10
InputFile = ${MYPATH}/datacontent.bin
Toogle = On

Pass the name-value collection using the Stride Runner

$ stride .. --run="MyTest(/path/to/myinput.ini)"

Now simply access your input within test logic using built-in services

<source lang=cpp>
{
// .. doing stuff ..
GetParam("Loopsize", buffer, size);
//.. using parameters for test logic
}
</source>

= Test Doubles & Test Points =
Stride offers advanced testing techniques that enable deeper and more effective testing with less effort.

===Test Doubles===
For more advanced testing scenarios, dependencies can be '''doubled'''. This feature provides a means for intercepting C/C++ global functions on the target and substituting a stub, fake, or mock. The substitution is all controllable via the runtime, allowing the software to continue executing normally when not running a test.

For example something like the following could be useful:

<source lang=cpp>
void MySuite::test1()
{
// inserting my own version of malloc()
srDOUBLE_SET(malloc, my_malloc_stub);

// calling a routine that uses malloc(). checking handled null correctly
ret_code = FuncThatUsersMalloc(42);
srEXPECT_EQ(ret_code, -1)

// restoring the real malloc()
srDOUBLE_RESET(malloc)
}
</source>

===Test Points ===
Stride leverages ''source instrumentation'' to provide an additional validation technique called '''Expectation Testing''' that can be applied to the executing software application. The execution sequencing of the code, along with state data, can be automatically validated based on ''what is expected''. This validation technique does not focus on calling functions / methods but rather verifies ''code sequencing''. This can span threads, process boundaries, and even multiple targets, as the application(s) is running. Leveraging simple macros -- called '''Test Points''' -- developers strategically instrument the source under test.

<source lang='c'>

/* a test point with formatted string payload */
srTEST_POINT_STR("IMPORTANT", "important date= %d", myVar);

</source>

The test validation is done without impacting the application's performance (the on-target test code is executed in a background thread and scripts are executed on the host machine). When failures do occur, context is provided with the file name and associated line number of the failed expectations. This type of validation can be applied to a wide-range of testing scenarios:
* State Machines
* Data flow through system components
* Sequencing between threads
* Drivers that don't return values
* and much more ...

The following is a snippet of what the test code might look like
<source = lang='c'>

srTestPointExpect_t expected[]={
{"IMPORTANT"}, myCheckData,
{"ANOTHER_TP"},
{0}};

// setup the expections
srTestPointSetup(..);

// start the operations

srTestPointWait(handle, 1000);

</source>

= Test Implementation =
Stride based tests are integrated with the existing build system enabling your software to be both fully ''functional'' and ''testable'' at the same time. The ''test logic'' is separated from the application source code and is '''not''' executed unless invoked via the Stride Runner. Any ''source instrumentation'' is only active when executing tests. The impact of built-in testability to the software application is nominal.

The test validation can be implemented in both ''C/C++'' on the target and ''Perl'' on the host.

'''C/C++ Test Example'''
<source lang=cpp>
#include <srtest.h>

class MyTest {
public:
void CheckFoo();
void CheckBoo();
};
#pragma scl_test_class(MyTest)
</source>

<source lang=cpp>
#include <mytest.h>

void MyTest::CheckFoo() {
..
srEXPECT_EQ(foo(2 + 2), 4);
}
void MyTest::CheckBoo() {
..
srEXPECT_GT(boo(3 * 3), 7);
}
</source>

'''Perl Script Example'''
<source lang="perl">
use strict;
use warnings;

package MyTests;
use base qw(STRIDE::Test);
use STRIDE::Test;

sub CheckFoo : Test
{
EXPECT_EQ(Remote->foo(2 + 2), 4);
}

sub CheckBoo : Test
{
EXPECT_GT(Remote->boo(3 * 3), 7);
}

1;
</source>

= Works with Testspace =

Stride have been designed to leverage Testspace features such as test design, execution control, and test analysis. Managing test content and status become better organized and easier to assess current progress. Refer to [http://www.testspace.com Testspace] website for more details.

<hr/>


'''For more details refer to our [[Frequently Asked Questions About STRIDE | FAQ]] '''

Reference

2015-07-07T03:10:00Z

Marku: Created page with "* Runner * Build Tools * Runtime * Platform Abstraction Layer (PAL) * Inter..."

Test Doubles

2015-07-07T03:09:30Z

Marku: Created page with "* Test Double * Function Pragma * Name Mangling"

* [[Test Double]]
* [[Scl function | Function Pragma]]
* [[Name Mangling]]

Testing with Perl

2015-07-07T03:09:27Z

Marku: Created page with "* Test Script * Test API * Examples"

* [[Test Script]]
* [[Perl Script APIs | Test API]]
* [[Perl Script Snippets | Examples]]

Test Points

2015-07-07T03:09:23Z

Marku:

* [[Test Point]]
* [[Expectations | Expectations]]
* [[Test_Point_Testing_in_C/C%2B%2B | Writing Tests]]
* [[Test Log | Test Logs]]

Test Points

2015-07-07T03:08:21Z

Marku: Created page with "* Test Double * Function Pragma * Name Mangling"

* [[Test Double]]
* [[Scl function | Function Pragma]]
* [[Name Mangling]]

Runtime Test Services

2015-07-07T03:06:16Z

Marku:

__NOTOC__
The Runtime Test Services (declared in '''srtest.h''') are a set of APIs in the STRIDE Runtime that facilitate the writing of target based test code. These APIs make up an optional portion of the Stride Runtime and can be used to communicate additional information about tests to the host based reporting mechanism. These APIs also allow target test code to create additional test suites and test cases dynamically at runtime.

There are two ways of accessing these APIs: through C-based functions, or through a C++ base class from which you may derive your C++ test class. These are discussed below in [[#C Test Functions|C Test Functions]], and [[#C++ Test Classes|C++ Test Classes]] sections below.

= C Test Functions =

The following C APIs are provided:

*'''[[#srTestGetParam|srTestGetParam]]''': returns an input parameters value.

*'''[[#srTestSuiteAddCase|srTestSuiteAddCase]]''': creates an additional test case at runtime.
*'''[[#srTestSuiteAddAnnotation|srTestSuiteAddAnnotation]]''': creates a suite annotation at runtime.
*'''[[#srTestSuiteSetData|srTestSuiteSetData]]''': associates a custom name|value.

*'''[[#srTestCaseSetStatus|srTestCaseSetStatus]]''': explicitly sets the status for the specified test case.
*'''[[#srTestCaseAddAnnotation|srTestCaseAddAnnotation]]''': creates a test case annotation at runtime.
*'''[[#srTestSuiteSetData|srTestSuiteSetData]]''': associates a custom name|value pair with the specified test case.

*'''[[#srTestAnnotationAddComment|srTestAnnotationAddComment]]''': adds a comment to the specified annotation.

== srTestGetParam ==
The srTestGetParam() function is used to get the value of a named parameter as a string.
<source lang=c>
srWORD srTestGetParam(const srCHAR * szName, srCHAR * szValue, srDWORD dwSize, const srCHAR * szDefValue);
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| szName
| Input
| Pointer to a null-terminated string that represents the name of the parameter. Cannot be null.
|-
| szValue
| Output
| Pointer to a block of memory to store the value of the parameter with a maximum size of dwSize chars
|-
| dwSize
| Input
| Size in chars of the buffer pointed to by szValue
|-
| szDefValue
| Input (optional)
| Pointer to a null-terminated string that represents a default value in case the parameter is not specified. By setting this value to srNULL (or omitting it), you can use srTestGetParam() to test for the existence of a named parameter. If the named parameter doesn't exist, the function will return srERR_HANDLE_INVALID.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srOK
| Success
|-
| srERR
| Null or empty szName string passed in.
|-
| srERR_HANDLE_INVALID
| The named parameter is not found and szDefValue is set to srNULL.
|}

<source lang=c>
#include <srtest.h>

#define MAX_VALUE_LEN 128
void tfsuite_getParam(void)
{
srCHAR szValue[MAX_VALUE_LEN] = {0};

srWORD wRet = srTestGetParam("ParamName", szValue, MAX_VALUE_LEN, "default value");
}

#ifdef _SCL
#pragma scl_test_flist("testfunc", tfsuite_getParam)
#endif
</source>

== srTestGetParamLong ==
The srTestGetParamLong() function is used to get the value of a named parameter as a long.
<source lang=c>
srLONG srTestGetParamLong(const srCHAR * szName, srLONG lDefValue);
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| szName
| Input
| Pointer to a null-terminated string that represents the name of the parameter.
|-
| lDefValue
| Input
| Default value that is returned if the parameter is not found.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srLONG
| Parameter value, or value specified by lDefValue if parameter isn't found.
|}

<source lang=c>
#include <srtest.h>

void tfsuite_getParam(void)
{
srLONG lVal = srTestGetParamLong("ParamName", -1);
}

#ifdef _SCL
#pragma scl_test_flist("testfunc", tfsuite_getParam)
#endif
</source>

== srTestGetParamDouble ==
The srTestGetParamDouble() function is used to get the value of a named parameter as a double.
<source lang=c>
double srTestGetParamDouble(const srCHAR * szName, double dbDefValue);
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| szName
| Input
| Pointer to a null-terminated string that represents the name of the parameter.
|-
| dbDefValue
| Input
| Default value that is returned if the parameter is not found.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| double
| Parameter value, or value specified by dbDefValue if parameter isn't found.
|}

<source lang=c>
#include <srtest.h>

void tfsuite_getParam(void)
{
double dbVal = srTestGetParamDouble("ParamName", 0.5772156649);
}

#ifdef _SCL
#pragma scl_test_flist("testfunc", tfsuite_getParam)
#endif
</source>

== srTestSuiteAddCase ==
The srTestSuiteAddCase() routine is used to add a new test case to the specified test suite.
<source lang=c>
srTestCaseHandle_t srTestSuiteAddCase(const srCHAR * szName, const srCHAR * szDescr)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| szName
| Input
| Pointer to a null-terminated string that represents the name of the new test case. If null, the default host naming scheme will be used.
|-
| szDescr
| Input
| Pointer to a null-terminated string representing the description of the new test case. If null, description will be empty.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srTestCaseHandle_t
| Handle of the newly created test case. srTEST_CASE_INVALID indicates failure to create a test case.
|}

<source lang=c>
#include <srtest.h>
#include <stdio.h>

void tfsuite_addCase(void)
{
for (int count = 0; count < 5; ++count)
{
char szName[25];
sprintf(szName, "dynamic test case %d", count);
srTestCaseHandle_t test = srTestSuiteAddCase(szName, "this is a dynamic test case");
srTestCaseSetStatus(test, srTEST_PASS, 0);
}
}

#ifdef _SCL
#pragma scl_test_flist("testfunc", tfsuite_addCase)
#endif
</source>

== srTestSuiteAddAnnotation ==
The srTestSuiteAddAnnotation() routine is used to add a new annotation to the specified test suite.
<source lang=c>
srTestAnnotationHandle_t srTestSuiteAddAnnotation(srTestAnnotationLevel_e eLevel,
const srCHAR * szName,
const srCHAR * szFmtDescr, ...)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| eLevel
| Input
| Annotation level.

* srTEST_ANNOTATION_LEVEL_TRACE,
* srTEST_ANNOTATION_LEVEL_DEBUG,
* srTEST_ANNOTATION_LEVEL_INFO,
* srTEST_ANNOTATION_LEVEL_WARN,
* srTEST_ANNOTATION_LEVEL_ERROR,
* srTEST_ANNOTATION_LEVEL_FATAL
|-
| szName
| Input
| Pointer to a null-terminated string representing the name of the new annotation. If null, the default host naming scheme will be used.
|-
| szFmtDescr
| Input
| Pointer to a null-terminated string representing the description of the new annotation. Cannot be null.
|-
| ...
| Input (Optional)
| Variable argument list to format szFmtDescr.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srTestAnnotationHandle_t
| Handle of the newly created test annotation. srTEST_ANNOTATION_INVALID indicates failure to create a test annotation.
|}

<source lang=c>
#include <srtest.h>

void tfsuite_addAnnotation(void)
{
for (int count = 0; count < 5; ++count)
{
char szName[25];
sprintf(szName, "annotation %d", count);
srTestAnnotationHandle_t annot =
srTestSuiteAddAnnotation(srTEST_ANNOTATION_LEVEL_ERROR,
szName,
"description of annotation");
}
}

#ifdef _SCL
#pragma scl_test_flist("testfunc", tfsuite_addAnnotation)
#endif
</source>

== srTestSuiteSetData ==
The srTestSuiteSetData() routine is used to associate a custom name|value pair with a test suite.
<source lang=c>
srWORD srTestSuiteSetData(const srCHAR * szName, const srCHAR * szFmtValue, ...)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| szName
| Input
| Pointer to a null-terminated string representing the name of the custom pair. Cannot be null.
|-
| szFmtValue
| Input
| Pointer to a null-terminated string representing the value of the custom pair. Cannot be null.
|-
| ...
| Input (Optional)
| Variable argument list to format szFmtValue.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srWORD
| srOK on success, srERR_HANDLE_INVALID on invalid handle.
|}

<source lang=c>
#include <srtest.h>

void tfcase_setData(void)
{
srTestSuiteSetData("MyName", "my value");
}

#ifdef _SCL
#pragma scl_test_flist("testfunc", tfcase_setData)
#endif
</source>

== srTestCaseSetStatus ==
The srTestCaseSetStatus() routine is used to set the result of test case.
<source lang=c>
srWORD srTestCaseSetStatus(srTestCaseHandle_t tTestCase, srTestStatus_e eStatus, srDWORD dwDuration)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| tTestCase
| Input
| Handle to a test case. srTEST_CASE_DEFAULT can be used for the default test case.
|-
| eStatus
| Input
| Result of the test. Possible values are:
* '''srTEST_FAIL'''
* '''srTEST_PASS'''
* '''srTEST_NOTINUSE'''
* '''srTEST_INPROGRESS'''
* '''srTEST_DONE''' - applicable to dynamic cases - sets the status to '''pass''' unless already set to '''fail''' or '''not-in-use'''
|-
| dwDuration
| Input
| The duration of the test in clock ticks. Using "0" allows the STRIDE Runtime (Intercept Module) to set the time automatically.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srWORD
| srOK on success, srERR_HANDLE_INVALID on invalid handle.
|}

<source lang=c>
#include <srtest.h>

void tfcase_setStatus(void)
{
srTestCaseSetStatus(srTEST_CASE_DEFAULT, srTEST_PASS, 0);
}

#ifdef _SCL
#pragma scl_test_flist("testfunc", tfcase_setStatus)
#endif
</source>

== srTestCaseSetStatusEx ==
The srTestCaseSetStatusEx() routine is used to set the result of test case and allow specification of an extendedFailureCode.
<source lang=c>
srWORD srTestCaseSetStatus(srTestCaseHandle_t tTestCase, srTestStatus_e eStatus, srDWORD dwDuration, srLONG lExtendedFailureCode)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| tTestCase
| Input
| Handle to a test case. srTEST_CASE_DEFAULT can be used for the default test case.
|-
| eStatus
| Input
| Result of the test.
|-
| dwDuration
| Input
| The duration of the test in clock ticks. Using "0" allows the STRIDE Runtime (Intercept Module) to set the time automatically.
|-
| lExtendedFailureCode
| Input
| The Stride framework uses the extendedFailureCode to capture the numeric results of test method when the test method fails via a numeric (non-void, nonbool) return type.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srWORD
| srOK on success, srERR_HANDLE_INVALID on invalid handle.
|}

<source lang=c>
#include <srtest.h>

void tfcase_setStatusEx(void)
{
srTestCaseSetStatusEx(srTEST_CASE_DEFAULT, srTEST_FAIL, 0, -5);
}

#ifdef _SCL
#pragma scl_test_flist("testfunc", tfcase_setStatusEx)
#endif
</source>

== srTestCaseAddAnnotation ==
The srTestCaseAddAnnotation() routine is used to add a new annotation to the specified test case.
<source lang=c>
srTestAnnotationHandle_t srTestCaseAddAnnotation(rTestCaseHandle_t tTestCase, srTestAnnotationLevel_e eLevel, const srCHAR * szName, const srCHAR * szFmtDescr, ...)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| tTestCase
| Input
| Handle to the parent test case to which new test annotation is to be added. srTEST_CASE_DEFAULT can be used for the default test case.
|-
| eLevel
| Input
| Annotation level.

* srTEST_ANNOTATION_LEVEL_TRACE,
* srTEST_ANNOTATION_LEVEL_DEBUG,
* srTEST_ANNOTATION_LEVEL_INFO,
* srTEST_ANNOTATION_LEVEL_WARN,
* srTEST_ANNOTATION_LEVEL_ERROR,
* srTEST_ANNOTATION_LEVEL_FATAL
|-
| szName
| Input
| Pointer to a null-terminated string representing the name of the new annotation. If null, the default host naming scheme will be used.
|-
| szFmtDescr
| Input
| Pointer to a null-terminated string representing the description of the new annotation. Cannot be null.
|-
| ...
| Input (Optional)
| Variable argument list to format szFmtDescr.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srTestAnnotationHandle_t
| Handle of the newly created test annotation. srTEST_ANNOTATION_INVALID indicates failure to create a test annotation.
|}

<source lang=c>
#include <srtest.h>

void tfcase_addAnnotation(void)
{
for (int count = 0; count < 5; ++count)
{
char szName[25];
sprintf(szName, "annotation %d", count);
srTestAnnotationHandle_t annot =
srTestCaseAddAnnotation(srTEST_CASE_DEFAULT,
srTEST_ANNOTATION_LEVEL_ERROR,
szName,
"description of annotation");
}
}

#ifdef _SCL
#pragma scl_test_flist("testfunc", tfcase_addAnnotation)
#endif
</source>

== srTestCaseSetData ==
The srTestCaseSetData() routine is used to associate a custom name|value pair with a test case.
<source lang=c>
srWORD srTestCaseSetData(srTestCaseHandle_t tTestCase, const srCHAR * szName, const srCHAR * szFmtValue, ...);
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| tTestCase
| Input
| Handle to a test case. srTEST_CASE_DEFAULT can be used for the default test case.
|-
| szName
| Input
| Pointer to a null-terminated string representing the name of the custom pair. Cannot be null.
|-
| szFmtValue
| Input
| Pointer to a null-terminated string representing the value of the custom pair. Cannot be null.
|-
| ...
| Input (Optional)
| Variable argument list to format szFmtValue.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srWORD
| srOK on success, srERR_HANDLE_INVALID on invalid handle.
|}

<source lang=c>
#include <srtest.h>

void tfcase_setData(void)
{
srTestCaseSetData(srTEST_CASE_DEFAULT, "MyName", "my value");
}

#ifdef _SCL
#pragma scl_test_flist("testfunc", tfcase_setData)
#endif
</source>

== srTestAnnotationAddComment ==
The srTestAnnotationAddComment() routine is used to add a comment (aka a log) to a test annotation.
<source lang=c>
srWORD srTestAnnotationAddComment(srTestAnnotationHandle_t tTestAnnotation, const srCHAR * szLabel, const srCHAR * szFmtComment, ...)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| tTestAnnotation
| Input
| Handle to a test annotation.
|-
| szLabel
| Input
| Pointer to a null-terminated string for the label. If null, the default label will be applied.
|-
| szFmtComment
| Input
| Pointer to a null-terminated string representing the new comment. Cannot be null.
|-
| ...
| Input (Optional)
| Variable argument list to format szFmtComment.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srWORD
| srOK on success, srERR on null string or invalid formatted string, srERR_HANDLE_INVALID on invalid handle, srERR_STR_TRUNCATED on truncated string.
|}

<source lang=c>
#include <srtest.h>

void tfsuiteAnnotation_addComment(void)
{
srTestAnnotationHandle_t annot =
srTestSuiteAddAnnotation(srTEST_ANNOTATION_LEVEL_ERROR,
"annot",
"annot description");
srTestAnnotationAddComment(annot,
srNULL,
"this comment should print %s", "A STRING");
}

#ifdef _SCL
#pragma scl_test_flist("testfunc", tfsuiteAnnotation_addComment)
#endif
</source>

= C++ Test Classes =
The Runtime Test Services C APIs work equally well from C test functions and C++ test classes. If, however, you are using C++ it might be more convinient for you to derive your C++ test classes from the Runtime Test Services C++ base class, ''srTest''. That way you will have access to a set of simpler to use C++ classes.

The following C++ classes are provided:
* '''[[#class srTest|class srTest]]''' - base test class
* '''[[#class srTestCase|class srTestCase]]''' - represents a test case
* '''[[#class srTestAnnotation|class srTestAnnotation]]''' - represents a test annotation

== class srTest ==
The srTest class provides one member object ''testCase'' of [[#class srTestCase|class srTestCase]] and the folowing methods:

=== method GetParam ===
This overload of the GetParam() method is used to get the value of a named parameter as a string.
<source lang=cpp>
static srWORD GetParam(const srCHAR * name, srCHAR * value, srDWORD size, const srCHAR * defval = srNULL)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| name
| Input
| Pointer to a null-terminated string that represents the name of the parameter. Cannot be null.
|-
| value
| Output
| Pointer to a block of memory to store the value of the parameter with a maximum size of size chars
|-
| size
| Input
| Size in chars of the buffer pointed to by value
|-
| defvalue
| Input (optional)
| Pointer to a null-terminated string that represents a default value in case the parameter is not specified. By setting this value to srNULL (or omitting it), you can use GetParam() to test for the existence of a named parameter. If the named parameter doesn't exist, the function will return srERR_HANDLE_INVALID.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srWORD
| srOK on success, error otherwise.
|}

<source lang=cpp>
#include <srtest.h>

class srtest_class : public stride::srTest
{
public:
void getParam()
{
GetParam("ParamName", szValue, MAX_VALUE_LEN, "default value");
}
};

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
</source>

=== method GetParam ===
This overload of the GetParam() method is used to get the value of a named parameter as a long.
<source lang=cpp>
static long GetParam(const srCHAR * name, srLONG defval = 0)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| name
| Input
| Pointer to a null-terminated string that represents the name of the parameter.
|-
| defvalue
| Input
| Default value that is returned if the parameter is not found.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srLONG
| Parameter value, or value specified by defvalue if parameter isn't found.
|}

<source lang=cpp>
#include <srtest.h>

class srtest_class : public stride::srTest
{
public:
void getParam()
{
srLONG lValue = GetParam("ParamName", -1);
}
};

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif</source>

=== method GetParam ===
This overload of the GetParam() method is used to get the value of a named parameter as a double.
<source lang=cpp>
static double GetParam(const srCHAR * name, const double& defval)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| name
| Input
| Pointer to a null-terminated string that represents the name of the parameter.
|-
| defvalue
| Input
| Default value that is returned if the parameter is not found.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| double
| Parameter value, or value specified by defvalue if parameter isn't found.
|}

<source lang=cpp>
#include <srtest.h>

class srtest_class : public stride::srTest
{
public:
void getParam()
{
double dbVal = GetParam("ParamName", 0.5772156649);
}
};

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif</source>

=== method AddCase ===
The AddCase method is used to new test case to the test suite.
<source lang=cpp>
srTestCase AddCase(const srCHAR * name, const srCHAR * descr = srNULL)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| name
| Input
| Pointer to null-terminated string that represents the name of the new test case. If empty, the default host naming scheme will be used.
|-
| descr
| Input (Optional)
| Pointer to null-terminated string representing the description of the new test case. If empty, description will be blank.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srTestCase
| Newly created test case instance.
|}
 
'''Example'''
<source lang=cpp>
#include <srtest.h>
#include <sstream>

class srtest_class : public stride::srTest
{
public:
void suiteAddSuite()
{
const std::string prefix("dynamic test case ");
for (int count = 0; count < 5; ++count)
{
std::stringstream strm;
strm << prefix << count;
stride::srTestCase tc = AddCase(strm.str().c_str());
tc.SetStatus(srTEST_PASS);
}
}
};

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
</source>

=== method AddAnnotation ===
The AddAnnotation method is used to add a new test annotation to the test suite.
<source lang=cpp>
srTestAnnotation AddAnnotation(srTestAnnotationLevel_e level, const srCHAR * name, const srCHAR * descr, ...)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| level
| Input
| Annotation level.

* srTEST_ANNOTATION_LEVEL_TRACE,
* srTEST_ANNOTATION_LEVEL_DEBUG,
* srTEST_ANNOTATION_LEVEL_INFO,
* srTEST_ANNOTATION_LEVEL_WARN,
* srTEST_ANNOTATION_LEVEL_ERROR,
* srTEST_ANNOTATION_LEVEL_FATAL
|-
| name
| Input
| Pointer to null-terminated string that represents the name of the new annotation. If empty, the default host naming scheme will be used.
|-
| descr
| Input
| Pointer to null-terminated string representing the description of the new annotation. Cannot be null.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srTestAnnotation
| Newly created annotation instance.
|}

<source lang=cpp>
#include <srtest.h>
#include <sstream>

class srtest_class : public stride::srTest
{
public:
void suiteAddAnnotation()
{
for (int count = 0; count < 5; ++count)
{
std::stringstream strmName;
std::stringstream strmDescr;
strmName << "annotation " << count;
strmDescr << "description of annotation " << count;
stride::srTestAnnotation ta = AddAnnotation(srTEST_ANNOTATION_LEVEL_INFO,
strmName.str().c_str(),
strmDescr.str().c_str());
}
}
};

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
</source>

=== method SetData ===
The SetData method is used to associate a custom name|value pair with the test suite.
<source lang=cpp>
srWORD SetData(const srCHAR * name, const srCHAR * value, ...)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| name
| Input
| Pointer to a null-terminated string representing the name of the custom pair. Cannot be null.
|-
| value
| Input
| Pointer to a null-terminated string representing the value of the custom pair. Cannot be null.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srWORD
| srOK on success, srERR_HANDLE_INVALID on invalid handle.
|}
 
'''Example'''
<source lang=cpp>
#include <srtest.h>
#include <sstream>

class srtest_class : public stride::srTest
{
public:
void suiteSetData()
{
SetData("MyName", "my value");
}
};

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
</source>

== class srTestCase ==
The srTestCase class provides the following methods:

=== method SetStatus ===
The SetStatus method is used to set the result of the default test case.
<source lang=cpp>
srWORD SetStatus(srTestStatus_e status, srDWORD duration = 0)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| status
| Input
| Result of the test. Possible values are:
* '''srTEST_FAIL'''
* '''srTEST_PASS'''
* '''srTEST_NOTINUSE'''
* '''srTEST_INPROGRESS'''
* '''srTEST_DONE''' - applicable to dynamic cases - sets the status to '''pass''' unless already set to '''fail''' or '''not-in-use'''
|-
| duration
| Input (Optional)
| The duration of the test in clock ticks. The default is 0.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srWORD
| srOK on success, srERR_HANDLE_INVALID on invalid handle.
|}

<source lang=cpp>
#include <srtest.h>

class srtest_class : public stride::srTest
{
public:
void caseSetStatus()
{
testCase.SetStatus(srTEST_NOTINUSE);
}
};

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
</source>

=== method AddAnnotation ===
The AddAnnotation method is used to add a new test annotation to the test case.
<source lang=cpp>
srTestAnnotation AddAnnotation(srTestAnnotationLevel_e level, const srCHAR * name, const srCHAR * descr, ...)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| level
| Input
| Annotation level.

* srTEST_ANNOTATION_LEVEL_TRACE,
* srTEST_ANNOTATION_LEVEL_DEBUG,
* srTEST_ANNOTATION_LEVEL_INFO,
* srTEST_ANNOTATION_LEVEL_WARN,
* srTEST_ANNOTATION_LEVEL_ERROR,
* srTEST_ANNOTATION_LEVEL_FATAL
|-
| name
| Input
| Pointer to null-terminated string that represents the name of the new annotation. If null, the default host naming scheme will be used.
|-
| descr
| Input
| Pointer to null-terminated string representing the description of the new annotation. Cannot be null.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srTestAnnotation
| Newly created annotation instance.
|}

<source lang=cpp>
#include <srtest.h>
#include <sstream>

class srtest_class : public stride::srTest
{
public:
void caseAddAnnotation()
{
for (int count = 0; count < 5; ++count)
{
std::stringstream strmName;
std::stringstream strmDescr;
strmName << "annotation " << count;
strmDescr << "description of annotation " << count;
stride::srTestAnnotation ta =
testCase.AddAnnotation(srTEST_ANNOTATION_LEVEL_INFO,
strmName.str().c_str(),
strmDescr.str().c_str());
}
}
};

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
</source>

=== method SetData ===
The SetData method is used to associate a custom name|value pair with the test case.
<source lang=cpp>
srWORD SetData(const srCHAR * name, const srCHAR * value, ...)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| name
| Input
| Pointer to a null-terminated string representing the name of the custom pair. Cannot be null.
|-
| value
| Input
| Pointer to a null-terminated string representing the value of the custom pair. Cannot be null.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srWORD
| srOK on success, srERR_HANDLE_INVALID on invalid handle.
|}

<source lang=cpp>
#include <srtest.h>

class srtest_class : public stride::srTest
{
public:
void caseSetData()
{
testCase.SetData("MyName", "my value");
}
};

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
</source>

== class srTestAnnotation ==
The srTestAnnotation class provides the following methods:

=== method AddComment ===
The AddComment method is used to add a comment to the test annotation.
<source lang=cpp>
srWORD AddComment(const srCHAR * label, const srCHAR * comment, ...)
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| label
| Input
| Pointer to null-terminated string for the label. If null, the default label will be applied.
|-
| comment
| Input
| Pointer to null-terminated string representing the new comment. Cannot be empty.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srWORD
| srOK on success, srERR on null string or invalid formatted string, srERR_HANDLE_INVALID on invalid handle, srERR_STR_TRUNCATED on truncated string.
|}

<source lang=cpp>
#include <srtest.h>

class srtest_class : public stride::srTest
{
public:
void suiteAnnotationAddComment()
{
stride::srTestAnnotation ta =
testSuite.AddAnnotation(srTEST_ANNOTATION_LEVEL_INFO,
"annot",
"annot description");
ta.AddComment("this comment on annotation should print %s", "A STRING");
}
};

#ifdef _SCL
#pragma scl_test_class(srtest_class)
#endif
</source>

Runtime Test Services

2015-07-07T02:56:36Z

Marku: /* srTestGetParam */

Runtime Test Services

2015-07-07T02:55:57Z

Marku:

Test Unit

2015-07-07T02:45:23Z

Marku: /* Test Method Return Values */

__NOTOC__
'''Stride''' groups similar test cases into a single ''runnable'' package called a '''Test Unit''' (also known as a '''Test Suite'''). These tests are written in C and C++ and are compiled and linked with your software and run, when called, on your target platform. They are suitable for both developer unit testing as well as end-to-end integration testing. An external [[Stride Runner]] is provided which controls the execution of '''Test Units''' and publishes test results to the local file system and optionally to [http://www.testspace.com Testspace]. The following is an example of the structure of a '''Test Unit''':

'''Declare tests'''
<source lang=cpp>
#include <srtest.h>

class MyTest {
public:
void CheckFoo();
void CheckBoo();
};
#pragma scl_test_class(MyTest)
</source>

'''Write tests'''
<source lang=cpp>
#include <mytest.h>

void MyTest::CheckFoo() {
..
srEXPECT_EQ(foo(2 + 2), 4);
}
void MyTest::CheckBoo() {
..
srEXPECT_GT(boo(3 * 3), 7);
}
</source>

== Test Unit ==
A single '''Test Unit''' is a set of '''Test Methods''' that always run together as an '''executable unit'''. Test Methods are the '''test cases''' that comprise a Test Unit. Each Test Method by default maps to a single Test Case in the results. When relying on this default behavior of Test Methods, we refer to these methods as ''static Test Cases''. A less common technique is to create and add a Test Case dynamically at runtime to an existing Suite via the [[Runtime_Test_Services#method_AddCase|Test Services]]. We refer to these as ''dynamic Test Cases''.

Individual test cases are implemented as test methods or functions which follow a four-phase testing pattern:

# Setting up a test fixture (optional)
# Exercising the Software Under Test (SUT)
# Verifying that the expected outcome has occurred
# Tearing down the test fixture (optional)

'''Test fixturing''' refers to the ''Setup'' and ''Teardown'' phases of the testing.

In the '''Setup''' phase, we put all of the things into place that are required in order to run a our test and expect a particular outcome. This includes things like:
* Acquiring resources such as memory, hardware, etc.
* Setting up required states such as input files in place, memory filled with a pattern, dependencies initialized, etc.

In the '''Tear down''' phase, we clean up the fixturing we did in the Setup phase, leaving the system in a state that is ready to be used by the next test.

== Packaging ==

Individual functions or methods, which typically implement a single test case are grouped into one or more Test Units which are executed as atomic entities.

Grouping of individual tests into a Test Unit can be accomplished in any of three ways:

* A Test Unit can be comprised of the member functions of a '''C++ class''',
* A Test Unit can be comprised of a set of '''C functions''',
* A Test Unit can be comprised of C functions pointed to by members of a '''C struct'''

The following table compares the three packaging variants.

{| class="wikitable"
|+ <big>'''''Comparison of Test Unit Packaging'''''</big>
! Test Unit Type!! Advantages !! Disadvantages !! scl pragma
|-
|
'''C++ class'''

[[Test_Unit#Test_Unit_as_C.2B.2B_Class|Example code]]

|
* Simple and easy to use
* Can be used with target code that is all C, all C++, or a mix
* Test utility methods can be encapsulated as test class members or members of a parent class
* Could be [[Parameterized_Test_Units | parametrized]] via constructor arguments
* Provides convenient syntax for augmenting report annotation (operator <tt><<</tt>)
|
* Requires a C++ build environment
| <tt>scl_test_class</tt>
|-
|
'''C class'''

[[Test_Unit#Test_Unit_as_C_Class|Example code]]

|
* Provides encapsulation of test methods
* Provides encapsulation of state via member variables
* Could be [[Parameterized_Test_Units |parametrized]] via init-function arguments
|
* Requires initialization code for structure function pointer setup
* [[Test_Documentation_in_C/C%2B%2B|Test Unit documentation]] must be associated with the structure function pointer members instead of the actual test method implementations.
| <tt>scl_test_cclass</tt>
|-
|
'''C functions'''

[[Test_Unit#Test_Unit_as_Group_of_Free_Functions|Example code]]

|
* Extremely simple syntax
|
* Parametrized test units are not supported
* No test unit support for constructor/initializer or destructor/de-initializer
* Changing test unit membership requires editing of the pragma statement
* [[Test_Documentation_in_C/C%2B%2B|Test Unit documentation]] for the FList must be associated with the header file - as such, you can only have one FList per header file if you want to document your test units.
| <tt>scl_test_flist</tt>
|}

The best choice is usually the C++ class since it offers the best mix of features and ease-of-use. (You can test code written in C or C++ using the C++ class test units.) However, compiling C++ is not always possible, in this case one of the C-based test unit packaging options must be used.

You can freely mix different deployment methods across a project if desired, the format of the results is consistent across all test unit packaging options.

== Test Method Return Values ==

Test Method return values must conform to certain return types as summarizing in the following table.

{| border="1" cellspacing="0" cellpadding="10" style="background-color:#ffffcc;font-family:arial;font-size:9pt;"
| '''Return Type'''
| '''Description'''
| '''How Return Value is Interpreted'''
| '''Default Status'''
|-
| <tt>void</tt>
| Most common return type
| Since there is no return, PASS/FAIL status is set in the body of the method using a
* [[Test_Macros|Test Macro]]
or a runtime call
* [[Runtime_Test_Services#srTestCaseSetStatus|srTestCaseSetStatus()]]
* [[Runtime_Test_Services#srTestCaseSetStatusEx|srTestCaseSetStatusEx()]]
* [[Runtime_Test_Services#method_SetStatus|srTestCase::SetStatus()]]
|
* If the method runs to completion without another status being set, the status is srTEST_PASS
* If the method does not run to completion (due to crash or hang in software under test), the status is srTEST_INPROGRESS
|-
| integer type
| Integer types include:
* <tt>int</tt>
* <tt>short</tt>
* <tt>char</tt>
* <tt>long</tt>
''may include signed/unsigned/const qualifiers''
|
* If you explicitly set the status in the method (using a test macro or a runtime call), this status will override the status that would otherwise be set by the return value
* If you don't explicitly set the status in the method, the return value determines the status as follows:
** Return is '''zero''' -> srTEST_PASS
** Return is '''non-zero''' -> srTEST_FAIL
|
* If the method runs to completion, the status is determined by the rules to the left (there is no default)
* If the method does not run to completion (due to crash or hang in software under test), the status is srTEST_INPROGRESS
|-
| <tt>bool</tt>
| c++ only
|
* If you explicitly set the status in the method (using a test macro or a runtime call), this status will override the status that would otherwise be set by the return value
* If you don't explicitly set the status in the method, the return value determines the status as follows:
** Return is '''true''' -> srTEST_PASS
** Return is '''false''' -> srTEST_FAIL
|
* If the method runs to completion, the status is determined by the rules to the left (there is no default)
* If the method does not run to completion (due to crash or hang in software under test), the status is srTEST_INPROGRESS
|}

== Examples ==

Following are a few short examples. In each example, a single test unit with the name "MyTest" is identified to the Stride compiler via a [[Test Pragmas]].

=== C++ Class ===

'''MyTest.h'''
<source lang=cpp>
#include <srtest.h>

class MyTest : public stride::srTest
{
public:
void ExpectPass()
{
srNOTE_INFO("this test should pass");
srEXPECT_EQ(2 + 2, 4);
}
void ExpectFail()
{
srNOTE_INFO("this test should fail");
srEXPECT_GT(2 * 3, 7);
}
};

#ifdef _SCL
// this pragma identifies MyTest as a test class to the STRIDE compiler
#pragma scl_test_class(MyTest)
#endif
</source>

=== C Class ===

'''MyTest.h'''

<source lang=c>
#include <srtest.h>

typedef struct MyTest
{
void (*ExpectPass)(struct MyTest* self);
void (*ExpectFail)(struct MyTest* self);
} MyTest;

void MyTest_Init(MyTest* self);

#ifdef _SCL
// This pragma identifies MyTest as a test c class to the STRIDE compiler.
// Extra instrumentation code will be generated to call MyTest_Init() before
// tests are run.
#pragma scl_test_cclass(MyTest, MyTest_Init)
#endif
</source>

'''MyTest.c'''

<source lang='c'>
#include "MyTest.h"

static void ExpectPass(MyTest* self)
{
srNOTE_INFO("this test should pass");
srEXPECT_EQ(2 + 2, 4);
}
static void ExpectFail(MyTest* self)
{
srNOTE_INFO("this test should fail");
srEXPECT_GT(2 * 3, 7);
}
void MyTest_Init(MyTest* self)
{
self->ExpectPass = ExpectPass;
self->ExpectFail = ExpectFail;
}
</source>

=== Free Functions ===

'''MyTest.h'''

<source lang=c>
#include <srtest.h>
void ExpectPass();
void ExpectFail();

#ifdef _SCL
// this pragma identifies MyTest as a test unit to the STRIDE compiler, specifying
// the four functions as members of the test unit
#pragma scl_test_flist("MyTest", ExpectPass, ExpectFail, ChangeMyName, ChangeMyDescription)
#endif
</source>

'''MyTest.c'''

<source lang='c'>
#include "MyTest.h"

void ExpectPass()
{
srNOTE_INFO("this test should pass");
srEXPECT_EQ(2 + 2, 4);
}
void ExpectFail()
{
srNOTE_INFO("this test should fail");
srEXPECT_GT(2 * 3, 7);
}
</source>

== C++ Only Features ==

=== Use Operator << to Augment Test Macros ===

In C++ test code [[Test Point]], [[Test Log]] and [[Test Macros]] macros support adding to the annotation using the << operator. For example:

<source lang="cpp">
srEXPECT_TRUE(a != b) << "My custom message" << " with more data " << 1234;
</source>

As delivered, the macros will support stream input annotations for:
* all numeric types,
* C string (char* or wchar_t*), and
* types allowing implicit cast to numeric type or "C" string.

=== Overloading the << operator ===

You can also overload the << operator in order to annotate reports using your own custom type. An example is below.

The following will compile and execute successfully given that the << operator is overloaded as shown:

<source lang="cpp">
#include <srtest.h>

// MyCustomClass implementation
class MyCustomClass
{
public:
MyCustomClass(int i) : m_int(i) {}

private:
int m_int;
friend stride::Message& operator<<(stride::Message& ss, const MyCustomClass& obj);
};

stride::Message& operator<<(stride::Message& ss, const MyCustomClass& obj)
{
ss << obj.m_int;
return ss;
}

void test()
{
MyCustomClass custom(34);

srEXPECT_FALSE(true) << custom;
}
</source>

Frequently Asked Questions About STRIDE

2015-07-07T02:39:50Z

Marku:

__NOTOC__
== Types of Testing Supported ==
One of the questions that should be asked is ''what is the '''value''' of the test?'' If the test does not discover any defects or does not provide ongoing regression, then the value is questionable. Also ''what is the '''effort''' in implementing the test?'' Stride has been uniquely designed to support maximizing the '' '''value''' '' of the test while minimizing the '' '''effort''' '' to implement it.

Stride supports three general types of testing:
* '''Unit Testing'''
* '''API Testing'''
* '''Integration Testing'''

=== Unit Testing ===
'''Unit Testing''' is supported following the model found in typical [http://en.wikipedia.org/wiki/XUnit xUnit-style] testing frameworks.

Traditional '''Unit Testing''' presents a number of challenges in testing embedded software:

* Testing functions/classes in '''isolation''' requires a lot of extra work, especially if your software was not designed upfront for testability
* The software is often not well suited for '''others''' to participate in the test implementation, since there is too much internal knowledge required to be productive
* It can be difficult to automate execution of the full set of tests on the real target device

'''Unit Testing''' legacy software may have limited value, particularly if the software is stable with respect to defects. The best ''return-of-effort'' is often experienced when focused on '''brand new''' software components.

=== API Testing ===
Stride supports '''API Testing''' by leveraging the same techniques available for Unit Testing.

'''API Testing''' differs from unit testing in that the tests focus on direct testing (calling) of a well-defined interface.

* The design of ''public interfaces'' often lends itself to testing in isolation ''without'' implementing special test logic (i.e. no stubbing required), which make the test implementation simpler.
* Public APIs are most likely documented and as a result, ''non domain experts'' can more easily participate in the test implementation

Although '''API Testing''' often represents a smaller percentage of the software being exercised, this kind of testing is typically well understood, easy to scope, and often has a better ''return-on-effort''.

=== Integration Testing ===
Stride also supports '''Integration Testing''', which is different than Unit Testing or API Testing in that it does not focus simply on calling functions and validating return values. To learn more about some of the unique testing techniques well suited for this type of testing [[Expectations | '''read here''']].

'''Integration Testing''' focuses on validating a larger scope of the software while executing under normal operating conditions.

* Tests are performed typically with fully functional software build
* There are minimal code isolation challenges
* Test results provide a sanity check on the health the software

We believe that '''Integration Testing''' has a very high ''return-on-effort'' and is more applicable to legacy software systems.

== Getting Started ==

=== How long does it take to install STRIDE? ===

==== Off-Target Installation ====
Off-target installation to a desktop or laptop PC takes just minutes; support is offered for Windows and Linux.

==== On-Target Installation ====
For standard embedded platforms such as Linux, Android and Windows the installation process varies between a few hours to a couple of days. We provide SDK packages that provide a reference to integrators.

For proprietary embedded targets, a custom [[Platform_Abstraction_Layer|Platform Abstraction Layer (PAL)]] is required. The PAL provides the glue between the [[STRIDE Runtime | Stride Runtime]] and services offered by the OS. It is the only piece of the runtime that is customized between operating systems. The implementation of the PAL ranges between a day to a week depending on the complexity of the OS. There is also a [[Build Integration | build integration]] step. This involves integrating the [[STRIDE Build Tools | Stride Build Tools]] into your software make process. The activity ranges from a single day to several days.

=== What kind of training is required? ===

Our training [[Training Overview | approach]] is based on wiki articles, samples, and leveraging the [[Stride Sandbox]. The training has been set up for self-guided instruction that can be leveraged for an initial introduction of the technology and on-demand for specific topics when required.

== Integration with Stride ==

=== What is the size of the Stride Runtime? ===

The [[STRIDE Runtime |Runtime]] is a source package that supports connectivity with the host system and provides[[Runtime_Test_Services | services for testing]] and [[Test Macros]]. The ''Runtime'' is tailored specifically to embedded applications, overhead is minimal. It consumes very little memory for table and control block storage. Resource usage is [[STRIDE_Runtime#Runtime_Configuration | configurable]] and can be tailored to the limitations of the target platform.

{| class="wikitable"
|+ Typical resource usage
! Aspect !! Resources
|-
| Code Space || About 90-130 KB depending on the compiler of use and the level of optimization.
|-
| Memory Usage || Configurable, by default set to about 10 KB
|-
| Threads || 3 Threads; configurable priority; blocked when inactive
|}

=== What is the processing overhead? ===

The [[STRIDE Runtime | Stride Runtime]] overhead is minimized by collecting raw data on the target, and transferring information at a low priority task to the host for processing. Processing is only active when executing tests via the [[Stride Runner]].

== Testing ==

=== What languages are supported? ===

Tests can be written using '''C''', '''C++''', and '''Perl'''.

=== Is there any alternative to running Stride tests with a real device? ===

Yes. Tests can be also be built and executed using the [[Stride Sandbox]]. In order for this to work, your device source code must be built along with the test code using the host's desktop toolchain (MSVC on Windows, gcc on Linux).

== Test Automation ==

=== What is continuous integration and why should I care? ===

The key principle of continuous integration is regular testing of your software--ideally done in an automated fashion. Stride tests are reusable and automated. Over time, these tests accumulate, providing more and more comprehensive coverage. By automating the execution of tests and results publication via [http://www.testspace.com Testspace] with every software build, development teams gain immediate feedback on defects and the health of their software. By detecting and repairing defects immediately, the expense and time involved with correcting bugs is minimized.

=== Does Stride support continuous integration? ===

Yes. The [[Stride Runner]] provides a straightforward means to connect to the device under test and execute the test cases you've implemented using Stride. The runner allows you to configure which tests to run and how to organize the results using sub-suites in the report. Since the runner supports an option-based command line interface, this tool is easy to integrate with typical continuous integration servers.

=== Where/how do you store test results? ===

When you execute your tests using the [[Stride Runner]], upon completion the results are written to an xml file. This xml file uses a custom schema for representing the hierarchy of results (suites, cases, etc.). These files also include a stylsheet specification (which will be written to the same directory as the xml file) that allows them to be viewed as HTML in a browser. You are free to store these files for future use/reference. The runner '''also''' supports direct publishing to [http://www.testspace.com Testspace] which is a hosted web application for viewing and collaborating on your test results. Once you are regularly executing tests, whether automatically or manually, we recommend you use the publish feature to persist and share your test results.

=== Can I get email containing test reports? ===

Yes. If you use [http://www.testspace.com Testspace] to store your results, you can optionally configure your test space(s) to automatically notify users when new results are uploaded. The email that is generated by Test Space contains only summary information and provides links so that you can view the complete report data.

If you are using a continuous integration server to initiate your testing, it's likely that it supports different forms of notification when the testing is complete, so it's often possible to attach the xml report data as part of the CI server notification.

== Source Instrumentation ==

=== What are the advantages of Test Points over logging? ===

[[Test Point | Test Points]] can be used for direct validation during real-time execution. Logging systems, while often useful for historical record, can only be used for post-processing validation at best. What's more, since there is no standard way to post-process logs, users often rely on manual inspection or non-scalable homegrown solutions for validation. Test Point validation is fully automated using the STRIDE Framework, which provides robust harnessing and reporting. Test Points also can include optional data payloads which can be very useful for validating state when a point is hit. Test Points have potentially lower impact on the system than standard logging since they '''(1)''' are only sent from the origin if a test is actively subscribed to test point (labels are used as a filter at runtime) and '''(2)''' test points can be completely disabled (no-opped) in your build by undefining a single preprocessor macro [[Runtime_Integration#STRIDE_Feature_Control | (STRIDE_ENABLED)]].

=== What about source instrumentation bloat? ===

Any mature code base has some level of diagnostic bloat to it. This often takes the form of ad-hoc logging or debug statements. With Stride [[Test Point | Test Points]] and [[Test Log | Test Logs]], you open your software to better automated test scenarios. All Stride instrumentation takes the form of single line macros, so the amount of instrumentation bloat is no worse than other typical ad-hoc diagnostics. What's more, the Stride macros are all designed to be quickly no-opped in a build via a single preprocessor macro, making it possible to completely eliminate any actual impact on certain builds, if so desired.

=== Are all Test Points active? ===

No. Under normal testing scenarios, only the specific test points that are needed for a test are actually broadcast through the system. We accomplish this by setting test point filters (by label) on the system whenever one of the Test Point setup functions is called (in script or native code). These filters are reset or removed at the end of the test, so in general ''none'' of the test points are actually sent through the system if no test is currently active. That said, there are a few special use cases in which ''all'' test points become active in the system - namely, when tracing is activated on the host runner and when a specific test case uses a set that include ''TEST_POINT_EVERYTHING_ELSE''. In general, however, the test points that are actually sent from the system are ''only'' those that are needed to execute the behavior validation for the current test.

=== Will it affect performance? ===

Our experience on a wide-range of systems has shown minimal impact from the Stride instrumentation. The Stride Runtime has been designed to be small, portable, and readily configurable, allowing it to be optimized for the platform's specific characteristics.

=== Should I leave Test Points in? ===

Yes. Once you have some behavior tests written, it's worthwhile to maintain that instrumentation and the corresponding tests, which allows you to run the tests on any stride-enabled build. All of the instrumentation macros are easily no-opped via a single preprocessor flag, so you can choose to effectively remove the instrumentation code on select builds (production/release builds, for example). The ultimate value of instrumentation is the continuous feedback you get by regularly executing the automated tests on the build.

Expectations

2015-07-07T02:25:51Z

Marku:

__NOTOC__
Stride supports '''Behavior Testing''', which is different than Unit Testing or API Testing in that it does not focus on calling functions and validating return values. Behavior Testing validates the ''expected sequencing and state of the software'' executing under normal operating conditions. In order to begin this type of testing, you must '''(1)''' instrument the source under test with [[Test Point | Test Points]] and '''(2)''' define the ''expectations'' of the Test Points for each test scenario.

The '''Validation Model''' is based on validating that the ''Expected List'' of Test Points have been hit as well as optionally validating any ''state data'' associated with a Test Point member. Through the use of an ''Unexpected List'', it's possible to simulataneously validate that specified Test Points are NOT hit during the execution of the scenario.

The ''Expected List'' and ''Unexpected List'' declared for a specific testing scenario define the ''active'' set of Test Points for the system. All other Test Points within the software are not active and have nominal impact.

= Expected List =

The first step in defining '''Expectations''' is describing the set of Test Points expected to be hit during a test scenario. This is the '''list''' of Test Points and the expected number of hits ('''count''') for each of the Test Points. Any expected '''state data''' associated with a Test Point that requires validation should be included.

The validation of the ''Expected List'' is done when any of the following conditions are met:
* The ''Expected List'' sequencing has been met completely
* The time allocated for validation has expired
* State data associated with a Test Point is declared invalid
* An out-of-sequence Test Point has been encountered (a concrete failure is identified)
* An [[Expectations#Unexpected_List | Unexpected Test Point]] has been encountered

'''Expected TEST POINTS list:'''

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Label'''
| '''Count'''
| '''Expected State Data'''
|-
| Name 1
| 1
| ''<describe data payload validation requirements if applicable>''
|-
| Name 2
| 1 +
| ''<describe data payload validation requirements if applicable>''
|-
| ''name ...''
| ''n''
| ''<...>''
|}

=== Sequencing Properties ===

The ''Expected List'' requires defining ''sequencing properties'' used for the validation. This involves establishing how to process the '''ordering''' of the Test Points being hit, how '''strict''' to handle duplications of Test Point members, and if to '''continue''' capturing Test Points for the entire time allocated for the test scenario even though the ''Expected List'' sequencing requirements have been met.

'''Expected TEST POINTS sequencing properties:'''

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Properties'''
| ''' Description'''
|-
| ''Ordered''
| Test Points are expected to be hit in the exact order defined in the list
|-
| ''Unordered''
| Test Points can be hit in any order defined in the list
|-
| ''Strict''
| Test Points specified in the list must match the exact count (i.e. no duplication)
|-
| ''Non-Strict''
| Test Points specified in the list can be duplicated anywhere in the sequence
|-
| ''Continue''
| Test Points validation will continue even though the ''Expected List'' has been met ''so far''
|}

=== State Data Validation ===

State data validation for a Test Point member is optional. The ''Expected List'' can associate a ''predicate''<ref name="predicate">Defined as a necessary condition of the Test Point being valid -- logic within the predicate determines if ''valid'', ''invalid'', or ''ignored'' </ref> as a necessary condition for validation of a Test Point member when being processed. The predicate provides an extra level of validation (beyond sequencing) focusing on the state data associated with the Test Point.

The predicate can declare the Test Point:
* Valid - successful, thus the expected count is decremented
* Invalid - the expectations is NOT met, indicating failure
* Ignored - no affect on the validation, continue to wait on the current Test Point

= Unexpected List =

An ''Unexpected List'' of Test Points can optionally be defined. This list works in conjunction with the [[Expectations#Expected_List | ''Expected List'']]. The list defines a set of Test Points that are to be treated as failures if any one of them is encountered (hit) during a testing scenario.

This list can be made up of the '' '''complement''' '' of the ''Expected List'', thus including ''all'' Test Points not included in that defined set. This type of setting requires enabling all Test Points defined in the software during the execution of the test.

If the ''Unexpected List'' members intersect with the ''Expected List'' members, that is considered an input error.

= Special Processing =

=== Members ===
There are two ''special members'' that can be used for the ''Expected List'' called '''ANY IN SET''' and '''ANY AT ALL'''. These special member values are used to customize the success criteria (i.e. trigger condition) for the entry defined within the ''Expected List''. Both types require a predicate<ref name="predicate"/> to determine if the Test Point is ''valid, invalid, or ignored''. The '''ANY IN SET''' refers to the set of points defined by the ''Expected List'' while the '''ANY AT ALL''' refers to all Test Points in the system. By definition the '''ANY AT ALL''' member requires enabling all Test Points defined in the software during the execution of the test.

Concerning an '''Unordered''' ''Expected List'', only one special member can be used. It will be processed first independent of its location within the list. The '''Count''' attribute is also optionally available to be used with the special members, indicating the number of valid returns from the predicate until the next Test Point in the list is validated.

There is one ''special member'' that can be used for the ''Unexpected List'' called '''EVERYTHING ELSE'''. This special member can only be used as a '''single member only''' within an ''Unexpected List''. It takes the ''complement'' of the set of Test Points defined within the ''Expected List''. This requires enabling all Test Points defined in the software during the execution of the test. Also, by definition any non-expected Test Point hit during the test would be deemed a failure.

=== Counts ===
There is one ''special count'' value that can be associated with a Test Point defined in the ''Expected List'' called '''ANY COUNT'''. This special count is used to focus on capturing a variable number of Test Point hits -- 0 or more. In essences the Test Point has no consequences of the Expected List sequencing, except if a predicate<ref name="predicate"/> associated with the Test Point returns ''invalid''.

<hr>

= Use Cases =

The following use cases provide examples of defining '''Expectations''' based on different testing scenarios.

=== Sequencing Properties ===
Use case examples for validating the [[Expectations#Sequencing_Properties | sequencing]] of Test Points.

Expecting an exact ordered sequence of Test Points to be hit
''Expect'' = [A,B,C,D]; ''ORDERED, STRICT''
Hits = [A,B,C,D] - PASS
Hits = [A,B,D,C] - FAIL (D hit before C)

Expecting an exact ordered of Test Points but with duplicates
''Expect'' = [A,B,C,D]; ''ORDERED, STRICT''
Hits = [A,B,B,C,D] - FAIL (B duplicated with strict defined)

''Expect'' = [A,B,C,D]; ''ORDERED, NON-STRICT''
Hits = [A,B,B,C,D] - PASS
Hits = [A,C,B,C,D] - PASS
Hits = [A,B,C,B,D] - PASS

Expecting a set of Test Points to be hit, but the order does not matter
''Expect'' = [A,B,C,D]; ''UNORDERED, STRICT''
Hits = [B,A,C,D] - PASS
Hits = [B,A,C,B,D] - FAIL (B hit twice)

''Expect'' = [A,B,C,D]; ''UNORDERED, NON-STRICT''
Hits = [B,A,C,B,D] - PASS

Expecting a set of Test Points to be hit, but order and duplications does not matter
''Expect'' = [A,B,C,D]; ''UNORDERED, NON-STRICT''
Hits = [B,A,C,B,D] - PASS
Hits = [B,A,A,A,D] - FAIL (C never hit)

Expecting an exact ordered sequence of Test Points to be hit, but checking for any ''extra'' hits for total duration of test scenario
''Expect'' = [A,B,C,D]; ''ORDERED, STRICT'', ''CONTINUE''
Hits = [A,B,C,D] - PASS
Hits = [A,B,C,D,A] - FAIL (A hit after valid sequence met based on test timeout)

=== Count Attribute ===
Use case examples that leverage the [[Expectations#Count | Count]] attribute associated with a list member.

Expecting a sequence of Test Points, including multiple hits for 2 members
''Expect'' = [A,B:3,C,D:2]; ''ORDERED, STRICT''
Hits = [A,B,B,B,C,D,D] - PASS
Hits = [A,B,B,C,B,D,D] - FAIL (B not hit three times before C)

''Expect'' = [A,B:3,C,D:2]; ''ORDERED, NON-STRICT''
Hits = [A,B,B,B,C,B,D,D] - PASS

Expecting a set of Test Points being hit, including multiple hits (with duplications), but order does not matter
''Expect'' = [A,B:3,C,D]; ''UNORDERED, STRICT''
Hits = [B,A,B,C,B,D] - PASS
Hits = [B,A,B,B,C,B,D] - FAIL (B duplicated on 4th hit)

''Expect'' = [A,B:3,C,D]; ''UNORDERED, NON-STRICT''
Hits = [B,A,B,B,C,B,D] - PASS

=== State Data Validation ===
Use case examples leveraging predicates<ref name="predicate"/> to [[Expectations#State_Data_Validation | validate state data]].

Simple example showing associating a predicate with Test Point named ''A''.
''Expect'' = [A:p1(tp),B,C,D]; ''ORDERED, STRICT''
p1(tp) { return VALID }
Hits = [A,B,C,D] - PASS

Expecting Test Point ''B'' to be hit 3 times
''Expect'' = [A,B:3:p1(tp),C,D]; ''UNORDERED, STRICT''
p1(tp) { return VALID unless 3rd time called than return INVALID}
Hits = [B,A,B,C,B,D] - FAIL (predicate returns invalid on 3rd B hit)

=== Special Members ===
Use case examples using the two [[Expectations#Members | special members]] within an ''Expected List''.

Waiting on a single trigger (D) to start the processing of an Expected List
''Expect'' = [ANY_IN_SET:p1(tp),A,B,C:2,D]; ''ORDERED, STRICT''
p1(tp) {return VALID when D hit otherwise IGNORE}
Hits = [A,B,A,D*,A,B,C,C,D] - PASS

Waiting on Test Point ''outside'' of the Expected List to trigger processing
''Expect'' = [ANY_AT_ALL:p1(tp),A,B,C:2,D]; ''ORDERED, STRICT''
p1(tp) {return VALID when x hit}
Hits = [A,B,A,x*,A,B,C,C,D] - PASS

Waiting on a multiple triggers - 1st trigger used to start processing and 2nd used midstream
''Expect'' = [ANY_IN_SET:p1(tp),A,B,ANY_IN_SET:p2(tp),C:2,D,E]; ''ORDERED, STRICT''
p1(tp) {return VALID when D hit}
p2(tp) {return VALID when E hit}
Hits = [A,B,A,D*,A,B,A,A,B,E*,C,C,D,E] - PASS

''Expect'' = [ANY_IN_SET:p1(tp),A,B,C:2,D]; ''UNORDERED, NON-STRICT''
p1(tp) {return VALID when D hit}
Hits = [A,B,A,D*,A,A,A,D,A,C,B,C,D] - PASS

This test uses the ANY AT ALL along with a fix count of 3
''Expect'' = [ANY_AT_ALL:p1(tp):3,A,B,C]; ''ORDERED, STRICT''
p1(tp) {return VALID}
Hits = [A,z,x,A,B,C] - PASS

=== Special Counts ===
Use case examples leveraging the [[Expectations#Special_Counts | special count attributes]] within an ''Expected List''.

Expecting between 0 and more Test Point As to be hit before processing the remaining members.
''Expect'' = [A:ANY_COUNT,B,C,D]; ''ORDERED, STRICT''
Hits = [B,C,D] - PASS
Hits = [A,A,B,C,D] - PASS

This test passes ''immediately'' after the A is hit (example of a weird expectation)
''Expect'' = [A,B:ANY_COUNT]; ''ORDERED, STRICT''
Hits = [A,..] - PASS

This test also passes, but will process all Test Point B hits until the test time expires
''Expect'' = [A,B:ANY_COUNT]; ''ORDERED, STRICT, CONTINUE''
Hits = [A,B,B,B] - PASS (the wait timeout will complete the validation)
Hits = [A] - PASS

This test fails because all Test Points are checked once the test time expires
''Expect'' = [A]; ''ORDERED, STRICT, CONTINUE''
Hits = [A,A] - FAILS (the wait timeout than will complete the validation)

This test validates a set of Test Points focusing exclusively on the associated ''state data''
''Expect'' = [A:p1(tp):ANY_COUNT, B:p2(tp):ANY_COUNT, C:p2(tp):ANY_COUNT]; ''UNORDERED, CONTINUE'' (STRICT NA)
Hits = [A,A,B,A,C,C,C,A,..] - PASS (unless any of the predicates fails)
Hits = [A,B,C,D] - PASS

=== Miscellaneous ===
The following are miscellaneous use cases.

Expecting an exact ordered sequence of Test Points to be hit, but verifying certain Test Points are NOT hit
''Expect'' = [A,B,C,D]; ''ORDERED, STRICT''
''Unexpect'' = [x,y]
Hits = [A,B,C,D] - PASS
Hits = [A,B,B,C,D] - FAIL
Hits = [A,B,C,y,..] - FAIL

This test will always pass and log every Test Point hit during test time duration. Note all Test Points are active.
''Expect'' = [ANY_AT_ALL:p1(tp):ANY_COUNT]; ''CONTINUE {other sequencing properties NA}''
p1(tp) {return VALID}
Hits = [A,A,B,x,y,D, ..] - PASS (the wait timeout will complete the validation)

This test will log every Test Point hit during test, looking for X and Y as failures. Note all Test Points are active.
''Expect'' = [ANY_AT_ALL:p1(tp):ANY_COUNT]; ''CONTINUE {other sequencing properties NA}''
p1(tp) {return VALID}
''Unexpect'' = [X,Y]
Hits = [A,A,B,x,y,D,Y] - FAIL (last Test Point hit causes a failure)

This test will fail on any Test Point that is hit. Note all Test Points are active. Test will last based on time duration or the 1st Test Point hit.
''Expect'' = NULL; ''CONTINUE {other sequencing properties NA}''
''Unexpect'' = [EVERYTHING_ELSE]
Hits = [A,..] - FAIL

This test will ignore any Test Point hit during test, looking for X and Y as failures.
''Expect'' = NULL; ''CONTINUE {other sequencing properties NA}''
''Unexpect'' = [X,Y]
Hits = [..,X,..] - FAIL

This test will fail on any Test Points hit except for A & B. Note all Test Points are active. Test will last until failure or timeout.
''Expect'' = [A,B]; ''UNORDERED, NON-STRICT, CONTINUE''
''Unexpect'' = [EVERYTHING_ELSE]
Hits = [..,A,..,X] - FAIL

== Notes ==
<references/>

Stride Overview

2015-07-07T02:22:02Z

Marku:

__NOTOC__
'''Stride™''' is a [http://en.wikipedia.org/wiki/XUnit xUnit-style] test framework for C/C++ software comprised of an external test ''Runner'' , ''Runtime'' library, and a set of ''Build Tools''. It was designed with a specific focus on enabling tests to be executed on a device using a fully functional build. This means that the software works the same as before, but also has built-in tests that can be invoked on-demand by the test ''Runner''. Stride has also been integrated with [http://www.testspace.com Testspace] for test content management.

Stride's unique host-target architecture allows easier and better On-Target/Device White-box testing. Software builds automatically become more testable; facilitating deeper code coverage opportunities.

[[image:Stride Overview.jpg | 600px | center | Stride block diagram]]

==== Runner ====
Stride includes a lightweight [[Stride_Runner | '''Runner''']] application that is a host-based command-line utility for interactive and automated test execution. It also integrates seamlessly with [http://www.testspace.com Testspace] for ''test content management''.
 

====Runtime====
Stride also contains a [[STRIDE Runtime | '''Runtime''']] software source package that supports connectivity with the host system. It is integrated with embedded application software to enable ''testability'' to be compiled into the software with minimal impact on performance or the size of the application. The software is agnostic to the RTOS, CPU, and transport. It can be configured to support multi-processes, multi-threads, user/kernel spaces, or a simpler single application process. Typically the runtime is configured as a background thread and only executes when running tests.

==== Build Tools ====
[[image:STRIDE 4.2 Framework-b.jpg |right|300px | border]]

The Stride [[STRIDE_Build_Tools | '''Build Tools''']] are a set of command-line utilities that integrate with your target software build process. They preprocess standard C/C++ header files that contain Stride test declarations to auto-generate source code comprising a custom [[Intercept_Module | '''Intercept Module''']]. The intercept module code is then built with the Stride Runtime to provide ''fixturing'' and ''harnessing'' test logic as part of your build image.

= Cross Platform =
Stride works on virtually any target platform. Stride's cross-platform framework facilitates a ''unified testing approach'' for all team members, enabling organizations to standardize on a test workflow that is independent of the target platform being used or what branch of software is being changed.

=== Runtime written in standard C ===
The Stride Runtime is written in standard C on top of a simple [[Platform_Abstraction_Layer | '''platform abstraction layer''']] that enables it to work across platforms. It can be configured for a single process multi-threading environment or multiple process environments (i.e. Linux, Windows, Embedded RTOS, etc.). It is delivered as source code to be included in the application's build system. The transport between the host and target is configurable and supports Serial and TCP/IP by default. Custom transports are also available. The runtime has also been tailored specifically to embedded applications, overhead is minimal. It consumes very little memory for table and control block storage. Resource usage is configurable and can be tailored to the limitations of the target platform.

===Integrates With Your Existing Build System ===
Stride also auto-generates ''intercept module'' used for harnessing and remote logic as source code during the make process, removing any dependencies on specific compilers and / or processors.

===Supports Off-Target Testing ===
Testing can also be conducted using a standard desktop machine (Windows, Linux, or FreeBSD). The Stride ''SDKs'' allow for a seamless transition between the real target and an Off-Target host environment.

= Rich set of Assertions =
Test cases are typically constructed leveraging a large set of available assertion macros. On failures macros provide such details as file name, line number, and why the condition failed. Stride provides are large set of optional macros available for automatic report annotation in the case of failures.

<source lang=cpp>
#include <mytest.h>

void MyTest::CheckFoo() {
..
srEXPECT_EQ(foo(2 + 2), 4);
}
void MyTest::CheckBoo() {
..
srEXPECT_GT(boo(3 * 3), 7);
}
</source>

= Extensive Parameter Support =

Parameterized testing is key to reducing test code duplication. Constructor arguments, name-value collections, and host file remoting are all supported. Stride provides an easy way for passing and accessing these types of parameters, allowing customization of test behavior at runtime.

Create an INI-type of file (i.e. myinput.ini)

Loopsize = 10
InputFile = ${MYPATH}/datacontent.bin
Toogle = On

Pass the name-value collection using the Stride Runner

$ stride .. --run="MyTest(/path/to/myinput.ini)"

Now simply access your input within test logic using built-in services

<source lang=cpp>
{
// .. doing stuff ..
GetParam("Loopsize", buffer, size);
//.. using parameters for test logic
}
</source>

= Test Doubles & Test Points =
Stride offers advanced testing techniques that enable deeper and more effective testing with less effort.

===Test Doubles===
For more advanced testing scenarios, dependencies can be '''doubled'''. This feature provides a means for intercepting C/C++ global functions on the target and substituting a stub, fake, or mock. The substitution is all controllable via the runtime, allowing the software to continue executing normally when not running a test.

For example something like the following could be useful:

<source lang=cpp>
void MySuite::test1()
{
// inserting my own version of malloc()
srDOUBLE_SET(malloc, my_malloc_stub);

// calling a routine that uses malloc(). checking handled null correctly
ret_code = FuncThatUsersMalloc(42);
srEXPECT_EQ(ret_code, -1)

// restoring the real malloc()
srDOUBLE_RESET(malloc)
}
</source>

===Test Points ===
Stride leverages ''source instrumentation'' to provide an additional validation technique called '''Expectation Testing''' that can be applied to the executing software application. The execution sequencing of the code, along with state data, can be automatically validated based on ''what is expected''. This validation technique does not focus on calling functions / methods but rather verifies ''code sequencing''. This can span threads, process boundaries, and even multiple targets, as the application(s) is running. Leveraging simple macros -- called [[Test_Point | '''Test Points''']] -- developers strategically instrument the source under test.

<source lang='c'>

/* a test point with formatted string payload */
srTEST_POINT_STR("IMPORTANT", "important date= %d", myVar);

</source>

The test validation is done without impacting the application's performance (the on-target test code is executed in a background thread and scripts are executed on the host machine). When failures do occur, context is provided with the file name and associated line number of the failed expectations. This type of validation can be applied to a wide-range of testing scenarios:
* State Machines
* Data flow through system components
* Sequencing between threads
* Drivers that don't return values
* and much more ...

The following is a snippet of what the test code might look like
<source = lang='c'>

srTestPointExpect_t expected[]={
{"IMPORTANT"}, myCheckData,
{"ANOTHER_TP"},
{0}};

// setup the expections
srTestPointSetup(..);

// start the operations

srTestPointWait(handle, 1000);

</source>

= Test Implementation =
Stride based tests are integrated with the existing build system enabling your software to be both fully ''functional'' and ''testable'' at the same time. The ''test logic'' is separated from the application source code and is '''not''' executed unless invoked via the Stride Runner. Any ''source instrumentation'' is only active when executing tests. The impact of built-in testability to the software application is nominal.

The test validation can be implemented in both ''C/C++'' on the target and ''Perl'' on the host.

'''C/C++ Test Unit Example'''
<source lang=cpp>
#include <srtest.h>

class MyTest {
public:
void CheckFoo();
void CheckBoo();
};
#pragma scl_test_class(MyTest)
</source>

<source lang=cpp>
#include <mytest.h>

void MyTest::CheckFoo() {
..
srEXPECT_EQ(foo(2 + 2), 4);
}
void MyTest::CheckBoo() {
..
srEXPECT_GT(boo(3 * 3), 7);
}
</source>

'''Perl Test Script Example'''
<source lang="perl">
use strict;
use warnings;

package MyTests;
use base qw(STRIDE::Test);
use STRIDE::Test;

sub CheckFoo : Test
{
EXPECT_EQ(Remote->foo(2 + 2), 4);
}

sub CheckBoo : Test
{
EXPECT_GT(Remote->boo(3 * 3), 7);
}

1;
</source>

= Works with Testspace =

Stride have been designed to leverage Testspace features such as test design, execution control, and test analysis. Managing test content and status become better organized and easier to assess current progress. Refer to [http://www.testspace.com Testspace] website for more details.

<hr/>


'''For more details refer to our [[Frequently Asked Questions About STRIDE | FAQ]] '''

Training Perl

2015-07-06T21:52:06Z

Marku:

The '''Perl''' training focuses on ''Test Points'' validation with limited ''Remoting'' for controlling the software under test.

''A few words of caution''. Using Perl for testing when applied to the right situation can be a very compelling. At the same time there are challenges to be considered. A script executed by Stride runs on the host .. '''not''' on the actual device. Because of this certain limits should be assessed. For example, although Stride supports remoting of global functions (i) ''how to qualify a function signature using pragmas (i.e. out pointers)'' and (ii) ''how to access these types of fields using Perl'' can be very challenging.

This sample demonstrates the advantages of host based scripting ''without'' the concerns mentioned above. The remoting used are for the following routines:

void sut_start_thread(void);
void sut_stop_thread(void);

Example option file ''my.opt'' used below for running

--database "%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb"
--device TCP:localhost:8000

Some of the advantages that will be shown:
* '''test coverage''' can be '''expanded''' without any changes to the target build
* nominal '''test code''' space requirements for software under test
* seamless '''integration''' of test results from both '''target-based tests''' and '''script tests'''

== Test Points ==
This test suite focuses on ''Test Points'' and how to validate them using a Perl script module.

The following articles are related to this example:
* Presentation of a [[Expectations | validation]] technique based on ''code sequencing'' and ''state data''
* [[Test Point]] definition
* Review of [[ Perl_Script_APIs#STRIDE::TestPoint | expectation tables and predicates]] using Perl

The Test Script is called '''testpoints.pm''' and is implemented in: '''testpoints.pm'''. The comments and descriptions are contained in the header blocks using POD. Three test cases (methods) are already implemented and one test method that can be used to make changes to is called '''TryStuff'''. Currently the ''TryStuff'' test method is set to ''not in use''.

First thing to do is run the ''TestPoints'' Test Script. Start the '''TestApp.exe''' and use the following command:

stride --options_file my.opt --run "..\training\testpoints.pm''

You can take a look at the results by opening the generated '''stride.html''' in your browser.

Consider the following for '''TryStuff''':
* Write a test to validate a subset of Test Points
* Write a test to wait on a trigger '''TRIGGER''' and then validate "A", "C", and "G"

Training Advanced

2015-07-06T21:49:03Z

Marku:

The '''Advanced''' training focuses on using ''Test Doubles'', ''Test Points'', and ''Fixtures''.

The ''Test Doubles'' will demonstrate how global functions can be dynamically intercepted and replaced with test logic (stub, fake, mock). The following functions contained in ''software_under_test'' will be used to demonstrate how to leverage interception when testing:

int sut_foo(int x);
int sut_boo(int x);
int sut_2xboo(int x);
int sut_strcheck(const char *s);

''Test Points'' enable a different style of testing using source instrumentation. This is especially applicable when normal API testing is not practical. The following ''sequencing'' functions will be called that issue Test Points. Note that the sequencing routines are very simple, keeping the focus on the technique that can be applied.

void sut_Sequence1(void);
void sut_Sequence2(void);
void sut_Sequence3(void);

''Fixturing'' shows how setup and teardown routines per test method can be incorporated within a Test Unit. The following functions will be used to demonstrate fixturing:

void sut_start_thread(void);
void sut_stop_thread(void);

Example option file ''my.opt'' used below for running

--database "%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb"
--device TCP:localhost:8000

== Test Doubles ==

This Test Unit focuses on leveraging '''Test Doubles''' in the context of executing a test.

The following articles are related to this example:

* Using [[Test Double | Test Doubles]]
** ''Definition'' verses ''Reference''
** ''Explicit'' verses ''Implicit''
** Setting and Resetting the Double implementation
* How to apply pragmas for [[Scl function | function intercepting]]

The Test Unit is called '''TestDoubles''' and is implemented in two source files: '''testdoubles.cpp''' and '''testdoubles.h'''. The comments and descriptions are contained in the header file. Three test cases (methods) are already implemented and one test method that can be used to make changes to is called '''TryStuff'''. Currently the ''TryStuff'' test method is set to ''not in use''.

First thing to do is run the ''TestDoubles'' Test Unit. Start the '''TestApp.exe''' and use the following command:

stride --options_file my.opt --run "TestDoubles"

You can take a look at the results by opening the generated '''stride.html''' in your browser.

Consider the following for '''TryStuff:'''
* Create your own stub for foo() and intercept it
** confirm it returns what you expect
* Double sut_boo()
** call sut_2xboo() with a number
** confirm within the test double (using a [[Test Macros | Test Macro]]) the expected passed in number (mocking example)

== Test Points ==
This Test Unit focuses on '''Test Points''' and how to validate them.

The following articles are related to this example:
* Presentation of a [[Expectations | validation]] technique based on ''code sequencing'' and ''state data''
* Overview of a [[Test Point]]
* [[Test_Point_Testing_in_C/C%2B%2B | Writing Tests]] leveraging Test Points

The Test Unit is called '''TestPoints''' and is implemented in two source files: '''testpoints.cpp''' and '''testpoints.h'''. The comments and descriptions are contained in the header file. Three test cases (methods) are already implemented and one test method that can be used to make changes to is called '''TryStuff'''. Currently the ''TryStuff'' test method is set to ''not in use''.

First thing to do is run the ''TestPoints'' Test Unit. Start the '''TestApp.exe''' and use the following command:

stride --options_file my.opt --run "TestPoints"

You can take a look at the results by opening the generated '''stride.html''' in your browser.

Consider the following for '''TryStuff:'''
* Write a test to validate a subset of Test Points
* Write a test to validate "E", "F", and "G", and a predicate to verify data associated with "E"

== Fixtures ==
This Test Unit focuses on using Fixutures in the context of an executing Test Unit.

The Test Unit is called '''TestFixtures''' and is implemented in two source files: '''testfixtures.cpp''' and '''testfixtures.h'''. The comments and descriptions are contained in the header file. One test cases (methods) is already implemented and one test method that can be used to make changes to is called '''TryStuff'''. Currently the ''TryStuff'' test method is set to ''not in use''.

First thing to do is run the ''TestFixtures'' Test Unit. Start the '''TestApp.exe''' and use the following command:

stride --options_file my.opt --run "TestFixtures"

You can take a look at the results by opening the generated '''stride.html''' in your browser.

Consider the following for '''TryStuff:'''
* Write a test to validate "A", "B", and "C" using the "TRIGGER"

Training Basics

2015-07-06T21:37:35Z

Marku:

The '''Basics''' training focuses on using ''Test Macros'', ''Parameters'', and ''File IO''. All testing is focused on verifying two simple routines:

int sut_add(int x, int y);
int sut_mult(int x, int y);

The routines are contained in ''software_under_test.c | h''. The suggested things to '''Try''' will be focused on applying testing techniques on ''sut_mult()''.

Example option file ''my.opt'' used below for running

--database "%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb"
--device TCP:localhost:8000

== Test Macros ==

This Test Unit sample focuses on leveraging test macros when writing test cases. Note that it is '''not''' required to use test macros. A test case (method) can simply return a value to indicate ''pass'' or ''fail''.

The following articles are related to this example:
* [[Test Unit]] packaging options
* Available [[Test Macros]]
* Adding [[Notes]] to the test logic
* Creating [[Test_Documentation_in_C/C%2B%2B | Doxygen style]] descriptions within test header files
* How [[Running Tests]] works

The Test Unit is called '''TestMacros''' and is implemented in two source files: '''testmacros.cpp''' and '''testmacros.h'''. The comments and descriptions are contained in the header file. One test case (method) is already implemented and one test method that can be used to make changes to is called '''TryStuff'''. Currently the ''TryStuff'' test method is set to ''not in use''.

First thing to do is run the ''TestMacros'' Test Unit. Start the '''TestApp.exe''' and use the following command:

stride --options_file my.opt --run "TestMacros"

You can take a look at the results by opening the generated '''stride.html''' in your browser.

Consider the following for '''TryStuff''':
* Add test code for the ''sut_mult()'' routine (remember to remove the ''set status'' call)
* Add an ''Assert'' macro (verses an ''Expect''). Do you understand the difference?
* Throw in a ''Note'' macro .. provides more viability to the test logic
* Change the test method documentation (in ''testmacros.h'') and see how it shows up automatically in the test report
* Change the test method name

== Parameters ==

This Test Unit sample focuses on using parameters when writing test cases. Note that this sample has been implemented to (i) ensure parameters have been passed and (ii) support two different techniques for passing in parameters.

The following articles are related to this example:

* [[Parameterized Test Units | Passing parameter input]] to a Test Unit
* [[Runtime_Test_Services#srTestGetParam | Reading parameters]] from a Test Unit

The Test Unit is called '''TestParameters''' and is implemented in two source files: '''testparameters.cpp''' and '''testparameters.h'''. The comments and descriptions are contained in the header file. Two test cases are already implemented and one test method that can be used to make changes to is called '''TryStuff'''. Currently the ''TryStuff'' test method is set to ''not in use''.

First thing to do is run the ''TestParameters'' Test Unit. Start the '''TestApp.exe''' and use the following command:

stride --options_file my.opt --run "TestParameters(10,10)"

You can take a look at the results by opening the generated '''stride.html''' in your browser.

Now run a second time and use a file with INI-formatted input:

stride --options_file my.opt --run "TestParameters(%STRIDE_DIR%/samples/testparameters.ini)"

Consider the following for '''TryStuff:'''
* Add test code for ''sut_mult()'' using arguments from the command line
* Add test code for ''sut_mult()'' using your own created ini file
* Change the content of the ini file .. re-run the Test Unit

== File IO ==
This Test Unit sample focuses on reading content from a host file to be used by the executing test logic. Note that this sample uses the [[Runtime_Test_Services#C.2B.2B_Test_Classes | srTest]] base class.

The following articles are related to this example:

* [[File_Transfer_Services | File IO Services]]
* [[Parameterized Test Units | Passing parameter input]] to a Test Unit

The Test Unit is called '''TestFileIO''' and is implemented in two source files: '''testfileio.cpp''' and '''testfileio.h'''. The comments and descriptions are contained in the header file. One test case (method) is already implemented and one test method that can be used to make changes to is called '''TryStuff'''. Currently the ''TryStuff'' test method is set to ''not in use''.

The Test Unit is using an INI file (''stride/samples/testfileio.ini'') to indicate what input file content to use:

## Required parameters used for testing file IO
## Using groups

[add]
File = %STRIDE_DIR%/samples/testfileio.input.add

[mult]

The corresponding ''add'' input file contains the following (note that the ''third column'' equals the sum):

1,2,3
5,6,11
53,27,80

First thing to do is run the ''TestFileIO'' Test Unit. Start the '''TestApp.exe''' and use the following command:

stride --options_file my.opt --run "TestFileIO(%STRIDE_DIR%/samples/testfileio.ini)"

You can take a look at the results by opening the generated '''stride.html''' in your browser.

Consider the following for '''TryStuff:'''
* Add test code for ''sut_mult()'' using a different input file
* Update the ''testfileio.ini'' file to include a second file for input

Stride Sandbox

2015-07-06T21:25:42Z

Marku:

__NOTOC__
Stride's cross-platform capabilities make it possible to use Stride in a '''host-only''' configuration called the '''Sandbox'''. This environment facilitates ''self-training'', ''evaluations'', and ''trying stuff''. It frees you from external hardware dependencies and provides for a rapid ''edit-build-test'' cycle.

The '''Sandbox''' utilizes the framework's SDK that can be built and executed on the host system. When using the SDK Makefile a simulated target ''native application'' is generated, which we call a '''Test Application (TestApp)'''. The Stride Runner application executes on the same host and communicates with the '''TestApp''' process over a TCP/IP connection.

The '''Sandbox''' requires the Stride framework package to be setup on your desktop. Refer to the [[Framework Setup | Framework Setup]] article for more information. It also requires that your desktop contains one of the following '''compilers''':
* For Windows, Microsoft Visual Studio 2008 or later is required. If you don't already have Visual Studio, the free [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio_Express Visual C++ Express] can be used (download [http://www.microsoft.com/express/download/#webInstall here]). In case you have [http://www.cygwin.com Cygwin] installed, the [http://en.wikipedia.org/wiki/GNU_Compiler_Collection GNU Compiler Collection] could be used as an alternative.
* For Linux and FreeBSD, the [http://en.wikipedia.org/wiki/GNU_Compiler_Collection GNU Compiler Collection] (included by default in almost all distros) is required.

== Building ==

=== SDK Makefile ===
The SDK Makefile is set up by ''default'' so that all <tt>.c</tt> <tt>.cpp</tt> and <tt>.h</tt> files found in the directory <tt>SDK\Windows\sample_src</tt> (or <tt>SDK/Posix/sample_src</tt> for Linux/FreeBSD) are included in the compile and link of the '''testapp''' target.

Further--as a pre-compilation step--any <tt>.h</tt> files found in <tt>sample_src</tt> are submitted to the [[STRIDE Build Tools]]. This will result in
* the detection of [[Test_Unit_Pragmas2| test pragmas]] used to declare Test Suites in these <tt>.h</tt> files
* the generation of a database (<tt>.sidb</tt>) file required for executing tests
* the generation of an [[Intercept Module]] required for executing tests

=== Build Steps ===
To begin, be sure that TestApp is not running then perform the following steps:

====Linux/FreeBSD====
<ol>
<li>Build the test app using GNU make
<source lang="bash">
make -C "$STRIDE_DIR/SDK/Posix/src" testapp
</source>
</li>
<li>Note that the following artifacts are produced by the build:

;<tt>$STRIDE_DIR/SDK/Posix/out/bin/TestApp</tt>
: the test application
;<tt>$STRIDE_DIR/SDK/Posix/out/TestApp.sidb</tt>
: the STRIDE interface database file which contains metadata describing the interfaces remoted by the test app (along with other data)
</li>
</ol>

====Windows====
NOTE: ''In case you have [http://www.cygwin.com Cygwin] and [http://en.wikipedia.org/wiki/GNU_Compiler_Collection GNU Compiler Collection] installed and prefer to use it, please follow the build steps for Linux (see previous section) and ignore the one in here.''
<ol>
<li>If using Microsoft Visual Studio, open a [http://msdn.microsoft.com/en-us/library/ms235639(v=vs.100).aspx Visual Studio Command Prompt] to ensure that the compiler and linker are on your PATH.
</li>
<li>Build the test app using the supplied GNU make. (You will get Makefile errors if you use the default make.)
<source lang="dos">
"%STRIDE_DIR%\SDK\Windows\bin\make" -C "%STRIDE_DIR%\SDK\Windows\src" testapp
</source>
<li>Note that the following artifacts are produced by the build:

;<tt>%STRIDE_DIR%\SDK\Windows\out\bin\TestApp.exe</tt>
: the test application
;<tt>%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb</tt>
: the STRIDE interface database file which contains metadata describing the interfaces remoted by the test app (along with other data)
</li>
</ol>

== Running ==
The test app we just built does not have any user tests in it. At this point it provides a starting point for test that we will subsequently add.

However, a set of diagnostic tests that verify operation of the STRIDE runtime itself are always built into the generated TestApp executable. If desired (we recommend you to do so) you could run them by doing the following:

<ol>
<li>Invoke the TestApp. In order to see TestApp's output, we recommend that you manually run in a console window (or Windows equivalent):
;Linux/FreeBSD
<source lang="bash">
$STRIDE_DIR/SDK/Posix/out/bin/TestApp
</source>
;Windows
<source lang="dos">
%STRIDE_DIR%\SDK\Windows\out\bin\TestApp
</source>
(...or launch from the file explorer)

<li> Note TestApp's output upon startup.
<pre>
--------------------------------------------------
STRIDE Test Console Application.
Enter 'Ctrl+C' to Quit.
--------------------------------------------------
Listening on TCP port 8000
starting up...
"_srThread" thread started.
"stride" thread started.
</pre>
</li>
<li>From a second console window, invoke <tt>[[STRIDE_Runner|stride]]</tt> as follows, to verify connectivity with the test app and STRIDE runtime operation:
 
;Linux/FreeBSD
<source lang="bash">
stride --diagnostics --database="$STRIDE_DIR/SDK/Posix/out/TestApp.sidb" --device=TCP:localhost:8000 --run="*"
</source>
;Windows
<source lang="dos">
stride --diagnostics --database="%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb" --device=TCP:localhost:8000 --run="*"
</source>

As the tests run you will see output in both the TestApp (target) and stride (host) console windows.

The host console window output is shown here:
<pre>
Executing diagnostics...
Connecting to device (TCP:localhost:8000)...
runtime version: 5.0.xy
test suite "/Link"
Loopback ..........
Payload Fragmentation
Stub-Proxy Deadlock
Target Characteristics
> 4 passed, 0 failed, 0 in progress, 0 unknown, 0 not in use, 777.77 ms.
test suite "/Stat"
> 2 passed, 0 failed, 0 in progress, 0 unknown, 0 not in use, 74.98 ms.
test suite "/Time"
> 2 passed, 0 failed, 0 in progress, 0 unknown, 0 not in use, 2559.23 ms.
Disconnecting from device...
---------------------------------------------------------------------
Summary: 8 passed, 0 failed, 0 in progress, 0 unknown, 0 not in use, 3411.98 ms.
</pre>
</li>
<li>Note the Summary results shown in the host output; all in use tests should pass.
</li>
<li>To exit TestApp, give the target window focus and enter <tt>Ctrl-C</tt>.
</ol>

==Build Problems==
This page describes several common problems encountered when building a STRIDE TestApp using the Sandbox and suggested solutions.

=== Make Error 1 ===
;Symptom
On Windows, when attempting to build the testapp from the command line, you encounter an error indicating that:
<pre>
‘cl’ is not recognized as an internal or external command,
operable program or batch file.
</pre>

For example:
<pre>
C:\STRIDE\SDK\Windows\src>..\bin\make.exe testapp
cl -c -nologo -W4 -D_UNICODE -DUNICODE -DWIN32 -D_CONSOLE -DUNDER_NT -I”.” -I”../../Runtime” -I”../../SLAP” -I”../../GRS” -I”../o
ut/src” -I”../sample_src” -GS -Zi -DNDEBUG -MD -O2 -D_LIB -DSTRIDE_STATIC -Fd”../out/desktop-Windows_NT-obj//cl.pdb” -Fo”../out/desktop-Windows_NT-o
bj/srapi.o” ”../../Runtime/srapi.c”
‘cl’ is not recognized as an internal or external command,
operable program or batch file.
make: *** [../out/desktop-Windows_NT-obj/srapi.o] Error 1
</pre>

;Cause
Microsoft Visual Studio 2008 or later is not installed or you are not building from a [http://msdn.microsoft.com/en-us/library/ms235639(v=VS.90).aspx Visual Studio Command Prompt].

;Solution
Make sure you have Microsoft Visual Studio 2008 or later installed.

To ensures that the compiler and linker are on your PATH open a Visual Studio Command prompt:
* Click the Start button, point to All Programs, Microsoft Visual Studio 20XX, Visual Studio Tools, and then click Visual Studio 20XX Command Prompt.

=== Make Error 2 ===
;Symptom
On Windows, when attempting to build the testapp from the command line the following errors are observed:

<pre>
syntax error near unexpected token `('
syntax error near unexpected token `('
syntax error: unexpected end of file
</pre>

For example:
<pre>
C:\stride\SDK\Windows\src>..\bin\make testapp
/bin/sh: -c: line 0: syntax error near unexpected token `('
/bin/sh: -c: line 0: `IF EXIST ../out. (IF NOT EXIST ../out/src mkdir "../out/src") ELSE mkdir "../out" && mkdir "../out/src".'
/bin/sh: -c: line 0: syntax error near unexpected token `('
/bin/sh: -c: line 0: `IF EXIST ../out. (IF NOT EXIST ../out/src mkdir "../out/src") ELSE mkdir "../out" && mkdir "../out/src".'
/bin/sh: -c: line 1: syntax error: unexpected end of file
make: *** [cleanapp] Error 258
</pre>

;Cause
This error occurs because gnu make on Windows will search for an Unix shell (<tt>sh</tt>, <tt>bash</tt> or <tt>csh</tt>) anywhere in your PATH when executing shell commands and only default to DOS shell (<tt>cmd.exe</tt>) when no Unix shell is found. The sandbox Makefile uses DOS shell syntax, so when a Unix shell is found on your PATH, this results to errors like above.

Most commonly, this problem is caused by an installation of [http://en.wikipedia.org/wiki/Cygwin Cygwin], though it can also be caused by an installation of the QNX Software Development Platform.

;Solution

Explicitly specify the DOS shell by invoking make like:
<pre>
..\bin\make SHELL=%ComSpec% testapp
</pre>

'''''Note:''' the value of %ComSpec% should be <tt>C:\Windows\system32\cmd.exe</tt>''

An alternative is to remove any directories from your PATH that contain an Unix shell executable or otherwise prevent such from being found. (e.g. rename its parent directory).

=== Make Error 3 ===
;Symptom
On Linux, when attempting to build the testapp from the command line, the following error is observed:
<pre>
g++: command not found
</pre>

For example:
<pre>
# make testapp
g++ -c -I”.” -I”../../Runtime” -I”../../SLAP” -I”../../GRS” -I”../out/src” -I”../sample_src” -fPIC -D_DEBUG -O0 -g3 -Wall -o ”../out/i386-Linux-obj/srtestpp.obj” ”../../Runtime/srtestpp.cpp”
/bin/sh: g++: command not found
make: *** [../out/i386-Linux-obj/srtestpp.obj] Error 127
</pre>

;Cause
The C++ compiler, <tt>g++</tt> can't be found on your PATH.

Most commonly, this problem is caused by not having a complete installation of [http://en.wikipedia.org/wiki/GNU_Compiler_Collection GNU Compiler Collection].

;Solution
Make sure you have a complete GNU Compiler Collection installed.

=== Make Error 4 ===
;Symptom
When building the testapp from the command line, you see the following compiler errors:

<pre>
../out/src/strideIM.cpp(50) : error C3861: '_srTestResultCountReset': identifier not found
../out/src/strideIM.cpp(54) : error C3861: '_srTestSendFinalStatus': identifier not found
../out/src/strideIM.cpp(56) : error C3861: '_srTestAddToTotal': identifier not found
../out/src/strideIM.cpp(56) : error C3861: '_srTestResultGetTotals': identifier not found
...
...
...
</pre>

;Cause
You are using an outdated version of the STRIDE build tools.

You can see which version of the tools were used to generate your STRIDE sources by looking at the top of the strideIM.cpp source file. The comment block at the top of the file shows this version.

;Solution
Remove the old tools and/or change your path so that the current tools are used.

STRIDE Extensions for Visual Studio

2015-07-06T21:22:40Z

Marku: /* Sort Out Header Files */

== Introduction ==
The Stride Extensions for Visual Studio can be used to integrate directly into your existing Visual Studio C/C++ project.

== Prerequisites ==
* Visual Studio 2008 or later (an express distribution is fine)
* a recent version of the Stride installed
* a static library version of the runtime built using the source files included in the [[Windows_SDK | Windows SDK]]
** to build the library open a [http://msdn.microsoft.com/en-us/library/ms235639(v=vs.100).aspx Visual Studio Command Prompt] and do as follows:
<pre>
> md %STRIDE_DIR%\SDK\Windows\lib
> cd %STRIDE_DIR%\SDK\Windows\src
> ..\bin\make clean
> ..\bin\make RTSINGLEPROC=0
> copy %STRIDE_DIR%\SDK\Windows\out\lib\stride-x86-Windows_NT.lib %STRIDE_DIR%\SDK\Windows\lib\stride.lib
> ..\bin\make clean
> ..\bin\make RTSINGLEPROC=0 DEBUG=1
> copy %STRIDE_DIR%\SDK\Windows\out\lib\stride-x86-Windows_NT-d.lib %STRIDE_DIR%\SDK\Windows\lib\stride-d.lib
</pre>
* an existing C/C++ Visual Studio project
** or you can easily create a new one, e.g. "Win32 Console Application", by using the project wizard provided within Visual Studio.

== Installation ==
STRIDE Extensions need to be added to any project that is to generate ''intercept module''.

=== Visual Studio 2010 (or newer) ===
* Right click on the project in the '''Solution Explorer''' window and choose '''Build Customizations…''' from the menu that is displayed.
* In the dialog that is displayed, click '''Find Existing…''' and select '''$(STRIDE_DIR)\SDK\Windows\settings\stride.targets''' (if asked, say “No” to adding to standard “Build Customizations Search Path”)
* Make sure the check box next to '''stride''' is ''enabled''.

=== Visual Studio 2008 ===
''NOTE:'' ''Due to limitations in Visual Studio 2008 the STRIDE integration is much more complicated compared to new versions. We recommend upgrading to Visual Studio 2010 or newer.''

* Right click on the project in the '''Solution Explorer''' window and choose '''Build Custom Build Rules...''' from the menu that is displayed.
* In the dialog that is displayed, click '''Find Existing…''' and select '''$(STRIDE_DIR)\SDK\Windows\settings\stride.rules''' (if asked, say “No” to adding to standard “Rule Files Search Path”)
* Make sure the check box next to '''STRIDE Rules''' is ''enabled''.

== Configuration ==
The STRIDE Extensions execute a set of pre-build steps on your header files that generate test harnessing code that is later compiled in your application. To support the pre-build steps your global project settings require updating.

=== Project Properties ===
Adjust your project properties to compile and link with the STRIDE Runtime and generated test harnessing code.

''NOTE:'' ''Make sure to apply the following changes to all project's configurations (e.g. Debug, Release).''

==== C/C++ Properties ====
* Right click on the project in the '''Solution Explorer''' window and choose '''Properties''' from the menu that is displayed.
* From the properties dialog, select '''Configuration Properties | C/C++ | General''' from the tree view in the left pane.
* From the right pane, add to '''Additional Include Directories''' the following:
<pre>
$(STRIDE_DIR)\SDK\Runtime
$(STRIDE_DIR)\SDK\Windows\src
</pre>
* Select '''Configuration Properties | C/C++ | Preprocessor''' from the tree view in the left pane.
* From the right pane, add to '''Preprocessor Definitions''' the following:
<pre>
STRIDE_ENABLED
STRIDE_STATIC
srCOMPLEX_TARGET
</pre>
* Select '''Configuration Properties | C/C++ | Code Generation''' from the tree view in the left pane.
* From the right pane, set '''Runtime Library''' to '''Multi-threaded DLL (/MD)''' (or '''Multi-threaded Debug DLL (/MDd)''' for Debug configuration).

==== Linker Properties ====
* Right click on the project in the '''Solution Explorer''' window and choose '''Properties''' from the menu that is displayed.
* From the properties dialog, select '''Configuration Properties | Linker | General''' from the tree view in the left pane.
* From the right pane, add to '''Additional Library Directories''' the following:
;If you built stride.lib from the makefile in the SDK
<pre>
$(STRIDE_DIR)\SDK\Windows\lib
</pre>

* Select '''Configuration Properties | Linker | Input''' from the tree view in the left pane.
* From the right pane, add to '''Additional Dependencies''' the following:
<pre>
stride.lib
ws2_32.lib
</pre>
''NOTE:'' ''For Debug configuration please specify "stride-d.lib".''

=== Sort Out Header Files ===
Every header file in the project that has [[Test Pragmas]] will cause harnessing code to be generated when processed by the STRIDE compiler. (Other non-stride header files can also be processed by the STRIDE compiler, but this only results in longer compile times.)

For each header file to be used as input to the STRIDE rules:
* Right click on the header file in the '''Solution Explorer''' window and choose '''Properties''' from the menu that is displayed.
* From the properties dialog, select '''Configuration Properties | General''' from the tree view in the left pane.
* From the right pane, set the '''Item Type''' property to '''STRIDE Compile-Instrument'''.

If there are any header files that contain no test code declarations, then we recommend disabling the build step for these files. This could be done in the last step from above by setting the '''Item Type''' to '''C/C++ header''' or by setting '''Exclude From Build''' property to '''Yes'''. Remember as you add more header source files to the project, they will be automatically processed by STRIDE unless you explicitly disable.

=== STRIDE Compile-Instrument Properties ===

==== Visual Studio 2010 / 2012 ====

The default settings are usually sufficient so you don't need to make any changes.

When you build your project, notice that:
* The STRIDE build tools automatically run and generate a STRIDE database (.sidb) file and intercept module (IM) source files. By default the names are derived from '''$(TargetName)''', and the associated files are written to the '''$(ProjectDir)''' directory.
* The IM source, ''$(TargetName)IM.cpp'', is automatically compiled and linked along with your other project sources.

If you desire to change the ''Database Name'' and/or the ''Intercept Module Name'' use the following steps:
* Right click on the project in the '''Solution Explorer''' window and choose '''Properties''' from the menu that is displayed.
* From the properties dialog, select '''Configuration Properties | STRIDE Compile-Instrument'''
* Update '''Database Name''' and/or the '''Intercept Module Name''' right pane values as desired.

''NOTE:'' ''If you do not see '''STRIDE Compile-Instrument''' try [[#Sort_Out_Header_Files | adding a header file]] to your project.''

The default harness generation assumes STUB settings for all captured functions. If you require different settings for any of your interfaces, you should create a text file containing the [[S2sinstrument#Options|additional settings]] and specify it as follows:
* Right click on the project in the '''Solution Exporer''' window and choose '''Properties''' from the menu that is displayed.
* From the properties dialog, select '''Configuration Properties | STRIDE Compile-Instrument | Command Line''' from the tree view in the left pane.
* From the right pane, add to '''Additional Options''' property in the right pane.
<pre>
--intercept_options_file="path\to\my\file.s2instrument"
</pre>

==== Visual Studio 2008 ====
* Right click on the project in the '''Solution Exporer''' window and choose '''Properties''' from the menu that is displayed.
* From the properties dialog, select '''Configuration Properties | STRIDE Compile-Instrument | Compile''' from the tree view in the left pane.
* From the right pane, add to '''Compile Options Files''' the following:
<pre>
$(STRIDE_DIR)\SDK\Windows\settings\stride.s2scompile
</pre>
* From the right pane, update '''Include Directories''' to be exactly the same the one specified in '''Additional Include Directories''' in '''Configuration Properties | C/C++ | General''' section.
* From the right pane, update '''Preprocessor Definitions''' to be exactly the same the one specified in '''Preprocessor Definitions''' in '''Configuration Properties | C/C++ | Preprocessor''' section.
* Build your project once to generate the STRIDE database (.sidb) and intercept module (IM) source files. Notice, that you will get linker errors during this build since you have not yet added the generated IM source to the project.
* From the '''$(ProjectDir)''' directory add the generated IM source file - '''strideIM.cpp''', '''strideIM.h''' and '''strideIMEntry.h''' - to your project.
** For '''strideIM.cpp''' make sure to set its '''Create/Use Precompiled Header''' property to '''Not Using...'''.
** For '''strideIM.h''' and '''strideIMEntry.h''' make sure to set their '''Excluded From Build''' property to '''Yes''' as specified [[#Sort_Out_Header_Files|above]].
* Rebuild your project and resolve any compiler errors/warnings.

The default harness generation assumes STUB settings for all captured functions. If you require different settings for any of your interfaces, you should create a text file containing the [[S2sinstrument#Options|additional settings]] and specify it as follows:
* Right click on the project in the '''Solution Exporer''' window and choose '''Properties''' from the menu that is displayed.
* From the properties dialog, select '''Configuration Properties | STRIDE Compile-Instrument | Instrument''' from the tree view in the left pane.
* From the right pane, add to '''Intercept Options File''' property in the right pane.
<pre>
path\to\my\file.s2instrument
</pre>

''NOTE:'' ''As you add sources, you will likely also need to add Additional Includes and Preprocessor Defines to the project's STRIDE Compile-Instrument settings in order to successfully build.''

== Update Your Startup ==

Update your application's main function to initialize the STRIDE subsystem as described in [[Windows_SDK#Target_Integration | this article]]. ''Note'' ''that the sample code assumes you have generated your Intercept Module with a name of '''myintercept'''. Make sure to replace any reference to that token in the code to the name you [[#STRIDE_Compile-Instrument_Properties | chose above]].''

Framework Setup

2015-07-06T21:19:44Z

Marku:

= Introduction =
The packages described below contain all of the ''source'' and ''binary'' components required to
* setup a desktop with the '''STRIDE Runner'''
* integrate the '''STRIDE Runtime''' with the target device
* add the '''STRIDE Compiler''' (aka ''Build tools'') to the software build system.

The desktops supported are Windows, Linux, and FreeBSD.

= Packages =
Files are installed by unzipping the provided package to your PC. Packages are available targeting the following operating systems (your version number may be different than that shown):
;Windows (x86)
:<tt>STRIDE_framework-windows_5.x.yy.zip</tt>
;Linux (x86)
:<tt>STRIDE_framework-linux_5.x.yy.tgz</tt>
;FreeBSD (x86)
:<tt>STRIDE_framework-freebsd_5.x.yy.tgz</tt>

Please see the appropriate installation instructions below.

= Windows =

The following installation example assumes the the installation package is located in your root directory and that the directory <tt>\stride</tt> exists. You can choose to install to a different location (all instructions below assume you are installing into <tt>\stride</tt>).

The example uses the open source [http://www.7-zip.org/ 7-Zip] utility to unzip the archive.

cd \stride
"\Program Files\7-Zip\7z" x ..\STRIDE_framework-windows_5.x.yy.zip

Once unzipped, files will have been installed under the <tt>\stride</tt> directory.

=== Updated PATH ===
As a final step, you will need to update your <tt>[http://en.wikipedia.org/wiki/Path_(variable) PATH]</tt> environment variable to include <tt>\stride\bin</tt>.
For instructions on modifying it, please see [http://support.microsoft.com/kb/310519 http://support.microsoft.com/kb/310519].

NOTE: ''Make sure to insert '''no spaces''' before and after the semicolon separators(;).''

=== Create/Update STRIDE_DIR===

Verify that the <tt>STRIDE_DIR</tt> environment variable exists and is set to the root installation directory (<tt>\stride</tt>). If this environment variable does not yet exist, you should create it as a user environment variable.

To confirm installation and display ''help'' run the following command in a console window:

stride -h

=== Uninstalling ===
To uninstall STRIDE simply:
* Remove any reference to <tt>\stride\bin</tt> in your <tt>PATH</tt> environment variable.
* Remove <tt>STRIDE_DIR</tt> environment variable.
* Remove <tt>\stride</tt> directory.

= Linux/FreeBSD =

The following installation example assumes the the installation package is located in your home directory and that the directory <tt>~/stride</tt> exists. You can choose to install to a different location (all instructions below assume you are installing into <tt>~/stride</tt>).

cd ~/stride
tar -zxvf ../STRIDE_framework-linux_5.x.yy.tgz

Once unzipped, files will have been installed under the <tt>~/stride</tt> directory.

=== Updated PATH ===
As a final step, you will need to update your <tt>[http://en.wikipedia.org/wiki/Path_(variable) PATH]</tt> environment variable to include <tt>~/stride/bin</tt>.

If you use the bash shell, enter the following at a command prompt, or to automatically set at each login, add to your <tt>.bashrc</tt>:
export PATH=$PATH:~/stride/bin

For other shells, and more information, please see the following articles:
* [http://www.linuxheadquarters.com/howto/basic/path.shtml http://www.linuxheadquarters.com/howto/basic/path.shtml].
* [http://en.wikipedia.org/wiki/Environment_variable#UNIX http://en.wikipedia.org/wiki/Environment_variable]

=== Create/Update STRIDE_DIR===
Verify that the <tt>STRIDE_DIR</tt> environment variable exists and is set to the root installation directory (<tt>~/stride</tt>). If this environment variable does not yet exist, you should automatically set at each login, add to your <tt>.bashrc</tt>:
export STRIDE_DIR=~/stride

To confirm installation and display ''help'' run the following command in a console window:

stride -h

NOTE: ''In a 64-bit environment the above may fail with errors like: <code>"/lib/ld-linux.so.2: bad ELF interpreter: No such file or directory"</code> or <code>"ELF interpreter /libexec/ld-elf32.so.1 not found"</code>. To resolve this issue install the appropriate 32-bit compatibility libraries for your Linux/FreeBSD distribution:''

* Debian / Ubuntu
sudo apt-get install ia32-libs
sudo apt-get install ia32-libs-multiarch:i386 (for 12.04 or higher)
* Fedora / CentOS / RHEL
sudo yum -y install glibc.i686 libstdc++.i686
* FreeBSD
Make sure to have <code>lib32</code> installed (via [http://www.freebsd.org/cgi/man.cgi?query=sysinstall&apropos=0&sektion=0&manpath=FreeBSD+8.4-RELEASE&arch=default&format=html sysinstall(8)] - Configure|Distributions|lib32) and have your kernel built with:
options COMPAT_FREEBSD32 # Compatible with i386 binaries

=== Uninstalling ===
To uninstall STRIDE simply:
* Remove any reference to <tt>~/stride/bin</tt> in your <tt>PATH</tt> environment variable.
* Remove <tt>STRIDE_DIR</tt> environment variable.
* Remove <tt>~/stride</tt> directory.

= Directories and Files =

To integrate Stride in to your target build system it is required to understand the directories layout and the files inside then. A quick orientation is shown below.

''NOTE:'' ''It's not necessary to understand the workings of Stride to perform evaluation or training. The framework package contains a [[Stride Sandbox]] that utilizes a SDK that is set up with appropriate options and settings to enable "out of the box" functionality.''

==<tt>bin</tt>==
This directory contains the [[Build Tools|Stride Build Tools]] and the [[Stride Runner]].

The build tools are invoked early on in the target software build process to generate special Stride artifacts that are used in subsequent build steps and later when running tests against the target. When using the Stride Sandbox, these files are needed on the host computer since this is where we are building the target application. In a production environment, these files are needed only on the computer that performs the target software build.

The [[Stride Runner]] is the program you use to run tests from the host.

==<tt>lib</tt>==
This directory contains a set of Stride specific core scripting libraries along with prebuild binaries intended to be used for Perl based test scripts.

==<tt>samples</tt>==
The Samples directory contains a number of source files used for training and evaluation.

==<tt>SDK</tt>==
This directory contains the sub-directories <tt>Posix/Windows</tt> and <tt>Runtime</tt>, which contain source code that comprises the [[Runtime_Reference|STRIDE Runtime]]. These sources are built in to a static libary (e.g. STRIDE Runtime library - <tt>stride.a/lib</tt>) as a dependency of your Test Application.

The <tt>Posix</tt> and <tt>Windows</tt> directories contain the target operating system specific source and configuration. If you are interested in the details, consult the articles [[Posix SDK]] and [[Windows SDK]]. Each of them contains the following sub-directories:

*<tt>settings</tt>
: This directory contains a set of <tt>stride.XXX.s2scompile</tt> files, where <tt>XXX</tt> coresponds to the target CPU architecture (i.e. X86, ARM...). These files, used by the [[s2scompile|STRIDE Compiler]], specify target CPU characteristics (endian-ness, data sizes and alignments). On Windows, this directory also contains a set of files for [[STRIDE_Extensions_for_Visual_Studio|building with Visual Studio]].
*<tt>src</tt>
: This directory contains the source of the target [[Platform Abstraction Layer]] PAL. In addition there is a sample Makefile used to produce a sandbox TestApp.

= Perl Installation (Optional) =
If you intend to use '''Test Scripts''' you will need a recent version of Perl (x86 with threads support) installed.

As of this writing, we support only the 32-bit versions 5.8.9, 5.10.x, 5.12.x, 5.14.x, 5.16.x and 5.18.x of Perl.

== Windows ==
It is required to use the standard 32-bit Perl distributions from [http://www.activestate.com/activeperl ActiveState].

The following additional (non-standard) Perl packages are also required for full functionality of STRIDE tests in perl:

* [http://search.cpan.org/perldoc/Class::ISA Class::ISA]
* [http://search.cpan.org/perldoc/Pod::POM Pod::POM]
* [http://search.cpan.org/perldoc/Devel::Symdump Devel::Symdump]
* [http://search.cpan.org/perldoc/Config::Tiny Config::Tiny]

You can easily install these packages using the [http://docs.activestate.com/activeperl/5.16/faq/ActivePerl-faq2.html ppm tool]. If you access the Internet via a proxy make sure to read [http://docs.activestate.com/activeperl/5.16/faq/ActivePerl-faq2.html#ppm_and_proxies this]. Simple command-line installation of PACKAGE_NAME (the package to install) typically just requires typing:

ppm install PACKAGE_NAME

== Linux/FreeBSD ==
We recommend you to use the standard 32-bit Perl distribution that comes with your OS version. In case you need to manually build from source make sure to configure "shared library" (<tt>-Duseshrplib</tt>), "thread support" (<tt>-Duseithreads</tt>) and no "64-bit support" (<tt>-Uuse64bitint -Uuse64bitall</tt>).

The following additional (non-standard) Perl packages are also required for full functionality of Stride tests in perl:

* [http://search.cpan.org/perldoc/YAML::XS YAML::XS]
* [http://search.cpan.org/perldoc/Class::ISA Class::ISA]
* [http://search.cpan.org/perldoc/Pod::POM Pod::POM]
* [http://search.cpan.org/perldoc/Devel::Symdump Devel::Symdump]
* [http://search.cpan.org/perldoc/Config::Tiny Config::Tiny]

If your perl is installed in a system directory (<tt>/usr/bin/perl</tt>, for instance), you will need root access to install shared modules. The simplest method for installing packages is via the [http://www.perl.com/doc/manual/html/lib/CPAN.html CPAN shell]. If you access the Internet via a proxy make sure to set the appropriate [http://search.cpan.org/dist/CPAN/lib/CPAN.pm#Config_Variables CPAN config variables]. To start the shell in interactive mode:

sudo perl -MCPAN -eshell

Once in the shell, search for and install the latest stable version of PACKAGE_NAME (the package to install):

install PACKAGE_NAME

The STRIDE perl packages also need to load your system's '''libperl.so''' (shared object file) at runtime. Depending on your system, this file should be loadable from a perl CORE directory or from one of the shared system directories. If you '''DO NOT''' have this shared library on your system, you might need to install a ''libperl-dev'', ''perl-devel'' or ''perl-libs'' package in order to get it. Here is how you can do that on the console of some Linux distributions:

* Debian / Ubuntu
sudo apt-get install libperl-dev
* Fedora / CentOS / RHEL
sudo yum -y install perl-devel

== Validation ==
Once you have installed Perl we recommend you to run the following command in a console window:

<source lang="bash">
stride --diagnostics Perl --output PerlCheck
</source>

If everything was properly set up you should get the following output:

<pre>
Executing diagnostics...
script "/Perl"
> 2 passed, 0 failed, 0 in progress, 0 not in use, 486.95 ms.
---------------------------------------------------------------------
Summary: 2 passed, 0 failed, 0 in progress, 0 not in use, 486.95 ms

Saving result file...
</pre>

In addition a report file with name <tt>PerlCheck.xml</tt> will be created in the current directory. If interested in the details you could open that report file in a browser of your choice.

Perl Script Snippets

2015-07-06T20:55:30Z

Marku: /* Standalone two-line script for invoking a remote function */

__NOTOC__
Below are some brief examples of script test modules that use the [[Perl Script APIs]] for Stride. Most of these examples are incomplete and are only intended to give a quick overview of how this API can be used.

With any perl module that you write, we recommend that you perform a quick syntax check before attempting to execute the module using the Stride Runner. Syntax checking with warnings is easily accomplished by running with the <tt>-cw</tt> option, e.g.:

* Windows
perl -I %STRIDE_DIR%\lib\perl -I %STRIDE_DIR%\lib\perl\5.10 -cw MyTests.pm

* Linux/FreeBSD
perl -I $STRIDE_DIR/lib/perl -I $STRIDE_DIR/lib/perl/5.10 -cw MyTests.pm

This will compile the perl script without running it, and it will emit any syntax errors that are found.

== Canonical module format ==

<source lang="perl">
use strict;
use warnings;

package MyTests;
use base qw(STRIDE::Test);
use STRIDE::Test;

sub test_one : Test
{
ASSERT_TRUE(1);
}

sub test_two : Test
{
ASSERT_TRUE(0);
}

1;
</source>

== Subroutine attributes to declare test methods and fixtures ==

<source lang="perl">

sub a_test : Test {
# test method
}

sub startup_method : Test(startup) {
# startup fixturing
}

sub shutdown_method : Test(shutdown) {
# shutdown fixturing
}

sub setup_method : Test(setup) {
# setup fixturing
}

sub teardown_method : Test(teardown) {
# teardown fixture
}
</source>

== Test module including POD documentation ==

<source lang="perl">
package MyTests;
use base qw(STRIDE::Test);
use STRIDE::Test;

=head1 NAME

MyTests - example tests

=head1 DESCRIPTION

This is MyTests_1, a deeply funky piece of Perl code.

=head1 METHODS

=cut

=head2 test_one

this is a simple "passing" test.

=cut
sub test_one : Test
{
ASSERT_TRUE(1);
}

=head2 test_two

this is a simple "failing" test.

=cut
sub test_two : Test
{
ASSERT_TRUE(0);
}

1;
</source>

== Simple expectation test ==

<source lang="perl">
sub sync_exact : Test {
my $h = TestPointSetup(
ordered => 1,
expected => [
"point a",
"point b",
"point c"
],
unexpected => [ TEST_POINT_EVERYTHING_ELSE ]
);

#...start source under test, if necessary

# use Check if the events have all happened by the time
# you validate the test points. Otherwise use Wait with
# a reasonable timeout value.
$h->Check();
}
</source>

== Expectation test with predicate==

Here's how to specify a predicate for validating a test point. The predicate
shown is just a stub and doesn't do any actual validation.

<source lang="perl">
sub _myPredicate
{
my ($test_point, $expected_data) = @_;
# $test_point is a hashref with the following fields:
# label, data, size, bin, file, line
# use the test point data to perform validation and return
# nonzero value on success, 0 on failure.

return 1;
}

sub use_a_predicate : Test {
# use arrayref notation to specify label, count, predicate and expected data
# if you want to specify a predicate. Predicates can be specified for some, all
# or none of the test points. The fourth field is optional - it is just passed
# to the predicate as the second argument and is typically used to specify some
# expected data for the predicate to validate against.
my $h = TestPointSetup(
expected => [
["point a", 1, \&_myPredicate, 54321],
"point b",
["point c", 1, \&_myPredicate]
]
);

$h->Check();
}
</source>

== Reusing a trace data file as expectation ==

This examples shows a simple way to reuse captured YAML trace data from a file as your expectation (this is an alternative to specifying the <tt>expected</tt> array directly). This example assumes the ''trace_data.yml'' file lives in the same directory as the test module file, but you are free to change this - just change the <tt>expect_file</tt> argument accordingly.

<source lang="perl">
# these standard modules are only required for the file
# path logic used to generate a full file path for trace_data.yml
use File::Basename;
use File::Spec;

sub trace_data : Test {
my $h = TestPointSetup(
ordered => 1,
expect_file => File::Spec->catfile(dirname(__FILE__), 'trace_data.yml'),
);

# make any function calls necessary to start the SUT,
# then do a Check or Wait, depending on your scenario

$h->Check();
}
</source>

== Invoking a function on the target ==

=== Standalone two-line script for invoking a remote function ===
This simple script shows the minimal code required to make a remote function call of a function that has been enabled with the [[Scl_function | Function Pragrma]] enabling remoting.
<source lang="perl">
use STRIDE;
$STRIDE::Remote->MyRemoteFunction();
</source>
More functions can be added to this script by making additional calls using the <tt>$STRIDE::Remote</tt> object. If your remote function accepts arguments, they should be added to the function call parameters (see the [[Perl_Script_APIs#STRIDE::Remote|Perl API Reference]] for more information).

=== Making calls with test modules ===

Within test modules, remote functions can be easily invoked using the exported <tt>Remote</tt> object. The following two examples illustrate this.

==== Blocking syntax (blocks until function returns) ====

<source lang="perl">
my $retval = Remote->foo(1, "input string");
Remote->bar();
</source>

==== asynchronous execution (return immediately, function continues to execute on device) ====

<source lang="perl">
my $fh = Remote->async->foo(1, "input string");
my $retval = $fh->Wait(1000); #waits up to one second for the function to return
</source>

== Accessing compiler macro values (constants) ==
<source lang="c">
/* some compilation unit included in the STRIDE compilation process */
#define SOME_DEFINED_VALUE 42
</source>

<source lang="perl">
my $value = Remote->SOME_DEFINED_VALUE;
</source>

Perl Script Snippets

2015-07-06T20:50:59Z

Marku:

__NOTOC__
Below are some brief examples of script test modules that use the [[Perl Script APIs]] for Stride. Most of these examples are incomplete and are only intended to give a quick overview of how this API can be used.

With any perl module that you write, we recommend that you perform a quick syntax check before attempting to execute the module using the Stride Runner. Syntax checking with warnings is easily accomplished by running with the <tt>-cw</tt> option, e.g.:

* Windows
perl -I %STRIDE_DIR%\lib\perl -I %STRIDE_DIR%\lib\perl\5.10 -cw MyTests.pm

* Linux/FreeBSD
perl -I $STRIDE_DIR/lib/perl -I $STRIDE_DIR/lib/perl/5.10 -cw MyTests.pm

This will compile the perl script without running it, and it will emit any syntax errors that are found.

== Canonical module format ==

<source lang="perl">
use strict;
use warnings;

package MyTests;
use base qw(STRIDE::Test);
use STRIDE::Test;

sub test_one : Test
{
ASSERT_TRUE(1);
}

sub test_two : Test
{
ASSERT_TRUE(0);
}

1;
</source>

== Subroutine attributes to declare test methods and fixtures ==

<source lang="perl">

sub a_test : Test {
# test method
}

sub startup_method : Test(startup) {
# startup fixturing
}

sub shutdown_method : Test(shutdown) {
# shutdown fixturing
}

sub setup_method : Test(setup) {
# setup fixturing
}

sub teardown_method : Test(teardown) {
# teardown fixture
}
</source>

== Test module including POD documentation ==

<source lang="perl">
package MyTests;
use base qw(STRIDE::Test);
use STRIDE::Test;

=head1 NAME

MyTests - example tests

=head1 DESCRIPTION

This is MyTests_1, a deeply funky piece of Perl code.

=head1 METHODS

=cut

=head2 test_one

this is a simple "passing" test.

=cut
sub test_one : Test
{
ASSERT_TRUE(1);
}

=head2 test_two

this is a simple "failing" test.

=cut
sub test_two : Test
{
ASSERT_TRUE(0);
}

1;
</source>

== Simple expectation test ==

<source lang="perl">
sub sync_exact : Test {
my $h = TestPointSetup(
ordered => 1,
expected => [
"point a",
"point b",
"point c"
],
unexpected => [ TEST_POINT_EVERYTHING_ELSE ]
);

#...start source under test, if necessary

# use Check if the events have all happened by the time
# you validate the test points. Otherwise use Wait with
# a reasonable timeout value.
$h->Check();
}
</source>

== Expectation test with predicate==

Here's how to specify a predicate for validating a test point. The predicate
shown is just a stub and doesn't do any actual validation.

<source lang="perl">
sub _myPredicate
{
my ($test_point, $expected_data) = @_;
# $test_point is a hashref with the following fields:
# label, data, size, bin, file, line
# use the test point data to perform validation and return
# nonzero value on success, 0 on failure.

return 1;
}

sub use_a_predicate : Test {
# use arrayref notation to specify label, count, predicate and expected data
# if you want to specify a predicate. Predicates can be specified for some, all
# or none of the test points. The fourth field is optional - it is just passed
# to the predicate as the second argument and is typically used to specify some
# expected data for the predicate to validate against.
my $h = TestPointSetup(
expected => [
["point a", 1, \&_myPredicate, 54321],
"point b",
["point c", 1, \&_myPredicate]
]
);

$h->Check();
}
</source>

== Reusing a trace data file as expectation ==

This examples shows a simple way to reuse captured YAML trace data from a file as your expectation (this is an alternative to specifying the <tt>expected</tt> array directly). This example assumes the ''trace_data.yml'' file lives in the same directory as the test module file, but you are free to change this - just change the <tt>expect_file</tt> argument accordingly.

<source lang="perl">
# these standard modules are only required for the file
# path logic used to generate a full file path for trace_data.yml
use File::Basename;
use File::Spec;

sub trace_data : Test {
my $h = TestPointSetup(
ordered => 1,
expect_file => File::Spec->catfile(dirname(__FILE__), 'trace_data.yml'),
);

# make any function calls necessary to start the SUT,
# then do a Check or Wait, depending on your scenario

$h->Check();
}
</source>

== Invoking a function on the target ==

=== Standalone two-line script for invoking a remote function ===
This simple script shows the minimal code required to make a remote function call of a function that has been enabled with [[Function Capturing#Remoting|STRIDE remoting]].
<source lang="perl">
use STRIDE;
$STRIDE::Remote->MyRemoteFunction();
</source>
More functions can be added to this script by making additional calls using the <tt>$STRIDE::Remote</tt> object. If your remote function accepts arguments, they should be added to the function call parameters (see the [[Perl_Script_APIs#STRIDE::Remote|Perl API Reference]] for more information).

=== Making calls with test modules ===

Within test modules, remote functions can be easily invoked using the exported <tt>Remote</tt> object. The following two examples illustrate this.

==== Blocking syntax (blocks until function returns) ====

<source lang="perl">
my $retval = Remote->foo(1, "input string");
Remote->bar();
</source>

==== asynchronous execution (return immediately, function continues to execute on device) ====

<source lang="perl">
my $fh = Remote->async->foo(1, "input string");
my $retval = $fh->Wait(1000); #waits up to one second for the function to return
</source>

== Accessing compiler macro values (constants) ==
<source lang="c">
/* some compilation unit included in the STRIDE compilation process */
#define SOME_DEFINED_VALUE 42
</source>

<source lang="perl">
my $value = Remote->SOME_DEFINED_VALUE;
</source>

Perl Script APIs

2015-07-06T20:49:17Z

Marku:

__NOTOC__
The Stride perl script model requires you to create perl modules (*.pm files) to group your tests. The following documents the API for the STRIDE::Test base class that you use when creating these modules.

= STRIDE::Test =

This is the base class for test modules. It provides the following methods.

== Declaring Tests ==

Once you have created a package that inherits from STRIDE::Test, you can declare any subroutine to be a test method by declaring it with the '': Test'' attribute. In addition to test methods, the following attributes declare other kinds of subroutines:

{| class="prettytable"
| colspan="2" | '''subroutine attributes'''

|-valign="top"
! attribute!! description

|-valign="top"
| Test
| declares a test method - will be executed automatically when the module is run.

|-valign="top"
| Test(startup)
| startup method, called once before any of the test methods have been executed.

|-valign="top"
| Test(shutdown)
| shutdown method, called once after all test methods have been executed.

|-valign="top"
| Test(setup)
| setup fixture, called before each test method.

|-valign="top"
| Test(teardown)
| teardown fixture, called after each test method.

|}

You are free to declare as many methods as you like with these attributes. When more than one method has been declared with the same attribute, the methods will be called at the appropriate time in the order declared.

== Methods ==

These methods are all available in the context of a test module that inherits from STRIDE::Test.

{| class="prettytable"
| colspan="2" | '''Methods'''

|-valign="top"
! name !! description
|-valign="top"
| <source lang="perl">TestCase</source>
| returns the default test case object. See [[#TestCase|here]] for more info.

|-valign="top"
| <source lang="perl">Remote</source>
| Returns a [[#STRIDE::Remote|STRIDE::Remote]] object that was initialized with the active database. This object is used for accessing captured at the time of compilation remote functions and constant values (macros and enums).

|-valign="top"
| <source lang="perl">GetParam(name, defval)</source>
| returns an input parameter's value associated with the default test suite.

|-valign="top"
| <source lang="perl">TestCase AddCase(name, description)</source>
| creates a new test case and adds it to the default test suite.

|-valign="top"
| <source lang="perl">TestAnnotation AddAnnotation(level, name, description)</source>
| creates a new test annotation and adds it to the default test suite. The value of ''level'' could be one of the predefined constants:
* '''ANNOTATION_TRACE'''<ref name="const">symbols like these are exported as perl constants - don't quote these values when you use them - rather, use the bare symbols and perl will use the constant value we provide</ref>
* '''ANNOTATION_DEBUG'''
* '''ANNOTATION_INFO'''
* '''ANNOTATION_WARNING'''
* '''ANNOTATION_ERROR'''
* '''ANNOTATION_FATAL'''
|

|-valign="top"
| <source lang="perl">SetData(name, value)</source>
| associates a custom name-value pair with the default test suite.

|-valign="top"
| <source lang="perl">TestPointSetup(
expected => [],
unexpected => [],
ordered => 0|1,
strict => 0|1,
continue => 0|1,
expect_file => filepath,
predicate => coderef,
replay_file => filepath,
test_case => case)</source>
| Creates a new instance of [[#STRIDE::TestPoint|STRIDE::TestPoint]], automatically passing the default TestCase() as the test case if none is provided. Options are passed using hash-style arguments. The supported arguments are:
; ordered : flag that specifies whether or not the expectation set must occur in the order specified. If set to true, the expectation list is treated as an ordered list, otherwise we assume the list is unordered. Default value is true (ordered processing).
; strict : flag that specifies if the list is exclusive. If strict is specified, then the actual events (within the specified universe of test points) must match the expectation list exactly as specified. If strict processing is disabled, then other test points within the universe are allowed to occur between items specified in the expectation list. The default value is true (strict processing).
; continue : flag that specifies whether the expectation should continue to process test points even after the specified expectation has been minimally satisfied. Normally, a test will complete as soon as the expectation is satisfied. In some circumstances, however, it is necessary to continue waiting until the full timeout period (specified to the Wait function) before exiting. The default value is 0, indicating that the test will exit as soon as it is satisfied (or otherwise fails).
; expected : is an array reference containing elements that are either strings representing the test point labels '''OR''' an anonymous sub-array that contains these four items: '''[label, count, predicate, expected_data]'''. The four element arrayref elements are:
* ''label'' is the test point label (same as the non array argument type). the label is also allowed to be specified using one of the predefined constant values: '''TEST_POINT_ANY_IN_SET'''<ref name="const"/> or '''TEST_POINT_ANY_AT_ALL'''. '''TEST_POINT_ANY_IN_SET''' is used to indicate a place in the expectation list where '''any''' test point that's otherwise explicitly defined in the set is permitted. '''TEST_POINT_ANY_AT_ALL''' is used to indicate a place in the expectation where '''any''' test point in the system is allowed. When either special value is used in the expectation list, a predicate must ''also'' be specified. The test point is only considered satisfied when the predicate returns true. These special values can be used to implement, among other things, startup scenarios where you want to defer your expectation list processing until a particular test point and/or data state have been encountered. When specifying either value in an '''unordered''' expectation, it is only allowed to appear '''once''' in the expectation list and, in that case, it is treated as a startup expectation whereby none of the other test points are processed until the startup expectation has been satisfied (when it's predicate returns true).
* ''count'' is the number of expected occurrences - can be any positive integer value, or the special value '''TEST_POINT_ANY_COUNT'''<ref name="const"/>.
* ''predicate'' must be a perl coderef for a function to call as a predicate. You are free to define your own predicate function OR use on the three [[#Predicates | standard ones]] provided by STRIDE::Test.
* ''expected_data'' will be passed as an argument to the predicate - intended to be used to specify expected data.
If using the array form of expectation, only the label entry is required - the remaining elements are optional.
; unexpected : is an array reference containing labels that are to be treated as failure if they are encountered. For either ''expected'' and ''unexpected'', the special value '''TEST_POINT_EVERYTHING_ELSE'''<ref name="const"/> can be used alone to indicate that any test points not explicitly listed in the set are considered part of this set.
; expect_file : if you have previously captured YAML trace data in a file (using the [[Stride Runner]]), you can specify the file as the source of expectations using this parameter. If you specify a YAML trace file, you should NOT also specify the '''expected''' items as they will be overridden by the YAML trace file.
; predicate : is a perl coderef to a default predicate function to use for all expectation items. If any specific entry in the expectation list has a predicate function, the expectation's predicate will override this global value. By default, no global predicate is assumed and no predicate is called unless specified for each expectation item.
; replay_file : allows you to specify a YAML trace file and '''''input''''' to the current test. If specified, the expectation will be validated against the events specified in the file rather than live events generated by a device under test.
; test_case : allows you to specify a test case to use for reporting the results. This is only useful for advanced users that are generating test cases dynamically within a test method.

The returned object is of type [[#STRIDE::TestPoint|STRIDE::TestPoint]] and has access to all it's member functions.

|}

== Assertions ==

Each of the following assertion methods are provided for standard comparisons. For each, there there are three different types, depending on the desired behavior upon failure: '''EXPECT''', '''ASSERT''', and '''EXIT'''. '''EXPECT''' checks will fail the current test case but continue executing the test method. '''ASSERT''' checks will fail the current test case and exit the test method immediately. '''EXIT''' checks will fail the current test case, immediately exit the current test method AND cease further execution of the test module.

{| class="prettytable"
| colspan="2" | '''Boolean'''

|-
! macro !! Pass if
|-
| '''''prefix'''''_TRUE(''cond'');
| ''cond'' is true

|-
| '''''prefix'''''_FALSE(''cond'');
| ''cond'' is false

|}

{| class="prettytable"
| colspan="2" | '''Comparison'''
|-
! macro !! Pass if

|-
| '''''prefix'''''_EQ(''val1'', ''val2'');
| ''val1'' == ''val2''

|-
| '''''prefix'''''_NE(''val1'', ''val2'');
| ''val1'' != ''val2''

|-
| '''''prefix'''''_LT(''val1'', ''val2'');
| ''val1''<nowiki> < </nowiki>''val2''

|-
| '''''prefix'''''_LE(''val1'', ''val2'');
| ''val1''<nowiki> <= </nowiki>''val2''

|-
| '''''prefix'''''_GT(''val1'', ''val2'');
| ''val1'' > ''val2''

|-
| '''''prefix'''''_GE(''val1'', ''val2'');
| ''val1'' >= ''val2''

|}

For all of the value comparison methods ('''_EQ''', '''_NE''', etc.), the comparison is numeric if both arguments are numeric -- otherwise the comparison is a case sensitive string comparison. If case insensitive comparison is needed, simply wrap both arguments with perl's builtin '''lc()''' (lowercase) or '''uc()''' (uppercase) functions.

{| class="prettytable"
| colspan="2" | '''Predicates'''
|-
! macro !! Pass if

|-
| '''''prefix'''''_PRED(''coderef'', ''data'')
| ''&coderef''(''data'') returns true. The predicate function is specified by ''coderef'' with optional data ''data''. The predicate can also return the special value '''TEST_POINT_IGNORE'''<ref name="const"/>' to indicate that the event should be ignored.

|}

Each of these expectation methods also supports the following optional named arguments:
; test_case => case : allows you to apply the check to a test case other than the current default
; message => "message" : allows you to specify an additional message to include if the check fails.

Because these arguments are optional, they are passed using named argument (hash-style) syntax after the required parameters that are shown above.

== Annotations ==

The following methods can be used to annotate a test case. Typically these methods are used to add additional information about the state of the test to the report.

{| class="prettytable"
| colspan="2" | '''Annotation Methods'''

|-valign="top"
! name !! description

|-valign="top"
| <source lang="perl">NOTE_INFO(message)</source>
| creates an info note in your test results report.

|-valign="top"
| <source lang="perl">NOTE_WARN(message)</source>
| creates a warning note in your test results report.

|-valign="top"
| <source lang="perl">NOTE_ERROR(message)</source>
| creates an error note in your test results report.

|}

Each of these note methods also supports the following optional named arguments:
; test_case => case : allows you to add the note to a test case other than the current default
; file => file : allows you to attach a file along with the annotation message.
; test_point => test_point_hashref : If you are annotating your report in the context of a predicate with a specific test point, you might want to specify the test point using this parameter. This will cause your annotation to be grouped in the final report with the annotation message that corresponds to the test point hit message. By default, a host timestamp value is used to generate the NOTE annotation, which generally causes the NOTE annotations to group toward the end of the test case report.

Because these arguments are optional, they are passed using named argument (hash-style) syntax after the required parameters that are shown above.

== Documentation ==

We have preliminary support for documentation extraction in the test modules using the standard perl POD formatting tokens.

The POD that you include in your test module currently must follow these conventions:

* it must begin with a ''head1 NAME'' section and the text of this section must contain the name of the package, preferably near the beginning.
* a ''head1 DESCRIPTION'' can follow the ''NAME'' section. If provided, it will be used as the description of the test suite created for the test unit.
* This NAME/DESCRIPTION block must finish with an empty ''head1 METHODS'' section.
* each of the test methods can be documented by preceding them with a ''head2'' section with the same name as the test method (subroutine name). The text in this section will be used as the testcase description.

== Predicates ==

STRIDE expectation testing allows you to specify predicate functions for sophisticated data validation. We provide several standard predicates in the STRIDE::Test package, or you are free to define your own predicate functions.

=== Builtin Predicates ===
The STRIDE::Test library provides a few standard predicates which you are free to use in your expectations:

{| class="prettytable"
| colspan="2" | '''Built-In Predicates'''

|-valign="top"
! predicate !! description

|-valign="top"
| '''TestPointStrCmp'''
| does a case '''''sensitive''''' comparison of the test point data and the ''expected_data'' (specified as part of the expectation)

|-valign="top"
| '''TestPointStrCaseCmp'''
| does a case '''''insensitive''''' comparison of the test point data and the ''expected_data''

|-valign="top"
| '''TestPointMemCmp'''
| does a bytewise comparison of the test point data and the ''expected_data''

|-valign="top"
| '''TestPointDefaultCmp'''
| pass-through function that calls '''TestPointMemCmp''' for binary test point data or '''TestPointStrCmp''' otherwise. This is useful as a global predicate since it implements an appropriate default data comparison.

|}

=== User Defined Predicates ===
User defined predicates are subroutines of the following form:

<source lang="perl">
sub myPredicate
{
my ($test_point, $expected_data) = @_;
my $status = 0;
# access the test point data as $test_point->{data},
# and the label as $test_point->{label}

# set $status according to whether or not your predicate passes
return $status;
}
</source>

The predicate function is passed two arguments: the current test point and the expected data that was specified as part of the expectation. The test point data is a reference to a hash with the following fields:
; label : the test point label
; data : the data payload for the test point (if any)
; data_as_hex : an alternate form of the data payload, rendered as a string of hex characters
; size : the size of the data payload
; bin : flag indicating whether or not the data payload is binary
; file : the source file for the test point
; line : the line number for the test point

The expected data is passed as a single scalar, but you can use references to compound data structures (hashes, arrays) if you need more complex expected data.

The predicate function should return a true value if it passes, false if not, or '''TEST_POINT_IGNORE'''<ref name="const"/> if the test point should be ignored completely.

= STRIDE::Remote =

The ''STRIDE::Remote'' class uses perl '''AUTOLOAD'''-ing to provide a convenient syntax for making simple function calls and retrieving database constants in perl. Given any properly initialized ''STRIDE::Test'' object, any captured function or constant (macro) is available directly as method or properties of the exported ''Remote'' object. Constants can also be accessed via the tied hash ''constants'' member.

For example, given a database with two functions and a macro:

<source lang="c">
int foo(const char * path);
void bar(double value);

#define MY_PI_VALUE 3.1415927
</source>

In perl these methods/constants are invokable using the exported ''STRIDE::Test'' Remote object:

<source lang="perl">
my $retval = Remote->foo("my string");
Remote->bar(Constants->{MY_PI_VALUE});
</source>

== Asynchronous invocation ==

Functions can also be called asynchronously by using the ''async'' delegator within the ''Remote'' object. When invoked this way, the function call will return a handle object that can be used to wait for the function return value - for example:

<source lang="perl">
my $h = Remote->async->foo("my string");
my $retval = $h->Wait(1000);
</source>

The '''Wait''' function takes one optional argument -- the timeout duration (in milliseconds) that indicates the maximum time to wait for the function to return. If the timeout value is not provided, ''Wait'' will wait indefinitely for the function to return. If a timeout is specified and expires before the function returns, the method with ''die'' with a timeout error message - so you might want to wrap your '''Wait''' call in an <tt>eval{};</tt> statement if you want to gracefully handle the timeout condition.

= STRIDE::TestPoint =

STRIDE::TestPoint objects are used to create test point expectation tests. These objects are created using the exported TestPointSetup factory function of the STRIDE::Test class. Once a STRIDE::TestPoint object has been created with the desired expectations, two functions can be called:

; Wait(timeout) : This method processes test points that have occurred on the target and assesses failure based on the parameters you provided when creating the TestPoint object. The timeout parameter indicates how long (in milliseconds) to Wait for the specified events. If no timeout value is provided, '''Wait''' will proceed indefinitely or until a clear pass/failure determination can be made.
; Check() : this is equivalent to Wait with a very small timeout. As such, it essentially verifies that your specified test points have already been hit.

= Reporting Model =

The STRIDE perl framework includes an implementation of our [[Reporting Model]] that is common across all STRIDE components. The Test module gives explicit access to key elements of the report model, Cases and Annotaions. Here are a description of the methods available for each of these Objects.

== TestCase ==

{| class="prettytable"
| colspan="2" | '''Methods'''

|-valign="top"
! name !! description

|-valign="top"
| <source lang="perl">SetStatus(status, duration) </source>
| sets the test case status. The value of ''status'' could be one of the predefined constants:
* '''TEST_FAIL'''<ref name="const"/>
* '''TEST_PASS'''
* '''TEST_NOTINUSE'''
* '''TEST_INPROGRESS'''
* '''TEST_DONE''' - applicable to dynamic cases - sets the status to '''pass''' unless already set to '''fail''' or '''not-in-use'''

|-valign="top"
| <source lang="perl">TestAnnotation AddAnnotation(level, name, description)</source>
| creates a new test annotation and adds it to the test case. The value of ''level'' could be one of the predefined consttants:
* '''ANNOTATION_TRACE'''<ref name="const"/>
* '''ANNOTATION_DEBUG'''
* '''ANNOTATION_INFO'''
* '''ANNOTATION_WARNING'''
* '''ANNOTATION_ERROR'''
* '''ANNOTATION_FATAL'''

|-valign="top"
| <source lang="perl">SetData(name, value)</source>
| associates a custom name-value pair with the test case.

|}

== TestAnnotation ==

{| class="prettytable"
| colspan="2" | '''Methods'''

|-valign="top"
! name !! description

|-valign="top"
| <source lang="perl">AddComment(label, message)</source>
| adds a new comment to the test annotation.

|}

== Notes ==
<references/>

Test Script

2015-07-06T20:46:48Z

Marku: /* Running a Test Script */

Stride's test script solution allows users to take advantage of the power of dynamic languages on the host to execute test scenarios on the device under test. Some of the advantages of host based script tests are:
* '''test coverage''' can be '''expanded''' without any changes to the target device.
* sophisticated '''device automation''' possible using facilities available on the host.
* no additional '''code space''' requirements on the device once the software under test.
* powerful '''data validation''' techniques are possible using the power of the host (validating large media files generated by the target device, for example).
* large library of '''extension modules''' are available for standard dynamic languages (e.g. [http://search.cpan.org cpan for perl])

'''Example Test Script (i.e. MyTest.pm)'''

<source lang="perl">
use strict;
use warnings;

package MyTests;
use base qw(STRIDE::Test);
use STRIDE::Test;

sub test_one : Test
{
ASSERT_TRUE(1);
}

sub test_two : Test
{
ASSERT_TRUE(0);
}

1;
</source>

== Running a Test Script ==
Stride executes tests using a runner controlled by a host computer. Refer to [[Running Tests]] for Test Units.

'''Invoking the Runner (aka <tt>[[STRIDE_Runner|stride]]</tt>) from a console'''

stride .. --run=MyTest.pm

Test Log

2015-07-06T20:36:43Z

Marku:

__NOTOC__
Test Log macros provide a simple means to add information from the source under test to the currently executing test case. This information is added to the test report as annotations with a level of either ''Error'', ''Warning'', or ''Info'' according to the macro that is used. The log messages are also captured when tracing is enabled in the [[Stride Runner]].

The <tt>srTEST_LOG_xx()</tt> macro is intended to be a general instrumentation macro for capturing information in the source under test. Logs differ from test points in that they cannot be used for the basis of expectation tests - they are strictly informational. The Stride log messages are intended to supplement, not supplant a [[Test Point | Test Points]].

= Reference =
To use these macros you should include the '''srtest.h''' header file from the STRIDE Runtime in your compilation unit. The Test Log macros are active only when <tt>STRIDE_ENABLED</tt> is <tt>#define</tt>d, therefore it is practical to place these macros in-line in production source. When <tt>STRIDE_ENABLED</tt> is not <tt>#define</tt>d, these macros evaluate to nothing.

{| class="prettytable"
| colspan="2" | '''Error Logging'''
|-
| srTEST_LOG_ERROR(''message'', ...)
| ''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string
|}

{| class="prettytable"
| colspan="2" | '''Warning Logging'''
|-
| srTEST_LOG_WARN(''message'', ...)
| ''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string
|}

{| class="prettytable"
| colspan="2" | '''Info Logging'''
|-
| srTEST_LOG_INFO(''message'', ...)
| ''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string
|}

* The maximum length of the message string approximately 1000 characters. If the maximum length is exceeded, the message string is truncated.

=== C++ Only Features ===
In C++ source the macros above support adding to other content to the message by using the << operator.

=== Log Level ===

By default only logs with level of either ''Error'' or ''Warning'' are propagated to the host. The [[STRIDE Runner#Options | Stride Runner]] via <tt>--log_level</tt> ''option'' provides finer control over that behavior.

== Code Snippets ==
<source lang='c'>
#include <srtest.h>
...
srTEST_LOG_ERROR("This is an error message.");

srTEST_LOG_WARN("This is a warning message with format string %d.", 123);

srTEST_LOG_INFO("This is an info message with format string %s and %s.", "this", "that");

#ifdef __cplusplus
srTEST_LOG_ERROR("some error: ") << "stream input supported under c++";
#endif
</source>

Test Point Testing in C/C++

2015-07-06T20:33:01Z

Marku:

__NOTOC__
Test code to validate the [[Expectations]] of the Test Points often follows this basic pattern:

# Specify an expectation set consisting of expected (i.e. the test points that are expected to be hit) and optionally unexpected (i.e. the test points that are not expected to be hit) test points
# Register the expectation set with the STRIDE runtime
# Invoke the software under test (causing instrumentation points to be hit). This may not be necessary if the instrumented software under test is constantly running).
# Wait for the expectation set to be satisfied or a timeout to occur

Here is an example:

<source lang="c">
#include <srtest.h>

void tf_testpoint_wait(void)
{
/* specify expected set */
srTestPointExpect_t expected[]= {
{"START"},
{"ACTIVE"},
{"IDLE"},
{"END"},
{0}};

/* specify unexpected set */
srTestPointUnexpect_t unexpected[]= {
{"INVALID"},
{0}};

/* register the expectation set with the STRIDE */
srWORD handle;
srTestPointSetup(expected, unexpected, srTEST_POINT_EXPECT_UNORDERED, srTEST_CASE_DEFAULT, &handle);

/* start your asynchronous operation */
...

/* wait for expectation set to be satisfied or a timeout to occur */
srTestPointWait(handle, 1000);
}

#ifdef _SCL
#pragma scl_test_flist(“testfunc”, tf_testpoint_wait)
#endif
</source>

== Reference ==

=== Expectation Set ===
An expectation set is specified with an [[Expectations#Expected_List|expected]] array of '''srTestPointExpect_t''' structures and a second optional [[Expectations#Unexpected_List|unexpected]] array of '''srTestPointUnexpect_t''' structures.

==== Expected Array ====
srTestPointExpect_t is typedef'd as follows:

<source lang='c'>
typedef struct
{
/* the label value is considered the test point's identity */
const srCHAR * label;
/* optional, count specifies the number of times the test point is expected to be hit */
srDWORD count;
/* optional, predicate function to use for payload validation against user data */
srTestPointPredicate_t predicate;
/* optional, user data to validate the payload against */
void * user;
} srTestPointExpect_t;
</source>

'''NOTES:'''
* The end of the array has to be marked by a srTestPointExpect_t set to all zero values
* The ''count'', ''predicate'' and ''user'' members may be omitted in the array declaration (they will be automatically set to 0 by the compiler)
* A ''count'' value of either 0 or 1 is interpreted as 1
* The ''count'' could be set as "0 or more" by using the [[Expectations#Special_Processing|special]] srTEST_POINT_ANY_COUNT symbolic constant
* A ''predicate'' value 0 indicates that any associated data with a test point payload will be ignored.
* A ''user'' value 0 indicates that there is no user data associated with this test point
* The ''label'' could be specified to ''any test point'' within the current expected set of test points by using the [[Expectations#Special_Processing|special]] srTEST_POINT_ANY_IN_SET or srTEST_POINT_ANY_AT_ALL symbolic constants.

==== Unexpected Array ====
srTestPointUnexpect_t is typedef'd as follows:

<source lang='c'>
typedef struct
{
/* the label value is considered the test point's identity */
const srCHAR * label;
} srTestPointUnexpect_t;
</source>

'''NOTES:'''
* The end of the array has to be marked by a srTestPointUnexpect_t set to all zero values
* The ''label'' could be specified to ''everything else'' relative to the '''expected''' array by using the [[Expectations#Special_Processing|special]] srTEST_POINT_EVERYTHING_ELSE symbolic constant.

==== srTestPointPredicate_t ====
When defining the expectation set per entry a [[Expectations#State_Data_Validation|payload validation]] predicate function could be specified. The signature of it should match the following type:

<source lang="c">
extern "C" typedef srBYTE (*srTestPointPredicate_t)(const srTestPoint_t* ptTP, void* pvUser);
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| ptTP
| Input
| Pointer to the currently processed Test Point.
|-
| pvUser
| Input
| Pointer to opaque user data associated with an entry in the expectation set.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srBYTE
| srTRUE for valid, srFALSE for invalid, srIGNORE otherwise.
|}

'''NOTE:'''
* As part of the standard STRIDE distribution there are three predefined function predicate helpers:
** srTestPointMemCmp - byte comparison
** srTestPointStrCmp - string case sensitive comparison
** srTestPointStrCaseCmp - string case insensitive comparison

==== srTestPointSetup ====
The srTestPointSetup() routine is used to register an expectation set.

<source lang="c">
srBOOL srTestPointSetup(srTestPointExpect_t* ptExpected,
srTestPointUnexpect_t* ptUnexpected,
srBYTE yMode,
srTestCaseHandle_t tTestCase,
srWORD* pwHandle);
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| ptExpected
| Input
| Pointer to an expectated array.
|-
| ptUnexpected
| Input
| Pointer to an unexpectated array. This is optional and could be set srNULL.
|-
| yMode
| Input
| Bitmask that specifies whether the expectated test points occur in [[Expectations#Sequencing_Properties|order and/or strict]]. Possible values are: 
srTEST_POINT_EXPECT_ORDERED - the test points are expected to be hit exactly in the defined order 
srTEST_POINT_EXPECT_UNORDERED - the test points could to be hit in any order 
srTEST_POINT_EXPECT_STRICT - the test points are expected to be hit exactly as specified (no consecutive duplicate hits) 
srTEST_POINT_EXPECT_NONSTRICT - other test points from the universe could to be hit in between 
srTEST_POINT_EXPECT_CONTINUE - on successful expectation satisfaction continue processing until the wait timeout expires
|-
| tTestCase
| Input
| Handle to a test case. srTEST_CASE_DEFAULT can be used for the default test case.
|-
| pwHandle
| Output
| Handle that represents the registered expectation set
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srBOOL
| srTRUE on success, srFALSE otherwise.
|}

==== srTestPointWait ====
The srTestPointWait() routine is used to wait for the expectation to be satisfied.

<source lang="c">
srBOOL srTestPointWait(srWORD wHandle,
srDWORD dwTimeout);
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| wHandle
| Input
| Handle to a registered expectation set.
|-
| dwTimeout
| Input
| Timeout value in milliseconds; 0 means just check without waiting.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srBOOL
| srTRUE on success, srFALSE otherwise.
|}

'''NOTES:'''
* The test thread blocks until either the expectation set is satisfied, unless srTEST_POINT_EXPECT_CONTINUE is specified on setup, or the timeout elapses.
* All test points hit during the wait (both expected and unexpected) are added to the test report as testcase comments
* Once the wait is over (whether the expectation set has been satisfied or there has been a test failure), the current expectation set is automatically unregistered from the STRIDE runtime and the handle is released
* If you want to return immediately from a test case if expectation fails then make the check/wait call an argument to the ''srASSERT_TRUE()'' macro.

==== srTestPointCheck ====
The srTestPointCheck() routine is used to check for the expectation post routine completion. This is useful for verifying a set of expectations events that should have already transpired (thus are waiting to be processed).

<source lang="c">
srBOOL srTestPointCheck(srWORD wHandle);
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| wHandle
| Input
| Handle to a registered expectation set.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srBOOL
| srTRUE on success, srFALSE otherwise.
|}

'''NOTES:'''
* All test points hit before the check (both expected and unexpected) are added to the test report as testcase comments
* Once the check is done (whether the expectation set has been satisfied or there has been a test failure), the current expectation set is automatically unregistered from the STRIDE runtime and the handle is released
* If you want to return immediately from a test case if expectation fails then make the check/wait call an argument to the ''srASSERT_TRUE()'' macro.

=== C++ Facade Class ===
The ''srtest.h'' file provides a simple [http://en.wikipedia.org/wiki/Facade_pattern facade] class that wraps the test point APIs described above in a simple C++ class called '''srTestPointsHandler'''. If you are writing your unit tests in C++ (using STRIDE test classes), then this class is available for your use. The class implements the following methods, which correspond exactly to the C API equivalents:

; srTestPointsHandler : constructor. Takes four arguments which match exactly the first four parameters of the [[#srTestPointSetup|srTestPointSetup]] function.
; Wait : instance method that provides same functionality as [[#srTestPointWait|srTestPointWait]].
; Check : instance method that provides same functionality as [[#srTestPointCheck|srTestPointCheck]].

Expectations

2015-07-06T20:27:31Z

Marku:

__NOTOC__
STRIDE supports '''Behavior Testing''', which is different than Unit Testing or API Testing in that it does not focus on calling functions and validating return values. Behavior Testing validates the ''expected sequencing and state of the software'' executing under normal operating conditions. In order to begin this type of testing, you must '''(1)''' instrument the source under test with [[Test Point | Test Points]] and '''(2)''' define the ''expectations'' of the Test Points for each test scenario.

The '''Validation Model''' is based on validating that the ''Expected List'' of Test Points have been hit as well as optionally validating any ''state data'' associated with a Test Point member. Through the use of an ''Unexpected List'', it's possible to simulataneously validate that specified Test Points are NOT hit during the execution of the scenario.

The ''Expected List'' and ''Unexpected List'' declared for a specific testing scenario define the ''active'' set of Test Points for the system. All other Test Points within the software are not active and have nominal impact.

= Expected List =

The first step in defining '''Expectations''' is describing the set of Test Points expected to be hit during a test scenario. This is the '''list''' of Test Points and the expected number of hits ('''count''') for each of the Test Points. Any expected '''state data''' associated with a Test Point that requires validation should be included.

The validation of the ''Expected List'' is done when any of the following conditions are met:
* The ''Expected List'' sequencing has been met completely
* The time allocated for validation has expired
* State data associated with a Test Point is declared invalid
* An out-of-sequence Test Point has been encountered (a concrete failure is identified)
* An [[Expectations#Unexpected_List | Unexpected Test Point]] has been encountered

'''Expected TEST POINTS list:'''

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Label'''
| '''Count'''
| '''Expected State Data'''
|-
| Name 1
| 1
| ''<describe data payload validation requirements if applicable>''
|-
| Name 2
| 1 +
| ''<describe data payload validation requirements if applicable>''
|-
| ''name ...''
| ''n''
| ''<...>''
|}

=== Sequencing Properties ===

The ''Expected List'' requires defining ''sequencing properties'' used for the validation. This involves establishing how to process the '''ordering''' of the Test Points being hit, how '''strict''' to handle duplications of Test Point members, and if to '''continue''' capturing Test Points for the entire time allocated for the test scenario even though the ''Expected List'' sequencing requirements have been met.

'''Expected TEST POINTS sequencing properties:'''

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Properties'''
| ''' Description'''
|-
| ''Ordered''
| Test Points are expected to be hit in the exact order defined in the list
|-
| ''Unordered''
| Test Points can be hit in any order defined in the list
|-
| ''Strict''
| Test Points specified in the list must match the exact count (i.e. no duplication)
|-
| ''Non-Strict''
| Test Points specified in the list can be duplicated anywhere in the sequence
|-
| ''Continue''
| Test Points validation will continue even though the ''Expected List'' has been met ''so far''
|}

=== State Data Validation ===

State data validation for a Test Point member is optional. The ''Expected List'' can associate a ''predicate''<ref name="predicate">Defined as a necessary condition of the Test Point being valid -- logic within the predicate determines if ''valid'', ''invalid'', or ''ignored'' </ref> as a necessary condition for validation of a Test Point member when being processed. The predicate provides an extra level of validation (beyond sequencing) focusing on the state data associated with the Test Point.

The predicate can declare the Test Point:
* Valid - successful, thus the expected count is decremented
* Invalid - the expectations is NOT met, indicating failure
* Ignored - no affect on the validation, continue to wait on the current Test Point

= Unexpected List =

An ''Unexpected List'' of Test Points can optionally be defined. This list works in conjunction with the [[Expectations#Expected_List | ''Expected List'']]. The list defines a set of Test Points that are to be treated as failures if any one of them is encountered (hit) during a testing scenario.

This list can be made up of the '' '''complement''' '' of the ''Expected List'', thus including ''all'' Test Points not included in that defined set. This type of setting requires enabling all Test Points defined in the software during the execution of the test.

If the ''Unexpected List'' members intersect with the ''Expected List'' members, that is considered an input error.

= Special Processing =

=== Members ===
There are two ''special members'' that can be used for the ''Expected List'' called '''ANY IN SET''' and '''ANY AT ALL'''. These special member values are used to customize the success criteria (i.e. trigger condition) for the entry defined within the ''Expected List''. Both types require a predicate<ref name="predicate"/> to determine if the Test Point is ''valid, invalid, or ignored''. The '''ANY IN SET''' refers to the set of points defined by the ''Expected List'' while the '''ANY AT ALL''' refers to all Test Points in the system. By definition the '''ANY AT ALL''' member requires enabling all Test Points defined in the software during the execution of the test.

Concerning an '''Unordered''' ''Expected List'', only one special member can be used. It will be processed first independent of its location within the list. The '''Count''' attribute is also optionally available to be used with the special members, indicating the number of valid returns from the predicate until the next Test Point in the list is validated.

There is one ''special member'' that can be used for the ''Unexpected List'' called '''EVERYTHING ELSE'''. This special member can only be used as a '''single member only''' within an ''Unexpected List''. It takes the ''complement'' of the set of Test Points defined within the ''Expected List''. This requires enabling all Test Points defined in the software during the execution of the test. Also, by definition any non-expected Test Point hit during the test would be deemed a failure.

=== Counts ===
There is one ''special count'' value that can be associated with a Test Point defined in the ''Expected List'' called '''ANY COUNT'''. This special count is used to focus on capturing a variable number of Test Point hits -- 0 or more. In essences the Test Point has no consequences of the Expected List sequencing, except if a predicate<ref name="predicate"/> associated with the Test Point returns ''invalid''.

<hr>

= Use Cases =

The following use cases provide examples of defining '''Expectations''' based on different testing scenarios.

=== Sequencing Properties ===
Use case examples for validating the [[Expectations#Sequencing_Properties | sequencing]] of Test Points.

Expecting an exact ordered sequence of Test Points to be hit
''Expect'' = [A,B,C,D]; ''ORDERED, STRICT''
Hits = [A,B,C,D] - PASS
Hits = [A,B,D,C] - FAIL (D hit before C)

Expecting an exact ordered of Test Points but with duplicates
''Expect'' = [A,B,C,D]; ''ORDERED, STRICT''
Hits = [A,B,B,C,D] - FAIL (B duplicated with strict defined)

''Expect'' = [A,B,C,D]; ''ORDERED, NON-STRICT''
Hits = [A,B,B,C,D] - PASS
Hits = [A,C,B,C,D] - PASS
Hits = [A,B,C,B,D] - PASS

Expecting a set of Test Points to be hit, but the order does not matter
''Expect'' = [A,B,C,D]; ''UNORDERED, STRICT''
Hits = [B,A,C,D] - PASS
Hits = [B,A,C,B,D] - FAIL (B hit twice)

''Expect'' = [A,B,C,D]; ''UNORDERED, NON-STRICT''
Hits = [B,A,C,B,D] - PASS

Expecting a set of Test Points to be hit, but order and duplications does not matter
''Expect'' = [A,B,C,D]; ''UNORDERED, NON-STRICT''
Hits = [B,A,C,B,D] - PASS
Hits = [B,A,A,A,D] - FAIL (C never hit)

Expecting an exact ordered sequence of Test Points to be hit, but checking for any ''extra'' hits for total duration of test scenario
''Expect'' = [A,B,C,D]; ''ORDERED, STRICT'', ''CONTINUE''
Hits = [A,B,C,D] - PASS
Hits = [A,B,C,D,A] - FAIL (A hit after valid sequence met based on test timeout)

=== Count Attribute ===
Use case examples that leverage the [[Expectations#Count | Count]] attribute associated with a list member.

Expecting a sequence of Test Points, including multiple hits for 2 members
''Expect'' = [A,B:3,C,D:2]; ''ORDERED, STRICT''
Hits = [A,B,B,B,C,D,D] - PASS
Hits = [A,B,B,C,B,D,D] - FAIL (B not hit three times before C)

''Expect'' = [A,B:3,C,D:2]; ''ORDERED, NON-STRICT''
Hits = [A,B,B,B,C,B,D,D] - PASS

Expecting a set of Test Points being hit, including multiple hits (with duplications), but order does not matter
''Expect'' = [A,B:3,C,D]; ''UNORDERED, STRICT''
Hits = [B,A,B,C,B,D] - PASS
Hits = [B,A,B,B,C,B,D] - FAIL (B duplicated on 4th hit)

''Expect'' = [A,B:3,C,D]; ''UNORDERED, NON-STRICT''
Hits = [B,A,B,B,C,B,D] - PASS

=== State Data Validation ===
Use case examples leveraging predicates<ref name="predicate"/> to [[Expectations#State_Data_Validation | validate state data]].

Simple example showing associating a predicate with Test Point named ''A''.
''Expect'' = [A:p1(tp),B,C,D]; ''ORDERED, STRICT''
p1(tp) { return VALID }
Hits = [A,B,C,D] - PASS

Expecting Test Point ''B'' to be hit 3 times
''Expect'' = [A,B:3:p1(tp),C,D]; ''UNORDERED, STRICT''
p1(tp) { return VALID unless 3rd time called than return INVALID}
Hits = [B,A,B,C,B,D] - FAIL (predicate returns invalid on 3rd B hit)

=== Special Members ===
Use case examples using the two [[Expectations#Members | special members]] within an ''Expected List''.

Waiting on a single trigger (D) to start the processing of an Expected List
''Expect'' = [ANY_IN_SET:p1(tp),A,B,C:2,D]; ''ORDERED, STRICT''
p1(tp) {return VALID when D hit otherwise IGNORE}
Hits = [A,B,A,D*,A,B,C,C,D] - PASS

Waiting on Test Point ''outside'' of the Expected List to trigger processing
''Expect'' = [ANY_AT_ALL:p1(tp),A,B,C:2,D]; ''ORDERED, STRICT''
p1(tp) {return VALID when x hit}
Hits = [A,B,A,x*,A,B,C,C,D] - PASS

Waiting on a multiple triggers - 1st trigger used to start processing and 2nd used midstream
''Expect'' = [ANY_IN_SET:p1(tp),A,B,ANY_IN_SET:p2(tp),C:2,D,E]; ''ORDERED, STRICT''
p1(tp) {return VALID when D hit}
p2(tp) {return VALID when E hit}
Hits = [A,B,A,D*,A,B,A,A,B,E*,C,C,D,E] - PASS

''Expect'' = [ANY_IN_SET:p1(tp),A,B,C:2,D]; ''UNORDERED, NON-STRICT''
p1(tp) {return VALID when D hit}
Hits = [A,B,A,D*,A,A,A,D,A,C,B,C,D] - PASS

This test uses the ANY AT ALL along with a fix count of 3
''Expect'' = [ANY_AT_ALL:p1(tp):3,A,B,C]; ''ORDERED, STRICT''
p1(tp) {return VALID}
Hits = [A,z,x,A,B,C] - PASS

=== Special Counts ===
Use case examples leveraging the [[Expectations#Special_Counts | special count attributes]] within an ''Expected List''.

Expecting between 0 and more Test Point As to be hit before processing the remaining members.
''Expect'' = [A:ANY_COUNT,B,C,D]; ''ORDERED, STRICT''
Hits = [B,C,D] - PASS
Hits = [A,A,B,C,D] - PASS

This test passes ''immediately'' after the A is hit (example of a weird expectation)
''Expect'' = [A,B:ANY_COUNT]; ''ORDERED, STRICT''
Hits = [A,..] - PASS

This test also passes, but will process all Test Point B hits until the test time expires
''Expect'' = [A,B:ANY_COUNT]; ''ORDERED, STRICT, CONTINUE''
Hits = [A,B,B,B] - PASS (the wait timeout will complete the validation)
Hits = [A] - PASS

This test fails because all Test Points are checked once the test time expires
''Expect'' = [A]; ''ORDERED, STRICT, CONTINUE''
Hits = [A,A] - FAILS (the wait timeout than will complete the validation)

This test validates a set of Test Points focusing exclusively on the associated ''state data''
''Expect'' = [A:p1(tp):ANY_COUNT, B:p2(tp):ANY_COUNT, C:p2(tp):ANY_COUNT]; ''UNORDERED, CONTINUE'' (STRICT NA)
Hits = [A,A,B,A,C,C,C,A,..] - PASS (unless any of the predicates fails)
Hits = [A,B,C,D] - PASS

=== Miscellaneous ===
The following are miscellaneous use cases.

Expecting an exact ordered sequence of Test Points to be hit, but verifying certain Test Points are NOT hit
''Expect'' = [A,B,C,D]; ''ORDERED, STRICT''
''Unexpect'' = [x,y]
Hits = [A,B,C,D] - PASS
Hits = [A,B,B,C,D] - FAIL
Hits = [A,B,C,y,..] - FAIL

This test will always pass and log every Test Point hit during test time duration. Note all Test Points are active.
''Expect'' = [ANY_AT_ALL:p1(tp):ANY_COUNT]; ''CONTINUE {other sequencing properties NA}''
p1(tp) {return VALID}
Hits = [A,A,B,x,y,D, ..] - PASS (the wait timeout will complete the validation)

This test will log every Test Point hit during test, looking for X and Y as failures. Note all Test Points are active.
''Expect'' = [ANY_AT_ALL:p1(tp):ANY_COUNT]; ''CONTINUE {other sequencing properties NA}''
p1(tp) {return VALID}
''Unexpect'' = [X,Y]
Hits = [A,A,B,x,y,D,Y] - FAIL (last Test Point hit causes a failure)

This test will fail on any Test Point that is hit. Note all Test Points are active. Test will last based on time duration or the 1st Test Point hit.
''Expect'' = NULL; ''CONTINUE {other sequencing properties NA}''
''Unexpect'' = [EVERYTHING_ELSE]
Hits = [A,..] - FAIL

This test will ignore any Test Point hit during test, looking for X and Y as failures.
''Expect'' = NULL; ''CONTINUE {other sequencing properties NA}''
''Unexpect'' = [X,Y]
Hits = [..,X,..] - FAIL

This test will fail on any Test Points hit except for A & B. Note all Test Points are active. Test will last until failure or timeout.
''Expect'' = [A,B]; ''UNORDERED, NON-STRICT, CONTINUE''
''Unexpect'' = [EVERYTHING_ELSE]
Hits = [..,A,..,X] - FAIL

== Notes ==
<references/>

Test Point

2015-07-06T20:23:07Z

Marku: /* Write the Test Unit */

__NOTOC__
Source instrumentation is the process by which developers and domain experts selectively instrument the source under test for the purpose of writing test scenarios against the executing application. Implementing tests that leverage source instrumentation is called '''Expectation Testing'''. This validation technique is very useful for verifying proper code sequencing based on the software's internal design.

<source lang='c'>
#include <srtest.h>
...
/* a test point with no payload */
srTEST_POINT("first test point");

/* a test point with binary payload */
srTEST_POINT_DATA("second test point", myData, sizeofMyData);

/* a test point with simple string payload */
srTEST_POINT_STR("third test point", "payload with simple string");

/* a test point with formatted string payload */
srTEST_POINT_STR("third test point", "payload with format string %d", myVar);

#ifdef __cplusplus
srTEST_POINT_STR("c++ test point", "") << "stream input supported under c++";
#endif
</source>

Unlike traditional unit testing that drives testing based on input parameters and isolating functionality, ''Expectation'' testing is executed within a fully functional software build running on a real target platform. Test Points are not dependent on input parameters, but often leverage the same types of input/output controls used by functional and black-box testing.

Another unique feature of this type of testing is that domain expertise is not required to implement a test. Developers and domain experts use instrumentation to export design knowledge of the software to the entire team. Furthermore, there is no stubbing required, no special logic to generate input parameters, and no advanced knowledge required of how the application software is coded.

To enable effective test coverage, developers and domain experts are required to insert instrumentation at key locations to gain insight and testability. Here are some general suggested source code areas to consider instrumenting:
* critical function entry/exit points
* state transitions
* critical or interesting data transitions (using optional payload to convey data values)
* callback routines
* data persistence
* error conditions

The steps required to implement an Expectation test are the following:
#[[#Instrumentation | Instrumentation]]
#[[#Define your Expectations | Define your Expectations]]
#[[#Write the Test Unit | Write the Test Unit]]

== Instrumentation ==
To make the software ''testable'', the first step in the process is for the experts to selectively insert instrumentation macros into the source code. Test Points themselves have nominal impact on the performance of the application – they are only active during test data collection<ref name="n1"> Test data collection is typically implemented in a low priority background thread. The data is captured in the calling routine's thread context (no context switch) but processed in the background or on the host. Instrumentation macros return immediately to the caller (i.e. no-op) when testing is not active.</ref>. Test Points contain names and optional payload data. When Test Points are activated, they are collected in the background, along with timing and any associated data. The set of Test Points hit, their order, timing, and data content can all be used to validate that the software is behaving as expected. [[Test_Log | Test Logs]] can also be added to the source code to provide additional information in the context of an executing test.

To specify a test point you should include the '''srtest.h''' header file from the STRIDE Runtime in your compilation unit. The Test Point macros are active only when <tt>STRIDE_ENABLED</tt> is <tt>#define</tt>d, therefore it is practical to place these macros in-line in production source. When <tt>STRIDE_ENABLED</tt> is not <tt>#define</tt>d, these macros evaluate to nothing.

{| class="prettytable"
| colspan="2" | '''Test Point Macros'''
|-valign="top"
| '''srTEST_POINT'''(''label'')
| ''label'' is a pointer to a null-terminated string

|-valign="top"
| '''srTEST_POINT_DATA'''(''label'', ''data'', ''size'')
| ''label'' is a pointer to a null-terminated string 
''data'' is a pointer to a byte sequence 
''size'' is the size of the ''data'' in bytes

|-valign="top"
| '''srTEST_POINT_STR'''(''label'', ''message'')
| ''label'' is a pointer to a null-terminated string 
''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string 
When used in the context of a c++ compilation unit, this macro also supports the streaming operator to append to the message string (see example below)
|}

== Define your Expectations ==

In addition to instrumenting the source code with Test Points, you must also define the [[Expectations]] of the Test Points. This involves defining the list of Test Points expected to be hit during a given test scenario. Expectation can also include any '''data''' associated with a Test Point that requires validation.

== Write the Test Unit ==
Once the source under test has been instrumented and the [[Expectations]] defined, Stride offers a number of techniques that can be used for implementing '''expectation tests'''.

* Tests can be written in [[Expectation_Tests_in_C/C%2B%2B | C or C++]] and executed on the device under test using the Stride framework.
* Tests can be written in [[Perl_Script_APIs | Perl]].

== Notes ==
<references/>

Name Mangling

2015-07-06T20:12:15Z

Marku:

== What is Name Mangling? ==

Name mangling describes the automatic source transformation required for an interceptor to insert itself between caller and callee. There are two options for handling name mangling, one of which must be chosen when using interceptors:

*A call to a function f() is transformed to "__im_f()” to allow interception the call. This is referred to as '''reference mangling'''.

*The implemention of f() can be transformed to "__im_f()”. This is referred to as '''definition mangling'''.

For example, to use a interceptor to trace all calls to a function, f():

*If we use '''reference mangling''', all calls to f() are transformed to be calls to __im_f(). The interceptor is implemented as __im_f(). The original implementation of f() remains unchanged. This allows interception of all calls and takes the appropriate action.

*If we use '''definition mangling''', the implementation of f() is changed from f(){...} to __im_f(){...}. The interceptor is implemented as f() in order to intercept all calls.

The name mangling is either done '''implicitly''' or '''explicitly''' during the compile process. By default, '''implicit''' mangling is used; this means that selected routines will be mangled by using a chosen [[Intercept_Module#Group_ID|Group ID]] and including the Intercept Module header file. Where '''explicit''' mangling requires the additional step of adding a macro.

== Name Mangling Conflicts ==

A mangling conflict applies only to interceptors. It exists when the name of a routine requires mangling, but there also exists other references to the same name within the same source file. The other references will inadvertently be mangled by the preprocessor.

'''Explicit mangling''' is used to resolve these type of name mangling conflicts. Explicit mangling uses the explicit macro "'''srTEST_INTERCEPT('''<function_name>''')'''", which explicitly mangles the name of "f()" to "__im_f()". The explicit macro is defined in the Intercept Module name mangling header file (xxxIM.h). Explicit mangling requires [[Intercept Module#Group ID|Group ID(s)]] to be defined (#define MyGroupID) in the source file of the caller/callee of the routine to protect against the inclusion of unnecessary defined types.

The following examples demonstrate situations where explicit mangling can be used to resolve mangling conflicts:

==== ''The reference and the definition of the same routine, f(), exist in the same source file'' ====

If the reference and the definition of the same routine, f(), exist in the same source file, the use of explicit mangling is required to resolve the mangling conflict. If the conflict is not resolved, the user will bypass the interceptor by using the mangled name when making the call to the routine.

==== ''Two (or more) references of a routine, f(), exist in the same source file, but only one is a reference interceptor'' ====

By default, interceptors are definition-focused. An referenced-focused interceptor is used to mangle the actual routine; thus, every caller of the routine goes through the interceptor. There are use cases when source code for the routines cannot be accessed (i.e., as with libraries). A referenced-focused interceptor can be used to mangle the caller instead of the routine to be called (callee). Reference-focused interceptors typically do not have mangling conflicts, except when multiple callers exist in the same source file, but only a subset is targeted to go through the interceptor. Explicit mangling is used to avoid this type of mangling conflict.

The Group ID(s) must be defined before including the Intercept Module name mangling header file (xxxIM.h) and after all defined types (additional header files) in the source file(s) of the caller/callee of the routine. 
<source lang="c">
#include "MyHeader.h"
#define MyGroupID
#include "xxxIM.h"
</source>

To enable Intercept Modules to be regenerated continuously and to allow the user the option to add and/or remove interceptors, an [[#Explicit_Mangling|explicit macro]] is generated for each routine. If a routine is not selected as an interceptor, an explicit macro will be generated so that name mangling will not occur. Also, if use of the Intercept Module is turned off via the source file (e.g., <tt>#undef STRIDE_ENABLED</tt> or <tt>#define STRIDE_ENABLED 0</tt>), the macros will not perform name mangling. This allows users to leave explicit macros in their source code, even if the Intercept Module is not being used.

Scl function

2015-07-06T20:09:44Z

Marku:

__NOTOC__
The ''SCL function'' pragma allows capturing a global function for doubling or remoting. During the build process the [[Build Tools | Stride Build Tools]] generates [[Intercept Module]] that provides the correct logic for the test.

<source lang=c>
int foo(int x);

void boo(void);

#pragma scl_function(foo)
#pragma scl_function(boo, "DEFINITION", "IMPLICIT", "TEST_GROUP")
</source>

=== Syntax ===
The ''scl_function'' pragma allows the user to capture a function. When captured for the purpose of interception ('''intercept-able''') the optional arguments are used to specify how the intercept should be executed. The [[s2sinstrument|Instrument Build Tool]] will use those options to generate appropriate function interception code.

#pragma scl_function(function-name [,context, name-mangling, group-id])

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| ''function-name''
| Identifier
| Name of the function to capture.
|-
| ''context''
| String
| Optional. Context in which the function is going to be intercepted. Possible values are ''"REFERENCE"'' - intercept at the function call (i.e. where the function is called) or ''"DEFINITION"'' - intercept at the function definition (i.e. where the function is implemented).
|-
| ''name-mangling''
| String
| Optional. Type of [[Name_Mangling|name mangling]] to be used when intercepted. Possible values are ''"EXPLICIT"'' or ''"IMPLICIT"''. If the ''context'' argument is defined the default will be '''IMPLICIT'''. Note that if '''EXPLICIT''' is set the '''srTEST_INTERCEPT(<function_name>)"''' macro is required.
|-
| ''group-id''
| String
| Optional. User defined identifier representing the group to which this function belongs when enabling interception. The default ''group-id'' is set to '''STRIDE_TEST_GROUP'''.
|}

=== Notes ===
* The function must be declared as a designator with external linkage.
* A compilation error is reported if an attempt is made to capture a function more than once.

=== Examples ===
Using the defaults for ''IMPLICIT'' and ''STRIDE_TEST_GROUP''
<source lang=c>
void boo(void);

#pragma scl_function(boo, "DEFINITION")
</source>

Setting a specific ''GROUP ID''
<source lang=c>
void boo(void);

#pragma scl_function(boo, "DEFINITION", ''IMPLICIT'', ''MY_TEST_GROUP'')
</source>

Using Test Doubles

2015-07-06T20:08:38Z

Marku: /* Creating Double Intercepts in the IM */

Test Macros

2015-07-06T19:53:27Z

Marku: /* Guidelines */

__NOTOC__
The Stride implementation of [http://en.wikipedia.org/wiki/Assertion_(software_development) assertions] are provided with a set of Test Macros (declared in srtest.h) available for use within test methods. The macros are optional - you are not required to use them in your '''Test Units'''. They provide shortcuts for testing assertions and automatic report annotation in the case of failures.

'''Example Test Macros'''
<source lang=cpp>
#include <mytest.h>

void MyTest::CheckFoo() {
..
srEXPECT_EQ(foo(2 + 2), 4);
}
void MyTest::CheckBoo() {
..
srEXPECT_GT(boo(3 * 3), 7);
}
</source>

== Guidelines ==

There are three types of macros: '''EXPECT''', '''ASSERT''', and '''EXIT''' macros. They have slight but important differences in their behavior.

'''srEXPECT_''xx''('''''condition''''')'''
* If the condition does match the expectation, this macro:
** sets the current test case status to PASS
* If the condition does not match the expectation, this macro:
** sets the current test case status to FAIL
** adds a comment to the current test case's results report which includes the condition as well as file and line number
** does not alter code execution

'''srASSERT_''xx''('''''condition''''')'''
* If the condition does match the assertion, this macro:
** sets the current test case status to PASS
* If the condition does not match the expectation, this macro:
** sets the current test case status to FAIL
** adds a comment to the current test case's results report which includes the condition as well as file and line number
** immediately returns from the current test function. The return specifies no value, therefore a function or method that uses <tt>srASSERT_xx</tt> should be declared to return <tt>void</tt>. In case when a test function is required to return other type then <tt>void</tt>, a simple trick could be applied - in C++ test code postpend any value of the required type using the C/C++ <tt>comma</tt> operator (e.g. <tt>srASSERT_TRUE(cond),1;</tt>); in C only test code postpend the value space separated (e.g. <tt>srASSERT_TRUE(cond) 1;</tt>).

'''srEXIT_''xx''('''''condition''''')'''
* The behavior and restrictions of the exit macros are identical to the '''ASSERT''' macros. '''EXIT''' macros differ only in that they cause the test unit to cease execution. That is, no subsequent test methods within the currently running test unit are executed once an EXIT macro has failed in its assertion. This macro is useful in test units that implement behavioral testing where each test methods depends on the successful completion of the preceding test method.
* If a teardown fixture is declared, and the srEXIT_xx() fires within a test method, the teardown method will be called.
* If srEXIT_xx() fires within a scl_test_cclass test unit, its de-init function will be called if declared.

Note that the supplemental information is put into the test report only if the macro fails. In the case above, if a equals 10, then no action is taken; if a is not equal to 10, then the normal srEXPECT actions are taken, and--in addition--the supplemental information is added to the failure record in the test report. The ''srEXIT_xx()'' macro does not support this feature.

The following sections document the several types of testing macros that are provided by the STRIDE Framework. For simplicity, we refer to all the macros using a '''''prefix''''' tag - when using the macros in test code, the '''''prefix''''' should be replaced by one of the following: '''srEXPECT''', '''srASSERT''', or '''srEXIT''', depending on how the test writer wants failures to be handled.

== Boolean Macros ==
The boolean macros take a single condition expression, ''cond'', that evaluates to an integral type or bool.

The condition will be evaluated once. When a failure is detected, the report will be annotated.

{| class="prettytable"
| colspan="2" | '''Boolean'''

|-
! macro !! Pass if
|-
| '''''prefix'''''_TRUE(''cond'');
| ''cond'' is non-zero

|-
| '''''prefix'''''_FALSE(''cond'');
| ''cond'' is zero

|}

==== Example ====
<source lang='c'>
int a = 5;
int b = 5;
srEXPECT_TRUE(a == b);

srEXPECT_FALSE(2 < 1);

srEXPECT_FALSE(a != b);
srEXPECT_TRUE(1 < 2);
srEXPECT_TRUE(1);
</source>

''' ''Hint:'' A simple way to add supplemental information to test failures'''

If the '''C++ compiler mode''' is enabled then the ''srEXPECT_xx()'' and ''srASSERT_xx()'' macros also support adding to a test's annotations using the '''<< operator'''. For example:

<source lang="cpp">
srEXPECT_TRUE(a == 10) << "My custom message" << " a equals: " << a;
</source>

== Comparison Macros ==

Comparison macros take two operands and compare them using the indicated operator.

The comparison macros will work for scalar types as well as C++ objects that have the corresponding comparison operator implemented.

{| class="prettytable"
| colspan="2" | '''Comparison'''
|-
! macro !! Pass if

|-
| '''''prefix'''''_EQ(''val1'', ''val2'');
| ''val1'' == ''val2''

|-
| '''''prefix'''''_NE(''val1'', ''val2'');
| ''val1'' != ''val2''

|-
| '''''prefix'''''_LT(''val1'', ''val2'');
| ''val1''<nowiki> < </nowiki>''val2''

|-
| '''''prefix'''''_LE(''val1'', ''val2'');
| ''val1''<nowiki> <= </nowiki>''val2''

|-
| '''''prefix'''''_GT(''val1'', ''val2'');
| ''val1'' > ''val2''

|-
| '''''prefix'''''_GE(''val1'', ''val2'');
| ''val1'' >= ''val2''

|}

==== Example ====
<source lang='c'>
int a = 5;
int b = 5;

srEXPECT_EQ( 4, 4 );
srEXPECT_NE( 6, 7 );
srEXPECT_EQ( a, b );
srEXPECT_GT( 2 , 1 );
srEXPECT_GE( 2 , 1 );
</source>

== C String Comparison Macros ==

C String Comparison Macros are intended only for use with C-style null terminated strings. The strings can be char or wchar_t based.

Don't use these macros to compare C++ objects representing strings since such classes typically have overloaded comparison operators. The standard comparison macros should be used instead.

* An empty string will appear in error message output as “”. A null string will appear as NULL with no surrounding quotes. Otherwise all output strings are quoted.
* The type of str1 and str2 must be compatible with ''const char*'' or ''const wchar_t*''.

{| class="prettytable"
| colspan="2" | '''C-string comparison'''
|-
! macro !! Pass if

|-
| '''''prefix'''''_STREQ(''str1'', ''str2'');
| ''str1'' and ''str2'' have the same content

|-
| '''''prefix'''''_STRNE(''str1'', ''str2'');
| ''str1'' and ''str2'' have different content

|-
| '''''prefix'''''_STRCASEEQ(''str1'', ''str2'');
| ''str1'' and ''str2'' have the same content, ignoring case.

|-
| '''''prefix'''''_STRCASENE(''str1'', ''str2'');
| ''str1'' and ''str2'' have different content, ignoring case.

|}

==== Example ====
<source lang='c'>
const char* s1 = "This String is unique";
const char* s2 = "This String has an equivalent";
const char* s2Twin = "This String has an equivalent";
const char* s2TwinNoCase = "this string has an equivalent";

srEXPECT_STREQ( s2, s2Twin);
srEXPECT_STREQ( s2, "This String has an equivalent");
srEXPECT_STRCASEEQ(s2, s2TwinNoCase);

srEXPECT_STRNE(s1, s2);
srEXPECT_STRNE(s2, s2TwinNoCase);
srEXPECT_STRCASENE(s1, s2);
</source>

== Predicate Macros ==
Predicate macros allow user control over the pass/fail decision making.

A predicate is a function returning bool that is implemented by the user that is the macro. Up to four arguments can also passed to the predicate through the macro.

{| class="prettytable"
| colspan="2" | '''Predicates'''
|-
! macro !! Pass if

|-
| '''''prefix'''''_PRED1(''pred'', ''val1'')
| ''pred''(''val1'') returns true

|-
| '''''prefix'''''_PRED2(''pred'', ''val1'', ''val2'')
| ''pred''(''val1'', ''val2'') returns true

|-
| …(up to arity of 4)
|
|

|}

==== Example ====
<source lang='c'>
static int alwaysTrueOneArg(int i)
{
return 1;
}

static int alwaysTrueTwoArgs(int i, int j)
{
return 1;
}

static void Pass()
{
// examples of passing expectations using predicates.

srEXPECT_PRED1(alwaysTrueOneArg, 25);
srEXPECT_PRED2(alwaysTrueTwoArgs, 100, 33);
}
</source>

== Floating Point Comparison Macros ==
Floating point macros are for comparing equivalence (or near equivalence) of floating point numbers.

These macros are are very useful for dealing with floating point round-off effects.

{| class="prettytable"
| colspan="2" | '''Floating Point comparison'''
|-
! macro !! Pass if

|-
| '''''prefix'''''_NEAR(val1, val2, epsilon);
| The absolute value of the difference between val1 and val2 is less than or equal to epsilon.
|}

==== Example ====
<source lang='c'>
float x = 2.00005f;
float nearX = 2.00006f;
float nearXEpsilon = .000019f;

double y = 1.2345;
double nearY = 1.23456;
double nearYEpsilon = .00019;

srEXPECT_NEAR(x, nearX, nearXEpsilon);
srEXPECT_NEAR(y, nearY, nearYEpsilon);
</source>

== Exception Macros ==
Exception macros are used to ensure that expected exceptions are thrown. They are applicable to C++ code only.

These macros require exception support from the target compiler. If the target compiler does not have exception support the macros cannot be used.

{| class="prettytable"
| colspan="2" | '''Exceptions'''
|-
! macro !! Pass if

|-
| '''''prefix'''''_THROW(statement, ex_type);
| ''statement'' throws an exception of type ''ex_type''

|-
| '''''prefix'''''_THROW_ANY(''statement'');
| ''statement'' throws an exception (type not important)

|-
| '''''prefix'''''_NO_THROW(''statement'');
| ''statement'' does not throw an exception
|}

===== Example =====
<source lang='cpp'>
srEXPECT_THROW(throwStdException(), std::exception);
srEXPECT_THROW(throwInt(), int);

srEXPECT_THROW_ANY(throwStdException());
srEXPECT_THROW_ANY(throwInt());

srEXPECT_NO_THROW(doesntThrow());
</source>

== Dynamic Test Case Macros ==
The macros presented so far assume that their actions are directed at the currenly in-scope test case. However, test cases can be created dynamically using STRIDE's [Runtime Test Services].

In order to handle dynamic test cases, each of the macros requires another parameter which specifies the test case to report against. Other than this, these macros provide exactly equivalent functionality to the non-dynamic peer. The dynamic macros are listed below. All require a test case, value of type '''srTestCaseHandle_t''' from srtest.h, to be passed as the first parameter).

{| class="prettytable"
! macro
|-
| colspan="1" | '''Boolean'''

|-
| '''''prefix'''''_TRUE_DYN(tc, ''cond'');

|-
| '''''prefix'''''_FALSE_DYN(tc, ''cond'');

|-
| colspan="1" | '''Comparison'''

|-
| '''''prefix'''''_EQ_DYN(tc, ''val1'', ''val2'');

|-
| '''''prefix'''''_NE_DYN(tc, ''val1'', ''val2'');

|-
| '''''prefix'''''_LT_DYN(tc, ''val1'', ''val2'');

|-
| '''''prefix'''''_LE_DYN(tc, ''val1'', ''val2'');

|-
| '''''prefix'''''_GT_DYN(tc, ''val1'', ''val2'');

|-
| '''''prefix'''''_GE_DYN(tc, ''val1'', ''val2'');

|-
| colspan="1" | '''C-string comparison'''

|-
| '''''prefix'''''_STREQ_DYN(tc, ''str1'', ''str2'');

|-
| '''''prefix'''''_STRNE_DYN(tc, ''str1'', ''str2'');

|-
| '''''prefix'''''_STRCASEEQ_DYN(tc, ''str1'', ''str2'');

|-
| '''''prefix'''''_STRCASENE_DYN(tc, ''str1'', ''str2'');

|-
| colspan="1" | '''Exceptions'''

|-
| '''''prefix'''''_THROW_DYN(statement, ex_type);

|-
| '''''prefix'''''_THROW_ANY_DYN(tc, ''statement'');

|-
| '''''prefix'''''_NO_THROW_DYN(tc, ''statement'');

|-
| colspan="1" | '''Predicates'''

|-
| '''''prefix'''''_PRED1_DYN(tc, ''pred'', ''val1'');

|-
| '''''prefix'''''_PRED2_DYN(tc, ''pred'', ''vall'', ''val2'');

|-
| …(up to arity of 4)

|-
| colspan="1" | '''Floating Point'''

|-
| '''''prefix'''''_NEAR_DYN(tc, ''val1'', ''val2'', ''epsilon'');

|}

Test Pragmas

2015-07-06T19:50:28Z

Marku:

__NOTOC__
Test Unit [http://en.wikipedia.org/wiki/Directive_(programming) pragmas] are a set of compiler directives that allow annotations within C and C++ source files that are meaningful to the Stride Build Tools but '''ignored''' by standard C/C++ compilers. During the build process the [[Build Tools]] generates an [[Intercept Module]] that provides [http://en.wikipedia.org/wiki/Test_harness harnessing] for the tests. Refer to the following header file that contains the ''scl_test_class'' pragma.

'''Header file'''
<source lang=cpp>
#include <srtest.h>

class MyTest {
public:
void CheckFoo();
void CheckBoo();
};
#pragma scl_test_class(MyTest)
</source>

= scl_test_class =

The ''scl_test_class'' pragma is used to declare a C++ class as a '''Test Unit'''.

=== Syntax ===

#pragma scl_test_class(class-name)

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| ''class-name''
| Identifier
| The name of the test unit. This is must be the name of an existing C++ class or a struct.
|}

=== Notes ===
* This pragma requires the compilation language to be C++. If the compilation language is not C++ and this pragma is encountered, then an error is issued and this pragma is ignored.

* The test class identified:
** must have single public constructor - default or explicit.
** must not be a templated class
** must not be a nested class
** must not be a pure virtual class
** must have one or more member functions that is suitable as a test method. For a member function to be a test method it:
*** must be declared within the test class (a method that is inherited from a base class cannot be a test method)
*** must be declared with public access
*** must have a return type of bool, an integral type (signed or unsigned long, int, short, char) or void.
*** must have an empty parameter list - declared as f() or f(void).
*** must not be a templatized function
*** must not be an overloaded operator
*** must not be a static member.

* Optionally if desired a set of setup/teardown fixtures could be applied.

* Optionally if desired the public constructor may have arguments (they must all be of [http://en.wikipedia.org/wiki/Null-terminated_string C-string] and numeric types).

=== Examples ===
<source lang=cpp>
#include <srtest.h>

class MyOtherTest
{
public:
// Declaring a constructor for an scl_test_class is optional, but
// if a constructor is declared all arguments must be of plain old data (POD) type.
MyOtherTest(int i, const char* s);

void CheckItOut();
};
#ifdef _SCL
#pragma scl_test_class(MyOtherTest)
#endif
</source>

= scl_test_cclass =

The ''scl_test_cclass'' pragma is used to declare a "C" language struct (class) to be captured as a '''Test Unit'''.

=== Syntax ===

#pragma scl_test_cclass(cclass-name, init-function-name { , deinit-function-name })

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| width="180" | '''Parameters'''
| width="60" | '''Type'''
| '''Description'''
|-
| ''cclass-name''
| Identifier
| The name of the test unit. This is must be the name of an existing struct in C/C++ or a class in C++.
|-
| ''init-function-name''
| Identifier
| The initialization function name. This is synonymous with a class constructor. There must be a prior user-declared function with this name. The first parameter must be a pointer to cclass-name. Additional parameters are left up to the user's implementation.
|-
| ''deinit-function-name (optional)''
| Identifier
| The deinitialization function name. This is synonymous with a class destructor. If declared, there must be a user-declared function with this name. The first parameter must be a pointer to cclass-name. Additional parameters are left up to the user's implementation.
|}

=== Notes ===
* The cclass-name identified:
** may not appear as a specifier of a prior pragma.
** must be a struct in C.
** must be either a struct or class in C++.
** must be [http://en.wikipedia.org/wiki/Plain_Old_Data_Structures POD] type.
** must not be a template class.
** must not be a nested class.
** must contain at least one member function pointer with a prototype that:
*** returns integral type: void, int, or bool (bool accepted only for C++).
*** has a first parameter that is a pointer to cclass-name.

* The init-function-name identified:
** must be declared prior to this pragma's declaration.
** must not have been used in any prior or subsequent SCL pragma.
** must have its first parameter declared as a pointer to the identified cclass.
** optionally if desired it may have additional parameters (they must all be of [http://en.wikipedia.org/wiki/Null-terminated_string C-string] and numeric types).

* The deinit-function-name:
** is optional.
** must be declared prior to this pragma's declaration if it appears in pragma.
** must not have been used in any prior or subsequent SCL pragma.
** must have its first parameter declared as a pointer to the identified class.

* Optionally if desired a set of setup/teardown fixtures could be applied.

=== Examples ===
<source lang=c>
#include <srtest.h>

typedef struct CClass_Basic_Simple {
int (*pf_TestMethod)(struct CClass_Basic_Simple* pcc);
} CClass_Basic_Simple;

#ifdef __cplusplus
extern "C" {
#endif
void CClass_Basic_Simple_init(struct CClass_Basic_Simple* pcc);
#ifdef __cplusplus
}
#endif

#ifdef _SCL
#pragma scl_test_cclass(CClass_Basic_Simple, CClass_Basic_Simple_init)
#endif
</source>

= scl_test_flist =

The ''scl_test_flist'' pragma is used to declare a list of external functions as a '''Test Unit'''.

=== Syntax ===

#pragma scl_test_flist(test-unit-name, function-1, function-2..n)

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| ''test-unit-name''
| String
| The name of the test unit. A special '''identifier''' with that name will be synthesized into which the list of external functions will be assembled.
'''''Note:''' this string may not contain a space.''
|-
| ''function-1''
| Identifier
| Name of the first external function to be captured.
|-
| ''function-2..n [Optional]''
| Identifier
| Optional names of second and subsequent external functions to be captured.
|}

=== Notes ===

* The external functions must meet the following conditions:
** Their parameter lists must be empty (void).
** They cannot be overloaded.
** They cannot be overloaded operators.
** They may not be template functions.
** They may throw exceptions when compiled in CPP mode.
** They cannot have been previously captured with the ''scl_function'' pragram or the ''scl_test_flist'' pragma.
** They cannot be ''public static class methods''.
** They must return a pass/fail result. The return type may be declared void or an integer type (bool is acceptable in C++ mode). If void is the return type, any calls to the test function default to successful.

* Once an external function has been captured with scl_test_flist, that function may not then be captured again with the ''scl_function'' pragma, or the ''scl_test_flist'' pragma, or used with any other qualification pragmas.
* Use of this pragma requires an include of ''srtest.h''.
* The test-unit-name may not have been specified with a prior scl_test_flist pragma.
* The test-unit-name may not be the name of an existing routine.
* Optionally if desired a set of setup/teardown fixtures could be applied.

=== Examples ===
<source lang=c>
#include <srtest.h>

#ifdef __cplusplus
extern "C" {
#endif

int test1();
int test2();

#ifdef __cplusplus
}
#endif

#ifdef _SCL
#pragma scl_test_flist("Foo", test1, test2)
#endif
</source>

= scl_test_setup =

The ''scl_test_setup'' pragma declares a member method to be a setup fixture for an existing '''Test Unit'''. The setup method will be called before the execution of each test method.

=== Syntax ===

#pragma scl_test_setup(test-unit-name, function-name)

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| ''test-unit-name''
| Identifier
| Name of a captured test unit.
|-
| ''function-name''
| Identifier
| Name of a member function of the test unit to be used as a setup fixture.
|}

=== Notes ===
* This pragma identifies the setup fixture of an existing test unit, i.e. either a class with ''scl_test_class'' applied to it, a set of functions with ''scl_test_flist'' applied to it, or a C struct with ''scl_test_cclass'' applied to it.
* There may be only one setup fixture per test unit.

=== Example ===
<source lang=cpp>
#include <srtest.h>

class MyTest {
public:
void FooSetup();
void CheckFoo();
};

#ifdef _SCL
#pragma scl_test_class(MyTest)
#pragma scl_test_setup(MyTest, FooSetup)
#endif
</source>

= scl_test_teardown =

The ''scl_test_teardown'' pragma declares a member method to be a teardown fixture for an existing '''Test Unit'''. The teardown method will be called after the execution of each test method.

=== Syntax ===

#pragma scl_test_teardown(test-unit-name, function-name)

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| ''test-unit-name''
| Identifier
| Name of a captured test unit.
|-
| ''function-name''
| Identifier
| Name of a member function of the test unit to be used as a teardown fixture.
|}

=== Notes ===
* This pragma identifies the setup fixture of an existing test unit, i.e. either a class with ''scl_test_class'' applied to it, a set of functions with ''scl_test_flist'' applied to it, or a C struct with ''scl_test_cclass'' applied to it.
* There may be only one teardown fixture per test unit.

=== Example ===
<source lang=cpp>
class MyTest {
public:
void FooSetup();
void FooTeardown();
void CheckFoo();
};

#ifdef _SCL
#pragma scl_test_class(MyTest)
#pragma scl_test_setup(MyTest, FooSetup)
#pragma scl_test_teardown(MyTest, FooTeardown)
#endif
</source>

Test Point

2015-07-06T19:40:33Z

Marku: Marku moved page Test Point2 to Test Point

__NOTOC__
Source instrumentation is the process by which developers and domain experts selectively instrument the source under test for the purpose of writing test scenarios against the executing application. Implementing tests that leverage source instrumentation is called '''Expectation Testing'''. This validation technique is very useful for verifying proper code sequencing based on the software's internal design.

<source lang='c'>
#include <srtest.h>
...
/* a test point with no payload */
srTEST_POINT("first test point");

/* a test point with binary payload */
srTEST_POINT_DATA("second test point", myData, sizeofMyData);

/* a test point with simple string payload */
srTEST_POINT_STR("third test point", "payload with simple string");

/* a test point with formatted string payload */
srTEST_POINT_STR("third test point", "payload with format string %d", myVar);

#ifdef __cplusplus
srTEST_POINT_STR("c++ test point", "") << "stream input supported under c++";
#endif
</source>

Unlike traditional unit testing that drives testing based on input parameters and isolating functionality, ''Expectation'' testing is executed within a fully functional software build running on a real target platform. Test Points are not dependent on input parameters, but often leverage the same types of input/output controls used by functional and black-box testing.

Another unique feature of this type of testing is that domain expertise is not required to implement a test. Developers and domain experts use instrumentation to export design knowledge of the software to the entire team. Furthermore, there is no stubbing required, no special logic to generate input parameters, and no advanced knowledge required of how the application software is coded.

To enable effective test coverage, developers and domain experts are required to insert instrumentation at key locations to gain insight and testability. Here are some general suggested source code areas to consider instrumenting:
* critical function entry/exit points
* state transitions
* critical or interesting data transitions (using optional payload to convey data values)
* callback routines
* data persistence
* error conditions

The steps required to implement an Expectation test are the following:
#[[#Instrumentation | Instrumentation]]
#[[#Define your Expectations | Define your Expectations]]
#[[#Write the Test Unit | Write the Test Unit]]

== Instrumentation ==
To make the software ''testable'', the first step in the process is for the experts to selectively insert instrumentation macros into the source code. Test Points themselves have nominal impact on the performance of the application – they are only active during test data collection<ref name="n1"> Test data collection is typically implemented in a low priority background thread. The data is captured in the calling routine's thread context (no context switch) but processed in the background or on the host. Instrumentation macros return immediately to the caller (i.e. no-op) when testing is not active.</ref>. Test Points contain names and optional payload data. When Test Points are activated, they are collected in the background, along with timing and any associated data. The set of Test Points hit, their order, timing, and data content can all be used to validate that the software is behaving as expected. [[Test_Log | Test Logs]] can also be added to the source code to provide additional information in the context of an executing test.

To specify a test point you should include the '''srtest.h''' header file from the STRIDE Runtime in your compilation unit. The Test Point macros are active only when <tt>STRIDE_ENABLED</tt> is <tt>#define</tt>d, therefore it is practical to place these macros in-line in production source. When <tt>STRIDE_ENABLED</tt> is not <tt>#define</tt>d, these macros evaluate to nothing.

{| class="prettytable"
| colspan="2" | '''Test Point Macros'''
|-valign="top"
| '''srTEST_POINT'''(''label'')
| ''label'' is a pointer to a null-terminated string

|-valign="top"
| '''srTEST_POINT_DATA'''(''label'', ''data'', ''size'')
| ''label'' is a pointer to a null-terminated string 
''data'' is a pointer to a byte sequence 
''size'' is the size of the ''data'' in bytes

|-valign="top"
| '''srTEST_POINT_STR'''(''label'', ''message'')
| ''label'' is a pointer to a null-terminated string 
''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string 
When used in the context of a c++ compilation unit, this macro also supports the streaming operator to append to the message string (see example below)
|}

== Define your Expectations ==

In addition to instrumenting the source code with Test Points, you must also define the [[Expectations]] of the Test Points. This involves defining the list of Test Points expected to be hit during a given test scenario. Expectation can also include any '''data''' associated with a Test Point that requires validation.

== Write the Test Unit ==
Once the source under test has been instrumented and the [[Expectations | expectations]] defined, Stride offers a number of techniques that can be used for implementing '''expectation tests'''.

* Tests can be written in [[Expectation_Tests_in_C/C%2B%2B | C or C++]] and executed on the device under test using the Stride framework.
* Tests can be written in [[Test_Modules_Overview | Perl Test Scripts]]

== Notes ==
<references/>

Main Page

2015-07-06T19:38:55Z

Marku:

Test Point (old)

2015-07-06T19:38:22Z

Marku: Marku moved page Test Point to Test Point (old)

[[Expectations|Expectation]] testing is made possible by selective developer instrumentation of the source under test. The instrumentation is unobtrusive and powerful, supporting the ability to optionally attach string or binary data to the Test Point.

== Reference ==
To specify a test point you should include the '''srtest.h''' header file from the STRIDE Runtime in your compilation unit. The Test Point macros are active only when <tt>STRIDE_ENABLED</tt> is <tt>#define</tt>d, therefore it is practical to place these macros in-line in production source. When <tt>STRIDE_ENABLED</tt> is not <tt>#define</tt>d, these macros evaluate to nothing.

{| class="prettytable"
| colspan="2" | '''Test Point Macros'''
|-valign="top"
| '''srTEST_POINT'''(''label'')
| ''label'' is a pointer to a null-terminated string

|-valign="top"
| '''srTEST_POINT_DATA'''(''label'', ''data'', ''size'')
| ''label'' is a pointer to a null-terminated string 
''data'' is a pointer to a byte sequence 
''size'' is the size of the ''data'' in bytes

|-valign="top"
| '''srTEST_POINT_STR'''(''label'', ''message'')
| ''label'' is a pointer to a null-terminated string 
''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string 
When used in the context of a c++ compilation unit, this macro also supports the streaming operator to append to the message string (see example below)
|}

=== C++ Only Features ===
In C++ source the string variants of the macros above support adding to other content to the message by using the << operator. For more information please read [[C%2B%2B_Only_Features|this]].

== Code Snippets ==
The source under test is instrumented simply by placing lines of the following form into the source code:

<source lang='c'>
#include <srtest.h>
...
/* a test point with no payload */
srTEST_POINT("first test point");

/* a test point with binary payload */
srTEST_POINT_DATA("second test point", myData, sizeofMyData);

/* a test point with simple string payload */
srTEST_POINT_STR("third test point", "payload with simple string");

/* a test point with formatted string payload */
srTEST_POINT_STR("third test point", "payload with format string %d", myVar);

#ifdef __cplusplus
srTEST_POINT_STR("c++ test point", "") << "stream input supported under c++";
#endif
</source>

When this code is executed it broadcasts a message via the STRIDE Runtime which is detected by the test code if it is currently looking for test points in an expectation test. We refer to the receipt of a test point as a ''test point hit''.

[[Category:Source Instrumentation]]

Main Page

2015-07-06T19:36:31Z

Marku:

Welcome to the STRIDE™ Wiki 

STRIDE™ has been designed specifically for '''On-Target White-box Testing'''. STRIDE™ is '''test infrastructure''' used by engineers to implement and execute [[Types_of_Testing_Supported_by_STRIDE | '''tests''']] for validating embedded software executing On-Target. The STRIDE™ system consists of a [[STRIDE Overview | '''framework''']] for testing and a [[STRIDE_Test_Space | '''hosted web application''']] for storing and analyzing test results.

<hr/>
'''For an overview of STRIDE™ [[STRIDE Overview| PLEASE CLICK HERE]]'''.

[[Image:screen_icon.png]] [[STRIDE Overview Video | STRIDE Overview Screencast]]
<hr/>

{| class="FCK__ShowTableBorders"
|-
| valign="top" |
{| class="FCK__ShowTableBorders" style="border-right: rgb(187,204,204) 1px solid; border-top: rgb(187,204,204) 1px solid; vertical-align: top; border-left: rgb(187,204,204) 1px solid; border-bottom: rgb(187,204,204) 1px solid" cellspacing="5" cellpadding="2"
|-
! style="border-right: rgb(163,176,191) 1px solid; padding-right: 0.4em; border-top: rgb(163,176,191) 1px solid; padding-left: 0.4em; font-weight: bold; font-size: 120%; background: rgb(206,223,242) 0% 50%; padding-bottom: 0.2em; margin: 0pt; border-left: rgb(163,176,191) 1px solid; color: rgb(0,0,0); padding-top: 0.2em; border-bottom: rgb(163,176,191) 1px solid; text-align: left; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial" | [[:Category:Source Instrumentation | Instrumentation]]

! style="border-right: rgb(163,176,191) 1px solid; padding-right: 0.4em; border-top: rgb(163,176,191) 1px solid; padding-left: 0.4em; font-weight: bold; font-size: 120%; background: rgb(206,223,242) 0% 50%; padding-bottom: 0.2em; margin: 0pt; border-left: rgb(163,176,191) 1px solid; color: rgb(0,0,0); padding-top: 0.2em; border-bottom: rgb(163,176,191) 1px solid; text-align: left; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial" | [[:Category:Tests in Script| Tests in Script]]

! style="border-right: rgb(163,176,191) 1px solid; padding-right: 0.4em; border-top: rgb(163,176,191) 1px solid; padding-left: 0.4em; font-weight: bold; font-size: 120%; background: rgb(206,223,242) 0% 50%; padding-bottom: 0.2em; margin: 0pt; border-left: rgb(163,176,191) 1px solid; color: rgb(0,0,0); padding-top: 0.2em; border-bottom: rgb(163,176,191) 1px solid; text-align: left; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial" | [[:Category:Test Units| Tests in C/C++]]

! style="border-right: rgb(163,176,191) 1px solid; padding-right: 0.4em; border-top: rgb(163,176,191) 1px solid; padding-left: 0.4em; font-weight: bold; font-size: 120%; background: rgb(206,223,242) 0% 50%; padding-bottom: 0.2em; margin: 0pt; border-left: rgb(163,176,191) 1px solid; color: rgb(0,0,0); padding-top: 0.2em; border-bottom: rgb(163,176,191) 1px solid; text-align: left; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial" | [[:Category:Running Tests | Running Tests]]

|-


| valign="top" |
* [[Source Instrumentation Overview | Overview]]
* [[Test Point | Test Points]]
* [[Test Log | Test Logs]]
* [[Function_Capturing | Functions]]
* [[Expectations | Expectations]]


| valign="top" |
* [[Test Modules Overview | Overview]]
* [[Perl Script APIs | Perl Script APIs]]
* [[Perl Script Snippets]]
* [[Script_Samples | Samples]]


| valign="top" |
* [[Test Units Overview | Overview]]
* [[Test Units]]
* [[Test Macros | Test Macros]]
* [[Test API | Test APIs]]
* [[C/C%2B%2B_Samples | Samples]]


| valign="top" |
* [[Running Tests (old) | Running Tests]]
* [[Listing Functions and Test Units|Listing Tests]]
* [[Tracing | Tracing]]
* [[Organizing Tests into Suites | Organizing Tests]]
* [[Setting up your CI Environment | Setting up CI]]

|-
! style="border-right: rgb(163,176,191) 1px solid; padding-right: 0.4em; border-top: rgb(163,176,191) 1px solid; padding-left: 0.4em; font-weight: bold; font-size: 120%; background: rgb(206,223,242) 0% 50%; padding-bottom: 0.2em; margin: 0pt; border-left: rgb(163,176,191) 1px solid; color: rgb(0,0,0); padding-top: 0.2em; border-bottom: rgb(163,176,191) 1px solid; text-align: left; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial" | [[:Category: Test Space|Test Space]]

! style="border-right: rgb(163,176,191) 1px solid; padding-right: 0.4em; border-top: rgb(163,176,191) 1px solid; padding-left: 0.4em; font-weight: bold; font-size: 120%; background: rgb(206,223,242) 0% 50%; padding-bottom: 0.2em; margin: 0pt; border-left: rgb(163,176,191) 1px solid; color: rgb(0,0,0); padding-top: 0.2em; border-bottom: rgb(163,176,191) 1px solid; text-align: left; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial" | [[:Category:Reference | Reference]]

! style="border-right: rgb(163,176,191) 1px solid; padding-right: 0.4em; border-top: rgb(163,176,191) 1px solid; padding-left: 0.4em; font-weight: bold; font-size: 120%; background: rgb(206,223,242) 0% 50%; padding-bottom: 0.2em; margin: 0pt; border-left: rgb(163,176,191) 1px solid; color: rgb(0,0,0); padding-top: 0.2em; border-bottom: rgb(163,176,191) 1px solid; text-align: left; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial" | [[:Category:Installation | Installation]]

! style="border-right: rgb(163,176,191) 1px solid; padding-right: 0.4em; border-top: rgb(163,176,191) 1px solid; padding-left: 0.4em; font-weight: bold; font-size: 120%; background: rgb(206,223,242) 0% 50%; padding-bottom: 0.2em; margin: 0pt; border-left: rgb(163,176,191) 1px solid; color: rgb(0,0,0); padding-top: 0.2em; border-bottom: rgb(163,176,191) 1px solid; text-align: left; -moz-background-clip: -moz-initial; -moz-background-origin: -moz-initial; -moz-background-inline-policy: -moz-initial" | [[:Category:Training | Training]]

|-




| valign="top" |
* [[STRIDE_Test_Space|Overview]]
* [[User Administration]]
* [[Creating Test Spaces]]
* [[Uploading Test Results]]
* [[Notifications]]
* [[Reporting Entities]]
* [[Creating And Using Baselines | Using Baselines]]


| valign="top" |
* [[STRIDE Runner]]
* [[STRIDE Build Tools]]
* [[STRIDE Runtime]]
* [[STRIDE Off-Target Environment]]
* [[Platform Abstraction Layer]]
* [[Intercept Module]]
* [[Test Unit Pragmas]]
* [[Reporting Model]]


| valign="top" |
* [[Installation Overview | Overview]]
* [[Desktop Installation | Framework setup]]
* [[Runtime Integration]]
* [[Build Integration]]
* [[Stride Sandbox]]


| valign="top" |
* [[Training Getting Started | Getting Started]]
* [[Training Test Macros | Test Macros]]
* [[Training File IO | File IO]]
* [[Training Parameters | Parameters]]
* [[Training Fixturing | Fixturing]]
* [[Training Expectations | Test Points]]
* [[Training Doubling | Test Doubles]]
|}

Click here for [[:Category:Release Notes| Release Notes]]

Test Implementation

2015-07-06T19:35:22Z

Marku:

* [[Test Unit]]
* [[Test Pragmas]]
* [[Test Macros]]
* [[Notes]]
* [[Parameterized_Test_Units | Parameters]]
* [[File Transfer Services | Remote Files]]
* [[Test_Documentation_in_C/C%2B%2B | Documenting Tests]]
* [[Running Tests]]
* [[Runtime Test Services | Test Services]]

Running Tests

2015-07-06T19:34:25Z

Marku: Marku moved page Running Tests2 to Running Tests

__NOTOC__
Tests are executed using the [[STRIDE Runner | Stride Runner]] controlled by a host computer that is physically connected to the target via a configurable communication channel (TCP/IP or serial port). The application software is required to be running, including the [[STRIDE Runtime | Stride Runtime]], enabling a ''connection'' between the ''Runner'' and ''Runtime''.

'''Block diagram'''

[[Image:Running_Tests.jpg|400px|Connection Block Diagram]]

'''Invoking the Runner (aka <tt>stride</tt>) from a console'''

stride --database="%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb" --device=TCP:localhost:8000 --run="*"

or for Linux/FreeBSD

stride --database="$STRIDE_DIR%/SDK/Posix/out/TestApp.sidb" --device=TCP:localhost:8000 --run="*"

Option files can be helpful (i.e. my.opt)

--database "%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb"
--device TCP:localhost:8000

stride --options_file my.opt --run "*"

'''Reviewing your Results'''

[[Image:Runner_Test_Report.jpg |Example Report]]

== Input ==
In order to run tests, you must provide the following information to stride.exe:

;database file
: This is the .sidb file that is created by the ''Stride Compiler'' during the target build process. This file contains meta-information used to run tests.

;device parameters
: This tells stride.exe how to connect to the target.

;tests to run
: A set of [[Test Unit| Test Units]] compiled in the database. You may also optionally specify named hierarchical suites in which to put the results of the tests you designate.

== Output ==
Upon test completion, test output is always written as follows:

;console output
: A quick summary of results is written to standard output. Test counts are shown for the categories of '''passed''', '''failed''', '''in progress''', and '''not in use'''

;local xml file
: Detailed results are written to a local xml file. By default, this file is written to the directory where the input Stride database file is located and given the same name as the database file with an extension of ".xml". If you open this file in a web browser, an xsl transform is automatically downloaded and applied before rendering.

Optionally, you may also publish the results to [http://www.testspace.com Test Space] upon test completion.