STRIDE Wiki - User contributions [en]

STRIDE Runner

2015-09-09T03:00:32Z

Jeffs: Switch to testrun from stride.exe

__NOTOC__
The '''testrun''' executable (a.k.a. Stride Runner) runs script or native target-based tests from a desktop (Windows, Linux or FreeBSD) host computer and optionally uploads results to [http://www.testspace.com Testspace].

testrun [options] [TU...]

Each command line option that accepts an argument should be entered with a space between the option and its corresponding argument; i.e.
testrun -o argument --option1 argument1

= Input =
In order to run tests, you must provide the following information to Runner:

;database file
: This is the .sidb file that is created by the [[Build Tools]] during the target build process. This file contains meta-information used to run tests.

;device parameters
: This tells how to connect to the target.

;tests to run
: A set of Test Units and or Test Scripts.

== Running ==
By default, no tests are run. You need to explicitly list them using the <tt>--run</tt> option.

=== Order ===
When explicitly specified, Tests are run in the order in which they are given on the command line.

It is possible to run all Test Units by explicitly specifying a wildcard (*). In that case they are run in alphabetical order.

=== Rules ===

* Test Units are specified by name or wildcard
* Scripts are specified by relative or absolute file name
* Tests (scripts or test units) are grouped together within curly braces, i.e. "{" and "}"
* When specifying more than one Test in a group, each Test instance must be delimited by a semicolon.
* When a Test accepts parameters they are passed following the Test name in parentheses comma separated, i.e. "<tt>myTest(param1, param2)</tt>"
* The output of each Test group is placed within a hierarchy of named suites. (The root suite does not have a name.)
* The suite into which a Test group is placed is specified immediately before the group, e.g. ''<tt>suitepath</tt>''<tt>{</tt>''<tt>testunitgroup</tt>''<tt>}</tt>
* Hierarchical suite paths are delimited by "/" (forward slash) and are always specified starting from the root.
* If the same Test is specified to be run more than once, and one or more results are to be written to the same suite, the each conflicting Test name is appended with an incrementing count in the form of "(n)". For example: Results from three runs of TestUnit "myTest" are all written to the root suite. The Tests will be reported with the names "myTest" "myTest(1)" and "myTest(2)".

==== Wildcard Matching ====
The following wildcard characters are recognized in Test Unit specifications:

* '''<code>*</code>''' (asterisk) matches all Test Units
* '''<code>-</code>''' (hyphen) matches all remaining Test Units (useful when putting Test Units into suites)

==== Examples ====

;<code>-r "/{*}"</code>
: Run all Test Units and put results into the root-level suite.

;<code>-r "/Suite{*}"</code>
: Run all tests and put results into a suite named "Suite" that is a child of the root suite.

;<code>-r "/{Test1;Test2;Test3}"</code>
: Run the Tests named "Test1", "Test2", and "Test3" in the designated order; put results into the root suite.

;<code>-r "/{Test1;Test2;Test3}" -r "/SecondPass{Test1}"</code>
: Run the Tests named "Test1", "Test2", and "Test3" in the designated order; put results into the root suite. Then run the Test Unit named "Test1" again and put the results into a suite named "SecondPass".

;<code>-r "/{Test1;Test2;Test3}" -r "/Remaining{-}"</code>
: Run the Test Units named "Test1", "Test2", and "Test3" in the designated order; put results into the root suite. Then run all remaining Test Units and put the results into a suite named "Remaining".

;<code>-r "/Suite1/Suite2{TestA; TestB}"</code>
: Run the Tests named "TestA" and "TestB", and put the results into a suite named "Suite2" that is a child of a suite named "Suite1" that is a child of the root. Note that we must enclose the specification in quotes since the specification contains a space.

;<code>-r "/{TestA(29); TestB(5.67, \"some text\")}"</code>
: Run the Tests named "TestA" with parameter 29 and "TestB" with parameters 5.67 and "some text", and put the results into the root suite. Note that we must enclose the specification in quotes since the specification contains spaces and embedded quotes (which needs to be escaped using backslash).

;<code>-r "TestX; TestY; TestZ"</code>
: This is a special convenience syntax. If the suite and grouping braces are omitted, the app runs the the named Test Units and puts their results put into the root suite.

= Output =
Upon test completion, test output is always written as follows:

;console output
: A quick summary of results is written to standard output. Test counts are shown for the categories of '''passed''', '''failed''', '''in progress''', and '''not in use'''

;local xml file
: Detailed results are written to a local xml file. By default, this file is written to the directory where the input STRIDE database file is located and given the same name as the database file with an extension of ".xml". If you open this file in a web browser, an xsl transform is automatically downloaded and applied before rendering.

Optionally, you may also publish the results to your [http://www.testspace.com Testspace] upon test completion.

;Testspace
: Results are uploaded using your Testspace URL and login credentials. You must specify the testspace name and project name when using this option.

= Options =
{| border="1" cellspacing="0" cellpadding="10" style="align:left;background-color:#ffffcc;"
!width="200pt"|'''Option'''
!width="500pt"|'''Description'''
|-
| '''--database''' [ -d ] ''arg''
| Specifies the name of an existing STRIDE database (.sidb) file that will be used for test execution.

This can be a relative or absolute path. If the path contains one or more spaces, it must be enclosed in quotes. Under Windows, both DOS and [http://en.wikipedia.org/wiki/Path_(computing)#Uniform_Naming_Convention UNC names] are accepted.

As a path delimiter you may use either the forward slash or backslash character. (Forward slashes are typically preferred due to the standard use of the backslash as an escape character.)
|-
| '''--device''' ''arg''
| Specifies the parameters to be used to connect to the target device (i.e. TCP:''host'':''port'' or COM''port'':''rate'':''mode''). For example:
* <code>TCP:localhost:8000</code>
* <code>COM7:28800:8N1</code>
* <code>/dev/ttyS2:57600:8N1</code>
|-
| '''--timeout''' ''arg'' (=0)
| Specifies a watchdog timeout (in seconds) per single test function or test method.
Default value is 0. (0 = infinite timeout)
|-
| '''--run''' [ -r ] ''arg''
| Specifies a list of Test Units or Scripts to execute and their order of execution with optional report grouping by suite. You may specify this option many times on the command line to include multiple groupings in a single test. A named Test Suite may be specified more than once to run it multiple times. See [[#Input|Input]] section above for more details.
Any nameless positional option is treated as this option.
|-
| '''--trace''' [ -t ] ''arg'' (=txt)
| Specifies if and how to trace on target instrumentation (functions, messages and test points/logs). 
The following types of tracing formats are supported:
* <code>"txt"</code> - single-entry-per-line plain text file
* <code>"yml"</code> - [http://en.wikipedia.org/wiki/Yaml YAML] formated file
* <code>"-"</code> - interactive single-entry-per-line plain text
With the exception of the interactive type all others are attached as an annotation to the output file. 
Default value is "txt".
|-
| '''--trace_timeout''' ''arg''
| Specifies a trace timeout (in seconds).
|-
| '''--upload''' [ -u ] ''arg'' (=start)
| Specifies if and how to upload the results to Testspace. Arg specifies type of upload; 
The following types of uploads are supported:
* <code>"full"</code> - This causes a new result set to be created and marked complete.
* <code>"start"</code> - This is used to create a new result set and mark it incomplete. An upload of this type should be followed by zero or more uploads that add results and one final upload that finishes the result set.
* <code>"add"</code> - This is used to add more data to an existing incomplete result set. The result set will still be marked incomplete after this upload type.
* <code>"finish"</code> - This is used to add more data to an existing incomplete result set and mark it complete. This is the last step in creating a result set from multiple executions.
Default value is "start".
|-
| '''--testspace''' ''arg''
| Specifies the Testspace URL to which the results will be uploaded, expressed in the following form: 
<code><nowiki>user:pwd@<domain>.testspace.com/<project>/<space></nowiki></code> 
Where 'domain' is your organization's subdomain, and 'project' and 'space' are the names of related project and space.
|-
| '''--result''' ''arg''
| Name of the output XML result file and optionally, when uploading, the result set name along with folder path (default "root").
The following ways to specify are supported:
* <code>"myfile.xml"</code> - set both, output XML result file and result set name, to be the same.
* <code>"myname|"</code> - set the result set name only, use the default "testresult.xml" as output XML result file.
* <code>"|myfile.xml"</code> - set the output XML result file only, let Testspace to choose the result set name.
* <code>"myname|myfile.xml"</code> - set both, result set name and output XML result file, to be different names.
* <code>"myname/path/to/folder|myfile.xml"</code> - set both, result set name and output XML result file, to be different and upload under an explicit folder path.
* <code>"/path/to/folder|myfile.xml"</code> - set the output XML result file, let Testspace to choose the result set name but upload under an explicit folder path.
Default value is "testresult.xml".
|-
| '''--list'''
| Lists all Test Units and intercepted Functions in the specified Stride database to standard out.
|-
| '''--diagnostics'''
| Performs a set of built-in diagnostic tests on the target. See [[Build_Integration#Running_Diagnostics|Running Diagnostics]] for details.
|-
| '''--log_level''' ''arg'' (=warn)
| Controls the target source [[Test Log |Test Logs]] level. 
The following levels are supported:
* <code>"all"</code> - enables all logging
* <code>"info"</code> - enables only INFO and higher
* <code>"warn"</code> - enables only WARN and higher
* <code>"err"</code> - enables only ERROR and higher
* <code>"off"</code> - disables all logging
Default value is "warn".
|-
| '''--options_file''' ''arg''
| Specifies a file from which the program reads command line options. The format is the same as the command line except that options may be split across multiple lines. Lines in an options file that begin with the character '#' are ignored.
|-
| '''--version'''
| Prints version information to the console.
|-
| '''--dry'''
| Dry run, just validate the input without device connection or test execution.
|}

=== Using a Proxy ===
If you access the Internet via an HTTP proxy, you must set the environment variables <tt>HTTP_PROXY</tt> and <tt>HTTPS_PROXY</tt> to specify the name and port of the proxy server.

Symptoms of needing to specify a proxy is the following errors:

<pre>
Unable to proceed. Invalid testspace address or credentials. Couldn't access the specified company.
Failed performing request: [7] Couldn't connect to server
</pre>
<pre>
Unable to proceed. Failed performing request: [6] Couldn’t resolve host name
</pre>

Define the environment variables in format '''<tt><nowiki>host:port</nowiki></tt>''' where ''host'':''port'' corresponding to your proxy server. For example, on Windows:
> set HTTP_PROXY=myproxy:8765
> set HTTPS_PROXY=myproxy:8765
would instruct the stride runner to use the proxy named <tt>myproxy</tt> on port <tt>8765</tt> to communicate via HTTP and HTTPS protocols.

If your proxy requires authentication use the format '''<tt><nowiki>username:password@host:port</nowiki></tt>''' where ''username'' and ''password'' are your credentials.

Test Log

2015-07-21T18:54:21Z

Jeffs:

__NOTOC__
Test Log macros provide a simple means to add information from the source under test to the currently executing test case. This information is added to the test report as annotations with a level of either ''Error'', ''Warning'', or ''Info'' according to the macro that is used. The log messages are also captured when tracing is enabled in the [[Stride Runner]].

The <tt>srTEST_LOG_xx()</tt> macro is intended to be a general instrumentation macro for capturing information in the source under test. Logs differ from Test Points in that they cannot be used for the basis of expectation tests - they are strictly informational. The Stride log messages are intended to supplement, not supplant a [[Test Point | Test Points]].

= Reference =
To use these macros you should include the '''srtest.h''' header file from the Stride Runtime in your compilation unit. The Test Log macros are active only when <tt>STRIDE_ENABLED</tt> is <tt>#define</tt>d, therefore it is practical to place these macros in-line in production source. When <tt>STRIDE_ENABLED</tt> is not <tt>#define</tt>d, these macros evaluate to nothing.

{| class="prettytable"
| colspan="2" | '''Error Logging'''
|-
| srTEST_LOG_ERROR(''message'', ...)
| ''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string
|}

{| class="prettytable"
| colspan="2" | '''Warning Logging'''
|-
| srTEST_LOG_WARN(''message'', ...)
| ''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string
|}

{| class="prettytable"
| colspan="2" | '''Info Logging'''
|-
| srTEST_LOG_INFO(''message'', ...)
| ''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string
|}

* The maximum length of the message string approximately 1000 characters. If the maximum length is exceeded, the message string is truncated.

=== C++ Only Features ===
In C++ source the macros above support adding to other content to the message by using the << operator.

=== Log Level ===

By default only logs with level of either ''Error'' or ''Warning'' are propagated to the host. The [[STRIDE Runner#Options | Stride Runner]] via <tt>--log_level</tt> ''option'' provides finer control over that behavior.

== Code Snippets ==
<source lang='c'>
#include <srtest.h>
...
srTEST_LOG_ERROR("This is an error message.");

srTEST_LOG_WARN("This is a warning message with format string %d.", 123);

srTEST_LOG_INFO("This is an info message with format string %s and %s.", "this", "that");

#ifdef __cplusplus
srTEST_LOG_ERROR("some error: ") << "stream input supported under c++";
#endif
</source>

Test Point

2015-07-21T17:21:26Z

Jeffs: Update implementation step titles

__NOTOC__
Source instrumentation is the process by which developers and domain experts selectively instrument the source under test for the purpose of writing test scenarios against the executing application. Implementing tests that leverage source instrumentation is called '''Expectation Testing'''. This validation technique is very useful for verifying proper code sequencing based on the software's internal design.

<source lang='c'>
#include <srtest.h>
...
/* a test point with no payload */
srTEST_POINT("first test point");

/* a test point with binary payload */
srTEST_POINT_DATA("second test point", myData, sizeofMyData);

/* a test point with simple string payload */
srTEST_POINT_STR("third test point", "payload with simple string");

/* a test point with formatted string payload */
srTEST_POINT_STR("third test point", "payload with format string %d", myVar);

#ifdef __cplusplus
srTEST_POINT_STR("c++ test point", "") << "stream input supported under c++";
#endif
</source>

Unlike traditional unit testing that drives testing based on input parameters and isolating functionality, Expectation testing is executed within a fully functional software build running on a real target platform. Test Points are not dependent on input parameters, but often leverage the same types of input/output controls used by functional and black-box testing.

Another unique feature of this type of testing is that domain expertise is not required to implement a test. Developers and domain experts use instrumentation to export design knowledge of the software to the entire team. Furthermore, no stubbing is required, no special logic to generate input parameters, and no advanced knowledge required of how the application software is coded.

To enable effective test coverage, developers and domain experts insert instrumentation at key locations to gain insight and testability. Here are some general suggested source code areas to consider instrumenting:
* entry/exit points of critical functions
* state transitions
* critical or interesting data transitions (using optional payload to convey data values)
* callback routines
* data persistence
* error conditions

The steps required to implement an Expectation test are the following:
#[[#Instrument Code | Instrument Code]]
#[[#Define Expectations | Define Expectations]]
#[[#Write Test Units | Write Test Units]]

== Instrument Code ==
To make the software ''testable'', the first step in the process is for the experts to selectively insert instrumentation macros into the source code. Test Points have a nominal impact on the performance of the application as they are only active during test data collection<ref name="n1"> Test data collection is typically implemented in a low priority background thread. The data is captured in the calling routine's thread context (no context switch) but processed in the background or on the host. Instrumentation macros return immediately to the caller (i.e. no-op) when testing is not active.</ref>. Test Points contain names and optional payload data. When Test Points are activated, they are collected in the background, along with timing and any associated data. The set of Test Points hit, their order, timing, and data content can all be used to validate that the software is behaving as expected. [[Test_Log | Test Logs]] can also be added to the source code to provide additional information in the context of an executing test.

To specify a Test Point you should include the '''srtest.h''' header file from the Stride Runtime in your compilation unit. The Test Point macros are active only when <tt>STRIDE_ENABLED</tt> is <tt>#define</tt>d, therefore it is practical to place these macros in-line in production source. When <tt>STRIDE_ENABLED</tt> is not <tt>#define</tt>d, these macros evaluate to nothing.

{| class="prettytable"
| colspan="2" | '''Test Point Macros'''
|-valign="top"
| '''srTEST_POINT'''(''label'')
| ''label'' is a pointer to a null-terminated string

|-valign="top"
| '''srTEST_POINT_DATA'''(''label'', ''data'', ''size'')
| ''label'' is a pointer to a null-terminated string 
''data'' is a pointer to a byte sequence 
''size'' is the size of the ''data'' in bytes

|-valign="top"
| '''srTEST_POINT_STR'''(''label'', ''message'')
| ''label'' is a pointer to a null-terminated string 
''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string 
When used in the context of a c++ compilation unit, this macro also supports the streaming operator to append to the message string (see example below)
|}

== Define Expectations ==

In addition to instrumenting the source code with Test Points, you must also define the [[Expectations]] of the Test Points. This involves defining the list of Test Points expected to be hit during a given test scenario. Expectations can also include any '''data''' associated with a Test Point that requires validation.

== Write Test Units ==
Once the source under test has been instrumented and the [[Expectations]] defined, Stride offers a number of techniques that can be used for implementing '''expectation tests'''.

* Tests can be written in [[Test_Point_Testing_in_C/C%2B%2B | C or C++]] and executed on the device under test using the Stride framework.
* Tests can be written in [[Perl_Script_APIs | Perl]].

== Notes ==
<references/>

Test Point

2015-07-21T17:16:24Z

Jeffs:

__NOTOC__
Source instrumentation is the process by which developers and domain experts selectively instrument the source under test for the purpose of writing test scenarios against the executing application. Implementing tests that leverage source instrumentation is called '''Expectation Testing'''. This validation technique is very useful for verifying proper code sequencing based on the software's internal design.

<source lang='c'>
#include <srtest.h>
...
/* a test point with no payload */
srTEST_POINT("first test point");

/* a test point with binary payload */
srTEST_POINT_DATA("second test point", myData, sizeofMyData);

/* a test point with simple string payload */
srTEST_POINT_STR("third test point", "payload with simple string");

/* a test point with formatted string payload */
srTEST_POINT_STR("third test point", "payload with format string %d", myVar);

#ifdef __cplusplus
srTEST_POINT_STR("c++ test point", "") << "stream input supported under c++";
#endif
</source>

Unlike traditional unit testing that drives testing based on input parameters and isolating functionality, Expectation testing is executed within a fully functional software build running on a real target platform. Test Points are not dependent on input parameters, but often leverage the same types of input/output controls used by functional and black-box testing.

Another unique feature of this type of testing is that domain expertise is not required to implement a test. Developers and domain experts use instrumentation to export design knowledge of the software to the entire team. Furthermore, no stubbing is required, no special logic to generate input parameters, and no advanced knowledge required of how the application software is coded.

To enable effective test coverage, developers and domain experts insert instrumentation at key locations to gain insight and testability. Here are some general suggested source code areas to consider instrumenting:
* entry/exit points of critical functions
* state transitions
* critical or interesting data transitions (using optional payload to convey data values)
* callback routines
* data persistence
* error conditions

The steps required to implement an Expectation test are the following:
#[[#Instrumentation | Instrumentation]]
#[[#Define your Expectations | Define your Expectations]]
#[[#Write the Test Unit | Write the Test Unit]]

== Instrumentation ==
To make the software ''testable'', the first step in the process is for the experts to selectively insert instrumentation macros into the source code. Test Points have a nominal impact on the performance of the application as they are only active during test data collection<ref name="n1"> Test data collection is typically implemented in a low priority background thread. The data is captured in the calling routine's thread context (no context switch) but processed in the background or on the host. Instrumentation macros return immediately to the caller (i.e. no-op) when testing is not active.</ref>. Test Points contain names and optional payload data. When Test Points are activated, they are collected in the background, along with timing and any associated data. The set of Test Points hit, their order, timing, and data content can all be used to validate that the software is behaving as expected. [[Test_Log | Test Logs]] can also be added to the source code to provide additional information in the context of an executing test.

To specify a Test Point you should include the '''srtest.h''' header file from the Stride Runtime in your compilation unit. The Test Point macros are active only when <tt>STRIDE_ENABLED</tt> is <tt>#define</tt>d, therefore it is practical to place these macros in-line in production source. When <tt>STRIDE_ENABLED</tt> is not <tt>#define</tt>d, these macros evaluate to nothing.

{| class="prettytable"
| colspan="2" | '''Test Point Macros'''
|-valign="top"
| '''srTEST_POINT'''(''label'')
| ''label'' is a pointer to a null-terminated string

|-valign="top"
| '''srTEST_POINT_DATA'''(''label'', ''data'', ''size'')
| ''label'' is a pointer to a null-terminated string 
''data'' is a pointer to a byte sequence 
''size'' is the size of the ''data'' in bytes

|-valign="top"
| '''srTEST_POINT_STR'''(''label'', ''message'')
| ''label'' is a pointer to a null-terminated string 
''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string 
When used in the context of a c++ compilation unit, this macro also supports the streaming operator to append to the message string (see example below)
|}

== Define your Expectations ==

In addition to instrumenting the source code with Test Points, you must also define the [[Expectations]] of the Test Points. This involves defining the list of Test Points expected to be hit during a given test scenario. Expectations can also include any '''data''' associated with a Test Point that requires validation.

== Write the Test Unit ==
Once the source under test has been instrumented and the [[Expectations]] defined, Stride offers a number of techniques that can be used for implementing '''expectation tests'''.

* Tests can be written in [[Test_Point_Testing_in_C/C%2B%2B | C or C++]] and executed on the device under test using the Stride framework.
* Tests can be written in [[Perl_Script_APIs | Perl]].

== Notes ==
<references/>

Test Log

2015-07-08T21:25:22Z

Jeffs: Replace STRIDE with Stride

__NOTOC__
Test Log macros provide a simple means to add information from the source under test to the currently executing test case. This information is added to the test report as annotations with a level of either ''Error'', ''Warning'', or ''Info'' according to the macro that is used. The log messages are also captured when tracing is enabled in the [[Stride Runner]].

The <tt>srTEST_LOG_xx()</tt> macro is intended to be a general instrumentation macro for capturing information in the source under test. Logs differ from test points in that they cannot be used for the basis of expectation tests - they are strictly informational. The Stride log messages are intended to supplement, not supplant a [[Test Point | Test Points]].

= Reference =
To use these macros you should include the '''srtest.h''' header file from the Stride Runtime in your compilation unit. The Test Log macros are active only when <tt>STRIDE_ENABLED</tt> is <tt>#define</tt>d, therefore it is practical to place these macros in-line in production source. When <tt>STRIDE_ENABLED</tt> is not <tt>#define</tt>d, these macros evaluate to nothing.

{| class="prettytable"
| colspan="2" | '''Error Logging'''
|-
| srTEST_LOG_ERROR(''message'', ...)
| ''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string
|}

{| class="prettytable"
| colspan="2" | '''Warning Logging'''
|-
| srTEST_LOG_WARN(''message'', ...)
| ''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string
|}

{| class="prettytable"
| colspan="2" | '''Info Logging'''
|-
| srTEST_LOG_INFO(''message'', ...)
| ''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string
|}

* The maximum length of the message string approximately 1000 characters. If the maximum length is exceeded, the message string is truncated.

=== C++ Only Features ===
In C++ source the macros above support adding to other content to the message by using the << operator.

=== Log Level ===

By default only logs with level of either ''Error'' or ''Warning'' are propagated to the host. The [[STRIDE Runner#Options | Stride Runner]] via <tt>--log_level</tt> ''option'' provides finer control over that behavior.

== Code Snippets ==
<source lang='c'>
#include <srtest.h>
...
srTEST_LOG_ERROR("This is an error message.");

srTEST_LOG_WARN("This is a warning message with format string %d.", 123);

srTEST_LOG_INFO("This is an info message with format string %s and %s.", "this", "that");

#ifdef __cplusplus
srTEST_LOG_ERROR("some error: ") << "stream input supported under c++";
#endif
</source>

Test Point Testing in C/C++

2015-07-08T21:14:08Z

Jeffs: Replace STRIDE with Stride

__NOTOC__
Test code to validate the [[Expectations]] of the Test Points often follows this basic pattern:

# Specify an expectation set consisting of expected (i.e. the test points that are expected to be hit) and optionally unexpected (i.e. the test points that are not expected to be hit) test points
# Register the expectation set with the Stride runtime
# Invoke the software under test (causing instrumentation points to be hit). This may not be necessary if the instrumented software under test is constantly running).
# Wait for the expectation set to be satisfied or a timeout to occur

Here is an example:

<source lang="c">
#include <srtest.h>

void tf_testpoint_wait(void)
{
/* specify expected set */
srTestPointExpect_t expected[]= {
{"START"},
{"ACTIVE"},
{"IDLE"},
{"END"},
{0}};

/* specify unexpected set */
srTestPointUnexpect_t unexpected[]= {
{"INVALID"},
{0}};

/* register the expectation set*/
srWORD handle;
srTestPointSetup(expected, unexpected, srTEST_POINT_EXPECT_UNORDERED, srTEST_CASE_DEFAULT, &handle);

/* start your asynchronous operation */
...

/* wait for expectation set to be satisfied or a timeout to occur */
srTestPointWait(handle, 1000);
}

#ifdef _SCL
#pragma scl_test_flist(“testfunc”, tf_testpoint_wait)
#endif
</source>

== Reference ==

=== Expectation Set ===
An expectation set is specified with an [[Expectations#Expected_List|expected]] array of '''srTestPointExpect_t''' structures and a second optional [[Expectations#Unexpected_List|unexpected]] array of '''srTestPointUnexpect_t''' structures.

==== Expected Array ====
srTestPointExpect_t is typedef'd as follows:

<source lang='c'>
typedef struct
{
/* the label value is considered the test point's identity */
const srCHAR * label;
/* optional, count specifies the number of times the test point is expected to be hit */
srDWORD count;
/* optional, predicate function to use for payload validation against user data */
srTestPointPredicate_t predicate;
/* optional, user data to validate the payload against */
void * user;
} srTestPointExpect_t;
</source>

'''NOTES:'''
* The end of the array has to be marked by a srTestPointExpect_t set to all zero values
* The ''count'', ''predicate'' and ''user'' members may be omitted in the array declaration (they will be automatically set to 0 by the compiler)
* A ''count'' value of either 0 or 1 is interpreted as 1
* The ''count'' could be set as "0 or more" by using the [[Expectations#Special_Processing|special]] srTEST_POINT_ANY_COUNT symbolic constant
* A ''predicate'' value 0 indicates that any associated data with a test point payload will be ignored.
* A ''user'' value 0 indicates that there is no user data associated with this test point
* The ''label'' could be specified to ''any test point'' within the current expected set of test points by using the [[Expectations#Special_Processing|special]] srTEST_POINT_ANY_IN_SET or srTEST_POINT_ANY_AT_ALL symbolic constants.

==== Unexpected Array ====
srTestPointUnexpect_t is typedef'd as follows:

<source lang='c'>
typedef struct
{
/* the label value is considered the test point's identity */
const srCHAR * label;
} srTestPointUnexpect_t;
</source>

'''NOTES:'''
* The end of the array has to be marked by a srTestPointUnexpect_t set to all zero values
* The ''label'' could be specified to ''everything else'' relative to the '''expected''' array by using the [[Expectations#Special_Processing|special]] srTEST_POINT_EVERYTHING_ELSE symbolic constant.

==== srTestPointPredicate_t ====
When defining the expectation set per entry a [[Expectations#State_Data_Validation|payload validation]] predicate function could be specified. The signature of it should match the following type:

<source lang="c">
extern "C" typedef srBYTE (*srTestPointPredicate_t)(const srTestPoint_t* ptTP, void* pvUser);
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| ptTP
| Input
| Pointer to the currently processed Test Point.
|-
| pvUser
| Input
| Pointer to opaque user data associated with an entry in the expectation set.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srBYTE
| srTRUE for valid, srFALSE for invalid, srIGNORE otherwise.
|}

'''NOTE:'''
* As part of the standard Stride distribution there are three predefined function predicate helpers:
** srTestPointMemCmp - byte comparison
** srTestPointStrCmp - string case sensitive comparison
** srTestPointStrCaseCmp - string case insensitive comparison

==== srTestPointSetup ====
The srTestPointSetup() routine is used to register an expectation set.

<source lang="c">
srBOOL srTestPointSetup(srTestPointExpect_t* ptExpected,
srTestPointUnexpect_t* ptUnexpected,
srBYTE yMode,
srTestCaseHandle_t tTestCase,
srWORD* pwHandle);
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| ptExpected
| Input
| Pointer to an expectated array.
|-
| ptUnexpected
| Input
| Pointer to an unexpectated array. This is optional and could be set srNULL.
|-
| yMode
| Input
| Bitmask that specifies whether the expectated test points occur in [[Expectations#Sequencing_Properties|order and/or strict]]. Possible values are: 
srTEST_POINT_EXPECT_ORDERED - the test points are expected to be hit exactly in the defined order 
srTEST_POINT_EXPECT_UNORDERED - the test points could to be hit in any order 
srTEST_POINT_EXPECT_STRICT - the test points are expected to be hit exactly as specified (no consecutive duplicate hits) 
srTEST_POINT_EXPECT_NONSTRICT - other test points from the universe could to be hit in between 
srTEST_POINT_EXPECT_CONTINUE - on successful expectation satisfaction continue processing until the wait timeout expires
|-
| tTestCase
| Input
| Handle to a test case. srTEST_CASE_DEFAULT can be used for the default test case.
|-
| pwHandle
| Output
| Handle that represents the registered expectation set
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srBOOL
| srTRUE on success, srFALSE otherwise.
|}

==== srTestPointWait ====
The srTestPointWait() routine is used to wait for the expectation to be satisfied.

<source lang="c">
srBOOL srTestPointWait(srWORD wHandle,
srDWORD dwTimeout);
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| wHandle
| Input
| Handle to a registered expectation set.
|-
| dwTimeout
| Input
| Timeout value in milliseconds; 0 means just check without waiting.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srBOOL
| srTRUE on success, srFALSE otherwise.
|}

'''NOTES:'''
* The test thread blocks until either the expectation set is satisfied, unless srTEST_POINT_EXPECT_CONTINUE is specified on setup, or the timeout elapses.
* All test points hit during the wait (both expected and unexpected) are added to the test report as testcase comments
* Once the wait is over (whether the expectation set has been satisfied or there has been a test failure), the current expectation set is automatically unregistered from the Stride runtime and the handle is released
* If you want to return immediately from a test case if expectation fails then make the check/wait call an argument to the ''srASSERT_TRUE()'' macro.

==== srTestPointCheck ====
The srTestPointCheck() routine is used to check for the expectation post routine completion. This is useful for verifying a set of expectations events that should have already transpired (thus are waiting to be processed).

<source lang="c">
srBOOL srTestPointCheck(srWORD wHandle);
</source>

{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Parameters'''
| '''Type'''
| '''Description'''
|-
| wHandle
| Input
| Handle to a registered expectation set.
|}
 
{| border="1" cellspacing="0" cellpadding="10" style="align:left;"
| '''Return Value'''
| ''' Description'''
|-
| srBOOL
| srTRUE on success, srFALSE otherwise.
|}

'''NOTES:'''
* All test points hit before the check (both expected and unexpected) are added to the test report as testcase comments
* Once the check is done (whether the expectation set has been satisfied or there has been a test failure), the current expectation set is automatically unregistered from the Stride runtime and the handle is released
* If you want to return immediately from a test case if expectation fails then make the check/wait call an argument to the ''srASSERT_TRUE()'' macro.

=== C++ Facade Class ===
The ''srtest.h'' file provides a simple [http://en.wikipedia.org/wiki/Facade_pattern facade] class that wraps the test point APIs described above in a simple C++ class called '''srTestPointsHandler'''. If you are writing your unit tests in C++ (using Stride test classes), then this class is available for your use. The class implements the following methods, which correspond exactly to the C API equivalents:

; srTestPointsHandler : constructor. Takes four arguments which match exactly the first four parameters of the [[#srTestPointSetup|srTestPointSetup]] function.
; Wait : instance method that provides same functionality as [[#srTestPointWait|srTestPointWait]].
; Check : instance method that provides same functionality as [[#srTestPointCheck|srTestPointCheck]].

Test Point

2015-07-08T21:05:24Z

Jeffs:

__NOTOC__
Source instrumentation is the process by which developers and domain experts selectively instrument the source under test for the purpose of writing test scenarios against the executing application. Implementing tests that leverage source instrumentation is called '''Expectation Testing'''. This validation technique is very useful for verifying proper code sequencing based on the software's internal design.

<source lang='c'>
#include <srtest.h>
...
/* a test point with no payload */
srTEST_POINT("first test point");

/* a test point with binary payload */
srTEST_POINT_DATA("second test point", myData, sizeofMyData);

/* a test point with simple string payload */
srTEST_POINT_STR("third test point", "payload with simple string");

/* a test point with formatted string payload */
srTEST_POINT_STR("third test point", "payload with format string %d", myVar);

#ifdef __cplusplus
srTEST_POINT_STR("c++ test point", "") << "stream input supported under c++";
#endif
</source>

Unlike traditional unit testing that drives testing based on input parameters and isolating functionality, ''Expectation'' testing is executed within a fully functional software build running on a real target platform. Test Points are not dependent on input parameters, but often leverage the same types of input/output controls used by functional and black-box testing.

Another unique feature of this type of testing is that domain expertise is not required to implement a test. Developers and domain experts use instrumentation to export design knowledge of the software to the entire team. Furthermore, there is no stubbing required, no special logic to generate input parameters, and no advanced knowledge required of how the application software is coded.

To enable effective test coverage, developers and domain experts are required to insert instrumentation at key locations to gain insight and testability. Here are some general suggested source code areas to consider instrumenting:
* entry/exit points of critical functions
* state transitions
* critical or interesting data transitions (using optional payload to convey data values)
* callback routines
* data persistence
* error conditions

The steps required to implement an Expectation test are the following:
#[[#Instrumentation | Instrumentation]]
#[[#Define your Expectations | Define your Expectations]]
#[[#Write the Test Unit | Write the Test Unit]]

== Instrumentation ==
To make the software ''testable'', the first step in the process is for the experts to selectively insert instrumentation macros into the source code. Test Points themselves have nominal impact on the performance of the application – they are only active during test data collection<ref name="n1"> Test data collection is typically implemented in a low priority background thread. The data is captured in the calling routine's thread context (no context switch) but processed in the background or on the host. Instrumentation macros return immediately to the caller (i.e. no-op) when testing is not active.</ref>. Test Points contain names and optional payload data. When Test Points are activated, they are collected in the background, along with timing and any associated data. The set of Test Points hit, their order, timing, and data content can all be used to validate that the software is behaving as expected. [[Test_Log | Test Logs]] can also be added to the source code to provide additional information in the context of an executing test.

To specify a test point you should include the '''srtest.h''' header file from the Stride Runtime in your compilation unit. The Test Point macros are active only when <tt>STRIDE_ENABLED</tt> is <tt>#define</tt>d, therefore it is practical to place these macros in-line in production source. When <tt>STRIDE_ENABLED</tt> is not <tt>#define</tt>d, these macros evaluate to nothing.

{| class="prettytable"
| colspan="2" | '''Test Point Macros'''
|-valign="top"
| '''srTEST_POINT'''(''label'')
| ''label'' is a pointer to a null-terminated string

|-valign="top"
| '''srTEST_POINT_DATA'''(''label'', ''data'', ''size'')
| ''label'' is a pointer to a null-terminated string 
''data'' is a pointer to a byte sequence 
''size'' is the size of the ''data'' in bytes

|-valign="top"
| '''srTEST_POINT_STR'''(''label'', ''message'')
| ''label'' is a pointer to a null-terminated string 
''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string 
When used in the context of a c++ compilation unit, this macro also supports the streaming operator to append to the message string (see example below)
|}

== Define your Expectations ==

In addition to instrumenting the source code with Test Points, you must also define the [[Expectations]] of the Test Points. This involves defining the list of Test Points expected to be hit during a given test scenario. Expectation can also include any '''data''' associated with a Test Point that requires validation.

== Write the Test Unit ==
Once the source under test has been instrumented and the [[Expectations]] defined, Stride offers a number of techniques that can be used for implementing '''expectation tests'''.

* Tests can be written in [[Test_Point_Testing_in_C/C%2B%2B | C or C++]] and executed on the device under test using the Stride framework.
* Tests can be written in [[Perl_Script_APIs | Perl]].

== Notes ==
<references/>

Test Point

2015-07-08T20:59:06Z

Jeffs: /* Instrumentation */

__NOTOC__
Source instrumentation is the process by which developers and domain experts selectively instrument the source under test for the purpose of writing test scenarios against the executing application. Implementing tests that leverage source instrumentation is called '''Expectation Testing'''. This validation technique is very useful for verifying proper code sequencing based on the software's internal design.

<source lang='c'>
#include <srtest.h>
...
/* a test point with no payload */
srTEST_POINT("first test point");

/* a test point with binary payload */
srTEST_POINT_DATA("second test point", myData, sizeofMyData);

/* a test point with simple string payload */
srTEST_POINT_STR("third test point", "payload with simple string");

/* a test point with formatted string payload */
srTEST_POINT_STR("third test point", "payload with format string %d", myVar);

#ifdef __cplusplus
srTEST_POINT_STR("c++ test point", "") << "stream input supported under c++";
#endif
</source>

Unlike traditional unit testing that drives testing based on input parameters and isolating functionality, ''Expectation'' testing is executed within a fully functional software build running on a real target platform. Test Points are not dependent on input parameters, but often leverage the same types of input/output controls used by functional and black-box testing.

Another unique feature of this type of testing is that domain expertise is not required to implement a test. Developers and domain experts use instrumentation to export design knowledge of the software to the entire team. Furthermore, there is no stubbing required, no special logic to generate input parameters, and no advanced knowledge required of how the application software is coded.

To enable effective test coverage, developers and domain experts are required to insert instrumentation at key locations to gain insight and testability. Here are some general suggested source code areas to consider instrumenting:
* critical function entry/exit points
* state transitions
* critical or interesting data transitions (using optional payload to convey data values)
* callback routines
* data persistence
* error conditions

The steps required to implement an Expectation test are the following:
#[[#Instrumentation | Instrumentation]]
#[[#Define your Expectations | Define your Expectations]]
#[[#Write the Test Unit | Write the Test Unit]]

== Instrumentation ==
To make the software ''testable'', the first step in the process is for the experts to selectively insert instrumentation macros into the source code. Test Points themselves have nominal impact on the performance of the application – they are only active during test data collection<ref name="n1"> Test data collection is typically implemented in a low priority background thread. The data is captured in the calling routine's thread context (no context switch) but processed in the background or on the host. Instrumentation macros return immediately to the caller (i.e. no-op) when testing is not active.</ref>. Test Points contain names and optional payload data. When Test Points are activated, they are collected in the background, along with timing and any associated data. The set of Test Points hit, their order, timing, and data content can all be used to validate that the software is behaving as expected. [[Test_Log | Test Logs]] can also be added to the source code to provide additional information in the context of an executing test.

To specify a test point you should include the '''srtest.h''' header file from the Stride Runtime in your compilation unit. The Test Point macros are active only when <tt>STRIDE_ENABLED</tt> is <tt>#define</tt>d, therefore it is practical to place these macros in-line in production source. When <tt>STRIDE_ENABLED</tt> is not <tt>#define</tt>d, these macros evaluate to nothing.

{| class="prettytable"
| colspan="2" | '''Test Point Macros'''
|-valign="top"
| '''srTEST_POINT'''(''label'')
| ''label'' is a pointer to a null-terminated string

|-valign="top"
| '''srTEST_POINT_DATA'''(''label'', ''data'', ''size'')
| ''label'' is a pointer to a null-terminated string 
''data'' is a pointer to a byte sequence 
''size'' is the size of the ''data'' in bytes

|-valign="top"
| '''srTEST_POINT_STR'''(''label'', ''message'')
| ''label'' is a pointer to a null-terminated string 
''message'' is a pointer to a null-terminated format string 
''...'' variable list matching the format string 
When used in the context of a c++ compilation unit, this macro also supports the streaming operator to append to the message string (see example below)
|}

== Define your Expectations ==

In addition to instrumenting the source code with Test Points, you must also define the [[Expectations]] of the Test Points. This involves defining the list of Test Points expected to be hit during a given test scenario. Expectation can also include any '''data''' associated with a Test Point that requires validation.

== Write the Test Unit ==
Once the source under test has been instrumented and the [[Expectations]] defined, Stride offers a number of techniques that can be used for implementing '''expectation tests'''.

* Tests can be written in [[Test_Point_Testing_in_C/C%2B%2B | C or C++]] and executed on the device under test using the Stride framework.
* Tests can be written in [[Perl_Script_APIs | Perl]].

== Notes ==
<references/>

Desktop Installation

2014-01-21T17:12:02Z

Jeffs: /* Create/Update STRIDE_DIR */

== Installation Packages ==
Files are installed by unzipping the provided package to your PC. Packages are available targeting the following operating systems (your version number may be different than that shown):
;Windows (x86)
:<tt>STRIDE_framework-windows_4.x.yy.zip</tt>
;Linux (x86)
:<tt>STRIDE_framework-linux_4.x.yy.tgz</tt>
;FreeBSD (x86)
:<tt>STRIDE_framework-freebsd_4.x.yy.tgz</tt>

Please see the appropriate installation instructions below.

== Windows Installation ==

=== Unpacking ===
The following installation example assumes the the installation package is located in your root directory and that the directory <tt>\stride</tt> exists. You can choose to install to a different location (all instructions below assume you are installing into <tt>\stride</tt>).

The example uses the open source [http://www.7-zip.org/ 7-Zip] utility to unzip the archive.

cd \stride
"\Program Files\7-Zip\7z" x ..\STRIDE_framework-windows_4.x.yy.zip

Once unzipped, files will have been installed under the <tt>\stride</tt> directory.

=== Verify Environment Variables ===

==== Updated PATH ====
As a final step, you will need to update your <tt>[http://en.wikipedia.org/wiki/Path_(variable) PATH]</tt> environment variable to include <tt>\stride\bin</tt>.
For instructions on modifying it, please see [http://support.microsoft.com/kb/310519 http://support.microsoft.com/kb/310519].

NOTE: ''Make sure to insert '''no spaces''' before and after the semicolon separators(;).''

==== Create/Update STRIDE_DIR====

Verify that the <tt>STRIDE_DIR</tt> environment variable exists and is set to the root installation directory (<tt>\stride</tt>). If this environment variable does not yet exist, you should create it as a user environment variable.

To confirm installation and display ''help'' run the following command in a console window:

stride -h

=== Uninstalling ===
To uninstall STRIDE simply:
* Remove any reference to <tt>\stride\bin</tt> in your <tt>PATH</tt> environment variable.
* Remove <tt>STRIDE_DIR</tt> environment variable.
* Remove <tt>\stride</tt> directory.

== Linux/FreeBSD Installation ==

=== Unpacking ===
The following installation example assumes the the installation package is located in your home directory and that the directory <tt>~/stride</tt> exists. You can choose to install to a different location (all instructions below assume you are installing into <tt>~/stride</tt>).

cd ~/stride
tar -zxvf ../STRIDE_framework-linux_4.x.yy.tgz

Once unzipped, files will have been installed under the <tt>~/stride</tt> directory.

=== Verify Environment Variables ===

==== Updated PATH ====
As a final step, you will need to update your <tt>[http://en.wikipedia.org/wiki/Path_(variable) PATH]</tt> environment variable to include <tt>~/stride/bin</tt>.

If you use the bash shell, enter the following at a command prompt, or to automatically set at each login, add to your <tt>.bashrc</tt>:
export PATH=$PATH:~/stride/bin

For other shells, and more information, please see the following articles:
* [http://www.linuxheadquarters.com/howto/basic/path.shtml http://www.linuxheadquarters.com/howto/basic/path.shtml].
* [http://en.wikipedia.org/wiki/Environment_variable#UNIX http://en.wikipedia.org/wiki/Environment_variable]

==== Create/Update STRIDE_DIR====
Verify that the <tt>STRIDE_DIR</tt> environment variable exists and is set to the root installation directory (<tt>~/stride</tt>). If this environment variable does not yet exist, you should automatically set at each login, add to your <tt>.bashrc</tt>:
export STRIDE_DIR=~/stride

To confirm installation and display ''help'' run the following command in a console window:

stride -h

NOTE: ''In a 64-bit environment the above may fail with errors like: <code>"/lib/ld-linux.so.2: bad ELF interpreter: No such file or directory"</code> or <code>"ELF interpreter /libexec/ld-elf32.so.1 not found"</code>. To resolve this issue install the appropriate 32-bit compatibility libraries for your Linux/FreeBSD distribution:''

* Debian / Ubuntu
sudo apt-get install ia32-libs
sudo apt-get install ia32-libs-multiarch:i386 (for 12.04 or higher)
* Fedora / CentOS / RHEL
sudo yum -y install glibc.i686 libstdc++.i686
* FreeBSD
Make sure to have <code>lib32</code> installed (via [http://www.freebsd.org/cgi/man.cgi?query=sysinstall&apropos=0&sektion=0&manpath=FreeBSD+8.4-RELEASE&arch=default&format=html sysinstall(8)] - Configure|Distributions|lib32) and have your kernel built with:
options COMPAT_FREEBSD32 # Compatible with i386 binaries

=== Uninstalling ===
To uninstall STRIDE simply:
* Remove any reference to <tt>~/stride/bin</tt> in your <tt>PATH</tt> environment variable.
* Remove <tt>STRIDE_DIR</tt> environment variable.
* Remove <tt>~/stride</tt> directory.

== Directories and Files ==

To integrate STRIDE in to your target build system it is required to understand the directories layout and the files inside then. A quick orientation is shown below.

''NOTE:'' ''It's not necessary to understand the workings of the STRIDE framework to perform evaluation or training. The desktop package contains an [[STRIDE Off-Target Environment]] that utilizes a SDK that is set up with appropriate options and settings to enable "out of the box" functionality.''

===<tt>bin</tt>===
This directory contains the [[Build Tools|STRIDE Build Tools]] and the [[STRIDE Runner]].

The build tools are invoked early on in the target software build process to generate special STRIDE artifacts that are used in subsequent build steps and later when running tests against the target. In an Off-Target Environment installation, these files are needed on the host computer since this is where we are building the target application. In a production environment, these files are needed only on the computer that performs the target software build.

The [[STRIDE Runner]] is the program you use to run tests from the host.

===<tt>lib</tt>===
This directory contains a set of STRIDE specific core scripting libraries along with prebuild binaries intended to be used for [[Test Modules Overview|testing in scripts]].

===<tt>Samples</tt>===
The Samples directory contains a number of sub-directories, each containing the source for a [[Samples|sample test]].

===<tt>SDK</tt>===
This directory contains the sub-directories <tt>Posix/Windows</tt>, <tt>Runtime</tt>, and <tt>SLAP</tt>, which contain source code that comprises the [[Runtime_Reference|STRIDE Runtime]]. These sources are built in to a static libary (e.g. STRIDE Runtime library - <tt>stride.a/lib</tt>) as a dependency of your Test Application.

The <tt>Posix</tt> and <tt>Windows</tt> directories contain the target operating system specific source and configuration. If you are interested in the details, consult the articles [[Posix SDK]] and [[Windows SDK]]. Each of them contains the following sub-directories:

*<tt>settings</tt>
: This directory contains a set of <tt>stride.XXX.s2scompile</tt> files, where <tt>XXX</tt> coresponds to the target CPU architecture (i.e. X86, ARM...). These files, used by the [[s2scompile|STRIDE Compiler]], specify target CPU characteristics (endian-ness, data sizes and alignments). On Windows, this directory also contains a set of files for [[STRIDE_Extensions_for_Visual_Studio|use in building target apps with Visual Studio]].
*<tt>src</tt>
: This directory contains the source of the target [[Platform Abstraction Layer]] PAL. In addition there is a sample Makefile used to produce a sandbox TestApp.

== Perl Installation (Optional) ==
''NOTE:'' ''This is '''NOT''' required if only [[Test_Units_Overview|tests in C/C++]] will be run or to complete the [[Training_Getting_Started|STRIDE training]].''

If you intend to use [[Test Modules Overview|STRIDE Script modules]] for testing in script, you will need a recent version of Perl (x86 with threads support) installed.

As of this writing, we support only the 32-bit versions 5.8.9, 5.10.x, 5.12.x, 5.14.x and 5.16.x of Perl.

=== Windows ===
It is required to use the standard 32-bit Perl distributions from [http://www.activestate.com/activeperl ActiveState].

The following additional (non-standard) Perl packages are also required for full functionality of STRIDE tests in perl:

* [http://search.cpan.org/~smueller/Class-ISA-0.36/lib/Class/ISA.pm Class::ISA]
* [http://search.cpan.org/~andrewf/Pod-POM-0.27/lib/Pod/POM.pm Pod::POM]
* [http://search.cpan.org/~andk/Devel-Symdump-2.08/lib/Devel/Symdump.pm Devel::Symdump]

You can easily install these packages using the [http://docs.activestate.com/activeperl/5.10/faq/ActivePerl-faq2.html ppm tool]. If you access the Internet via a proxy make sure to read [http://docs.activestate.com/activeperl/5.10/faq/ActivePerl-faq2.html#ppm_and_proxies this]. Simple command-line installation of PACKAGE_NAME (the package to install) typically just requires typing:

ppm install PACKAGE_NAME

=== Linux/FreeBSD ===
We recommend you to use the standard 32-bit Perl distribution that comes with your OS version. In case you need to manually build from source make sure to configure "shared library" (<tt>-Duseshrplib</tt>), "thread support" (<tt>-Duseithreads</tt>) and no "64-bit support" (<tt>-Uuse64bitint -Uuse64bitall</tt>).

The following additional (non-standard) Perl packages are also required for full functionality of STRIDE tests in perl:

* [http://search.cpan.org/~ingy/YAML-LibYAML-0.38/lib/YAML/XS.pm YAML::XS]
* [http://search.cpan.org/~smueller/Class-ISA-0.36/lib/Class/ISA.pm Class::ISA]
* [http://search.cpan.org/~andrewf/Pod-POM-0.27/lib/Pod/POM.pm Pod::POM]
* [http://search.cpan.org/~andk/Devel-Symdump-2.08/lib/Devel/Symdump.pm Devel::Symdump]

If your perl is installed in a system directory (<tt>/usr/bin/perl</tt>, for instance), you will need root access to install shared modules. The simplest method for installing packages is via the [http://www.perl.com/doc/manual/html/lib/CPAN.html CPAN shell]. If you access the Internet via a proxy make sure to set the appropriate [http://search.cpan.org/dist/CPAN/lib/CPAN.pm#Config_Variables CPAN config variables]. To start the shell in interactive mode:

sudo perl -MCPAN -eshell

Once in the shell, search for and install the latest stable version of PACKAGE_NAME (the package to install):

install PACKAGE_NAME

The STRIDE perl packages also need to load your system's '''libperl.so''' (shared object file) at runtime. Depending on your system, this file should be loadable from a perl CORE directory or from one of the shared system directories. If you '''DO NOT''' have this shared library on your system, you might need to install a ''libperl-dev'', ''perl-devel'' or ''perl-libs'' package in order to get it. Here is how you can do that on the console of some Linux distributions:

* Debian / Ubuntu
sudo apt-get install libperl-dev
* Fedora / CentOS / RHEL
sudo yum -y install perl-devel

=== Validation ===
Once you have installed Perl we recommend you to run the following command in a console window:

<source lang="bash">
stride --device NULL --diagnostics Perl --output PerlCheck
</source>

If everything was properly set up you should get the following output:

<pre>
Loading database...
Executing diagnostics...
script "diagnostics.pl"
> 2 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 2 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...
</pre>

In addition a report file with name <tt>PerlCheck.xml</tt> will be created in the current directory. If interested in the details you could open that report file in a browser of your choice.

[[Category:Installation]]

Training Doubling

2013-02-27T17:41:22Z

Jeffs: /* Implement Exercise */

== Objectives ==

This Training Module is focused on explaining how to leverage '''Test Doubles''' in the context of executing a test. For an overview of '''intercepting''' existing functions please refer to [[Using_Test_Doubles | Test Doubles]]. The module covers following topics:
* How to apply pragmas for [[Function_Capturing | function intercepting]]
* How to decide what kind of ''mangling'' is required
** ''Definition'' verses ''Reference''
** ''Explicit'' verses ''Implicit''
* Setting and Resetting the Double implementation

There are two new files used -- '''TestDouble.cpp & TestDouble.h''' --- that implement two test Units:
* '''TestDouble_Reference'''
* '''TestDouble_Definition'''

All of the Test Units have test cases already implemented (used for reference) and have one test method that you are required to implement called '''Exercise'''. Currently the exercise methods return a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Double'' Test Units only

> stride --options_file myoptions.txt --run TestDouble_Reference --run TestDouble_Definition

Loading database...
Connecting to device...
Executing...
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------
Summary: 4 passed, 0 failed, 0 in progress, 2 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestDouble_Reference::Exercise'''
** Implement a new ''test double'' for the ''strlen()'' routine:
*** Add a ''NOTE'' capturing its name when it is called
*** Uses the real ''strlen()'' to return the length of the passed in string
** ''Hint:'' You will probably want to read about [[Using Test Doubles]]
** Use ''srEXPECT_EQ()'' to validate that ''sut_strcheck()'' returns correct length
** Make sure to restore the original routine

* '''TestDouble_Definition::Exercise'''
** Implement a new ''test double'' for the ''sut_strcpy()'' routine:
*** Log its name when it is called
*** Validate string passed to ''sut_strsave()'' is received correctly by the ''test double''
*** Remember that ''sut_strsave()'' calls ''sut_strcpy()'' with the string passed to it
*** Can use a test macro to validate the string is correctly passed to the double
*** Call the original function (''sut_strcpy()'') with a '''DIFFERENT STRING''' (i.e "Exercise Test String 2")
**** Call the original within the test double
**** Make sure to reset the original function within the mock before calling it (i.e. srDOUBLE_RESET)
** Call ''sut_strsave()'' with a string (i.e. "Exercise Test String 1")
** Use ''sut_strget()'' to retrieve a string
** Compare retrieved string with the '''DIFFERENT STRING''' inserted by the ''test double''
** NOTE - won't work if you wait until the end of the test to restore the original routine (i.e. srDOUBLE_SET)

* Execute ''Test Double'' Test Units only

> stride --options_file myoptions.txt --run TestDouble_Reference TestDouble_Definition

Loading database...
Connecting to device...
Executing...
test unit "TestDouble_Reference"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestDouble_Definition"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------
Summary: 6 passed, 0 failed, 0 in progress, 0 not in use.

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. If you have not added test space options to your options file (<tt>myoptions.txt</tt>) please see [[Training_Getting_Started#Test_Space_Access| testspace access]].

> stride --options_file myoptions.txt --run TestDouble_Reference TestDouble_Definition --space TestDouble --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===
* [[Using_Test_Doubles | Using Test Doubles]] - Outlines the basic steps to enable a ''double''
** Capture the f(x) that is going to be doubled
** Code the test logic to additionally set the Double and restore original f(x)

* [[Scl_function | Intercepting a function]] - Pragma specifics outline
** For this exercise, there are '''NO''' optional parameters when capturing a f(x) to be '''intercepted'''

* Details of the pragma parameters can be understood in the [[Intercept_Module#Interceptor | Intercept Module]] article. Please review in detail the following:
** ''context'' - This refers to incepting a callee or the called routine.
** ''name-mangling'' - How the function name is switched during the compiling process
** ''group-id'' - Used to associate a group of intercepted functions

* Some '''external links''' to review:
** [http://www.martinfowler.com/bliki/TestDouble.html Martin Fowler definition]
** [http://en.wikipedia.org/wiki/Test_Double Wikipedia definition]
** [http://msdn.microsoft.com/en-us/magazine/cc163358.aspx MSDN Magazine description]

=== Samples ===

* [[Test_Double_Sample | Test Double Sample]] that can be a useful reference. This reference example can be built and executed like all of samples using the Off-Target environment.

[[Category: Training]]

Training Expectations

2013-02-27T17:39:04Z

Jeffs: /* Implement Exercise */

== Objectives ==

This Training Module is focused on ''Test Points'' and how to validate them using ''Expectations''. The module covers the following topics:
* Presentation of a [[Expectations | validation]] technique based on ''code sequencing'' and ''state data''
* Overview of [[Source_Instrumentation_Overview#Instrumentation | source instrumentation]]
* Review of [[Test_Point_Testing_in_C/C%2B%2B | expectation tables and predicates]]
* Example use cases such as concurrent validation, using trigger conditions, etc.

There are two test files used -- '''TestExpect.cpp & TestExpect.h''' -- that implement three Test Units:
* '''TestExpect_Seq'''
* '''TestExpect_Data'''
* '''TestExpect_Misc'''

The first two Test Units have two test methods already implemented and have one method each that you are required to implement called '''Exercise'''. The third Test Unit has four test methods already implemented and also has one method to implement as an exercise. Currently the ''exercise methods'' return a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Expectations'' Test Units only

> stride --options_file myoptions.txt --run TestExpect_Seq --run TestExpect_Data --run TestExpect_Misc

Loading database...
Connecting to device...
Executing...
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
-----------------------------------------------------------
Summary: 7 passed, 1 failed, 0 in progress, 3 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestExpect_Seq::Exercise'''
** Validate '''ALL''' upper case Test Points (A - I)
** Use ''Unordered'' and ''Nonstrict'' sequencing
** Use ''sut_DoSequencing(SEQ_1)'' to generate part of the sequence
** Use ''sut_start_thread(SEQ_3)'' to generate the rest of the sequence
** Add a ''NOTE_INFO(..)'' to log that the test is executing
** ''Hint:'' You will probably want to read about [[Test Point Testing in C/C++]]

* '''TestExpect_Data::Exercise'''
** Validate the following Test Points '''{D, G, F, H}'''
** Check that '''F''' occurs 2 times
** Use ''Unordered'' sequencing
** Write a new custom predicate that validates data for both '''D''' and '''H''' Test Points
*** Add ''NOTE_INFO(..)'' to capture content of the Test Points
*** Add extra check for '''D''' that the status is '''GOOD'''
*** Confirm that data fields '''d1''' and '''d2''' are as expected
*** Pass the expected data fields (for both Test Points) as part of the '''user''' data within the setup
*** Use ''sut_start_thread(SEQ_3)'' to generate the sequence
** Write another custom predicate for validating data for '''G'''
*** Add ''NOTE_INFO(..)'' to capture content of the Test Point
*** Validate the expected string using '''user''' data
** Add a ''NOTE_INFO(..)'' to log that the test is executing

* '''TestExpect_Misc::Exercise'''
** Validate 2 sequences using a Trigger in between
*** Sequence 1 = '''D''' '''E''' and '''A'''
*** Trigger = '''C'''
*** Sequence 2 = '''F''' and '''F''' (2 occurrences)
** Use ''ANY AT ALL'' special member with trigger
** Use ''Ordered'' and ''Strict'' sequencing
** Use ''sut_start_thread(SEQ_2)'' to generate the expected sequences

* Execute ''Test Expectations'' Test Units only

> stride --options_file myoptions.txt --run TestExpect_Seq --run TestExpect_Data --run TestExpect_Misc

Loading database...
Connecting to device...
Executing...
test unit "TestExpect_Seq"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestExpect_Data"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestExpect_Misc"
> 4 passed, 1 failed, 0 in progress, 0 not in use.
-----------------------------------------------------------
Summary: 10 passed, 1 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. If you have not added test space options to your options file (<tt>myoptions.txt</tt>) please see [[Training_Getting_Started#Test_Space_Access| testspace access]].

> stride --options_file myoptions.txt --run TestExpect_Seq --run TestExpect_Data --run TestExpect_Misc --space TestExpect --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===

* [[Source_Instrumentation_Overview | Instrumentation Overview]] providing high-level concepts of this validation technique
* [[Test_Point |Test Point]] Macro definition
* [[Expectations |Expectations]] definition and how to set your ''Expectations''

=== Samples ===

* [[Test_Point_Sample | Test Point Sample]] - Demonstrates simple technique to monitor and test activity occurring in another thread

[[Category: Training]]

Building an Off-Target Test App

2013-02-27T16:56:00Z

Jeffs:

== Prerequisite ==
This article guides you through building a test application ('''TestApp''') for the purpose of running sample code using the [[STRIDE Off-Target Environment]].

It requires an installation of the STRIDE Framework desktop package. If not installed, please see [[Desktop Installation]] for more information.

It also requires that your desktop contains one of the following '''compilers''':
* For Windows, Microsoft Visual Studio 2008 or later is required. If you don't already have Visual Studio, the free [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio_Express Visual C++ Express] can be used (download [http://www.microsoft.com/express/download/#webInstall here]). In case you have [http://www.cygwin.com Cygwin] installed, the [http://en.wikipedia.org/wiki/GNU_Compiler_Collection GNU Compiler Collection] could be used as an alrternative.
* For Linux, the [http://en.wikipedia.org/wiki/GNU_Compiler_Collection GNU Compiler Collection] (included by default in almost all Linux distros) is required.

== Building a TestApp ==

=== SDK Makefile ===
The SDK Makefile is set up so that all <tt>.c</tt> <tt>.cpp</tt> and <tt>.h</tt> files found in the directory <tt>SDK\Windows\sample_src</tt> (or <tt>SDK/Posix/sample_src</tt> for Linux) are included in the compile and link of the '''testapp''' target.

Further--as a pre-compilation step--any <tt>.h</tt> files found in <tt>sample_src</tt> are submitted to the [[STRIDE Build Tools]]. This will result in
* the detection of [[Test_Unit_Pragmas| test pragmas]] used to declare Test Units in these <tt>.h</tt> files
* the detection of [[Scl_function | function pragmas]] used to declare remoting of functions also found in <tt>.h</tt> files
* the inclusion of metadata into the <tt>sidb</tt> file created
* the generation of an [[Intercept Module]] required for executing tests

=== Build Steps ===
To begin, be sure that TestApp is not running then perform the following steps:

NOTE: ''If you experience any build problem please make sure to read [[Troubleshooting Build Problems]] for possible resolution.''

====Linux====
<ol>
<li>Build the test app
<source lang="bash">
cd $STRIDE_DIR/SDK/Posix/src
make testapp
</source>
</li>
<li>Note that the following artifacts are produced by the build:

;<tt>$STRIDE_DIR/SDK/Posix/out/bin/TestApp</tt>
: the test application
;<tt>$STRIDE_DIR/SDK/Posix/out/TestApp.sidb</tt>
: the STRIDE interface database file which contains metadata describing the interfaces remoted by the test app (along with other data)
</li>
</ol>

====Windows====
NOTE: ''In case you have [http://www.cygwin.com Cygwin] and [http://en.wikipedia.org/wiki/GNU_Compiler_Collection GNU Compiler Collection] installed and prefer to use it, please follow the build steps for Linux ([[#Linux|above]]) and ignore the one in here.''
<ol>
<li>If using Microsoft Visual Studio, open a [http://msdn.microsoft.com/en-us/library/ms235639(v=vs.100).aspx Visual Studio Command Prompt] to ensure that the compiler and linker are on your PATH.
</li>
<li>Build the test app using the supplied GNU make. (You will get Makefile errors if you use the default make.)
<source lang="dos">
cd %STRIDE_DIR%\SDK\Windows\src
..\bin\make testapp
</source>
<li>Note that the following artifacts are produced by the build:

;<tt>%STRIDE_DIR%\SDK\Windows\out\bin\TestApp.exe</tt>
: the test application
;<tt>%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb</tt>
: the STRIDE interface database file which contains metadata describing the interfaces remoted by the test app (along with other data)
</li>
</ol>

NOTE: ''If you prefer to use Visual Studio to build/debug/run your testapp, we provide instructions [[STRIDE_Extensions_for_Visual_Studio|here]] about how to accomplish this.''

== Diagnostics ==
The test app we just built does not have any user tests in it. At this point it provides a starting point for test that we will subsequently add.

However, a set of diagnostic tests that verify operation of the STRIDE runtime itself are always built into the generated TestApp executable. If desired (we recommend you to do so) you could run them by doing the following:

<ol>
<li>Invoke the TestApp. In order to see TestApp's output, we recommend that you manually run in a console window (or Windows equivalent):
;Linux
<source lang="bash">
$STRIDE_DIR/SDK/Posix/out/bin/TestApp
</source>
;Windows
<source lang="dos">
%STRIDE_DIR%\SDK\Windows\out\bin\TestApp
</source>
(...or launch from the file explorer)

<li> Note TestApp's output upon startup.
<pre>
--------------------------------------------------
STRIDE Test Console Application.
Enter 'Ctrl+C' to Quit.
--------------------------------------------------
Listening on TCP port 8000
starting up...
"_srThread" thread started.
"stride" thread started.
</pre>
</li>
<li>From a second console window, invoke <tt>[[STRIDE_Runner|stride]]</tt> as follows, to verify connectivity with the test app and STRIDE runtime operation:
 
;Linux
<source lang="bash">
stride --diagnostics --database="$STRIDE_DIR/SDK/Posix/out/TestApp.sidb" --device=TCP:localhost:8000 --run="*"
</source>
;Windows
<source lang="dos">
stride --diagnostics --database="%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb" --device=TCP:localhost:8000 --run="*"
</source>

As the tests run you will see output in both the TestApp (target) and stride (host) console windows.

The host console window output is shown here:
<pre>
Loading database...
Connecting to device...
runtime version: 4.3.0x
Executing diagnostics...
test unit "Link"
Loopback ............
Payload Fragmentation
Stub-Proxy Deadlock
Target Characteristics
> 4 passed, 0 failed, 0 in progress, 0 not in use.
test unit "Stat"
> 2 passed, 0 failed, 0 in progress, 0 not in use.
test unit "Time"
> 2 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 8 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...
</pre>
</li>
<li>Note the Summary results shown in the host output; all in use tests should pass.
</li>
<li>To exit TestApp, give the target window focus and enter Ctrl-C (or 'q' under Windows).
</ol>

== Samples ==

The initial desktop installation of STRIDE does not set up any source code (with the exception of a set of system diagnostic tests) for automatic inclusion in a test application. The [[Desktop Installation | desktop framework]] distribution, however, comes with a set of [[Samples_Overview | Samples]].

To demonstrate how to build a sample, will will add the [[Test Intro Sample]] that provide an overview of STRIDE testing techniques. For an overview from a C++ perspective, please see [[Test Intro Cpp Sample]].

The following steps are applicable for '''all''' [[Samples_Overview | Samples]].

To begin, be sure that TestApp is not running, then copy the <tt>.c</tt> and <tt>.h</tt> files found in <tt>Samples/test_in_c_cpp/TestIntro</tt> to <tt>SDK/Windows/sample_src</tt> (or <tt>SDK/Posix/sample_src</tt> for Linux).

'''''Note:''' Only files in the sample_src directory will be picked up by the makefile. Files in any subdirectories will be igored.
''

Once the files have been copied to <tt>sample_src</tt>, simply build TestApp as described above. Note if you had previously copied other sample source to this directory, you should decide whether to remove those files first. When you complete the test app build, any source that is in this directory at the time of build will be included in the test app.

=== Running the Tests ===
Here we will run all tests in the <tt>TestApp.sidb</tt> database.<ref>Note that the S2 diagnostic tests are treated separately, and are not run unless the <tt>--diagnostics</tt> option is specified to <tt>stride</tt>.</ref>

# Run the build above TestApp in a console window.
# Invoke <tt>stride</tt> in a separate console window (different from the running TestApp) -- as shown below and verify Summary results.

Here are the command line parameters that we will submit to <tt>stride</tt>

<pre>
--database ./out/TestApp.sidb
--device TCP:localhost:8000
</pre>

A couple of things to note:
* If you setup an [[STRIDE_Runner#Environment_Variables | environment variable]] for the '''device''' option than it is not required in the option file. Note: Command line options override environment variables.
* You may want to create a text file named ''RunTestIntro.txt'' in the <tt>SDK\Windows</tt> (or <tt>SDK/Posix</tt> for Linux) directory as an option file to submit to <tt>stride</tt>.
* ''stride --help'' provides [[STRIDE_Runner#Options | options information]]

If you haven't done so already, start <tt>TestApp</tt> running in a separate console window.

Now run stride as follows (starting from the <tt>SDK\Windows</tt> or <tt>SDK/Posix</tt> directory):

<source lang="bash">
stride --options_file RunTestIntro.txt --run="*"
</source>

The output should look like this:
<pre>
Loading database...
Connecting to device...
Executing...
test unit "s2_testintro_flist"
> 2 passed, 1 failed, 0 in progress, 0 not in use.
test unit "s2_testintro_cclass"
> 1 passed, 1 failed, 0 in progress, 0 not in use.
test unit "s2_testintro_testdoubles"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "s2_testintro_testpoints"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "s2_testintro_parameters"
> 2 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 11 passed, 2 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...
</pre>

=== Interpreting Results ===
Open '''TestApp.xml''' in your browser; this file is created in the directory from which you ran '''stride''' (or the directory via the <tt>--output</tt> command line option). If you were connected to the Internet when you ran the tests, the TestApp.xsl file is also generated in the directory. By opening TestApp.xml in a web browser, the xsl is automatically applied to create html in the browser. If you use Google Chrome, please see [[Browser Compatibility]].

If you're interested in the details of the tests, please see to the test documentation contained in the test report.

==Notes==
<references/>

[[Category:Installation]]

Training Getting Started

2013-02-27T16:47:31Z

Jeffs: /* Run Training Tests */

== Overview ==

Our training approach is based on a self-guided tour of the [[STRIDE_Overview#STRIDE_Testing_Features | STRIDE Testing Features]] using reference examples and assigned exercises. The set of examples and the implemented exercises will be built and executed using a standard desktop computer.

The ''software under test'', ''reference examples'', and ''exercises'' have been designed to be as '''''simple as possible while sufficiently demonstrating the topic at hand'''''. In particular, the ''software under test'' is '''very light''' on core application logic -- the focus instead is on the test code that leverages STRIDE to define and execute tests.

The user is expected to work through each of the training modules covering a specific testing feature. Once the exercises are completed (actual test cases implemented), the results are published to [[STRIDE_Overview#STRIDE_Test_Space | Test Space]] (and validated against a pre-created baseline).

The training collateral consists of the following:
# The [[STRIDE_Overview#STRIDE_Framework | STRIDE Framework]] used to implement and execute tests
#* The Framework is configured in an [[STRIDE_Off-Target_Environment | Off-Target Environment]]
# A set of specific [[#Training | Training Modules]] that will guide you through the exercises
# [[Main_Page | ''Wiki articles'']] that will provide background material and other technical information
# [[STRIDE_Overview#STRIDE_Test_Space | STRIDE Test Space]]

For more details concerning STRIDE refer to the following:
* ''[[STRIDE_Overview_Video | Overview screencast]]''
* ''[[What is Unique About STRIDE | What is Unique About STRIDE]]''
* ''[[Types of Testing Supported by STRIDE | Types of Testing Supported]]''
* ''[[Frequently Asked Questions About STRIDE | Frequently Asked Questions]]''

''For questions and support [mailto:training@s2technologies.com email us]''

== Before Starting ==
Before you can start setting up your environment you need the following 3 items:
* '''STRIDE Desktop Installation Package''' (one of the following)
**<tt>STRIDE_framework-windows_4.3.yy.zip</tt> (Windows desktop)
**<tt>STRIDE_framework-linux_4.3.yy.tgz</tt> (Linux desktop)

* '''Training Source files'''
:<tt>STRIDE_training_source.zip</tt>

* '''Test Space User Account URL and Logon Credentials'''
** Test Space URL (<tt>YourCompany.stridetestspace.com</tt>)
** User-name
** User-password

=== Desktop Setup ===
The training requires that you install the [[Desktop_Installation | Desktop Framework]] and that the [[STRIDE_Off-Target_Environment | Off-Target Environment]] is setup correctly and verified.

For an overview of the installation steps please refer to [[Installation_Overview | this]] article.

The following steps are required:
# Install your [[Desktop_Installation | desktop Framework package]]
# Read about the [[STRIDE_Off-Target_Environment | Off-Target Environment]]
# Install [[STRIDE_Off-Target_Environment#Host_Compiler | host C++ compiler]] for your desktop (if needed)
# Use the Off-Target SDK to [[Building_an_Off-Target_Test_App | build a Test App]]
# [[Building_an_Off-Target_Test_App#Diagnostics | Run STRIDE diagnostics]] with the Test App built

=== Training Setup ===
The following source code can be found in the '''STRIDE_training_source.zip''' file:

software_under_test.c | h
TestBasic.cpp | h
TestParam.cpp | h
TestFixture.cpp | h
TestExpect.cpp | h
TestRuntime.cpp | h
TestFile.cpp | h
TestDouble.cpp | h

The ''software under test'' is contained in the file '''software_under_test.c | h'''. All of the public functions use '''sut_''' as a prefix. All training modules test against this file. Although the test examples are contained in C++ files, most of the test logic is written in standard C.

* Extract STRIDE_training_source.zip into the directory <tt>%STRIDE_DIR%\SDK\Windows\sample_src</tt> (or <tt>$STRIDE_DIR/SDK/Posix/sample_src</tt> for Linux)
* Rebuild the TestApp following the instructions for [[Building_an_Off-Target_Test_App#Build_Steps | Building a TestApp]].
* List all [[Test_Units | Test Units]] within the generated database file using the following command:

'''Windows'''
> stride --database="%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb" --list

'''Linux'''
> stride --database="$STRIDE_DIR/SDK/Posix/out/TestApp.sidb" --list

The following remoted Functions and Test Units should be displayed:

Functions
sut_strcpy(char const * input, char * output) : void
strlen(char const * _Str) : size_t
Test Units
TestBasic()
TestDouble_Reference()
TestDouble_Definition()
TestExpect_Seq()
TestExpect_Data()
TestExpect_Misc()
TestFile()
TestFixture()
TestParam(int data1, int data2, char * szString)
TestRuntime_Static()
TestRuntime_Dynamic(int NumberOfTestCases)

=== Run Training Tests ===
Here we will run all tests in the <tt>TestApp.sidb</tt> database.<ref>Note that the S2 diagnostic tests are treated separately, and are not run unless the <tt>--diagnostics</tt> option is specified to <tt>stride</tt>.</ref>

* If you haven't done so already, [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using the SDK makefile
* Invoke TestApp found in the ''/out/bin'' directory
* Create an [[Stride_Runner#Options | option file]] (<tt>myoptions.txt</tt>) in a directory of your choice.
'''Windows'''
##### Command Line Options ######
--device "TCP:localhost:8000"
--database "%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb"
--output "%STRIDE_DIR%\SDK\Windows\sample_src\TestApp.xml"
--log_level all

'''Linux'''
##### Command Line Options #####
--device "TCP:localhost:8000"
--database "$STRIDE_DIR/SDK/Posix/out/TestApp.sidb"
--output "$STRIDE_DIR/SDK/Posix/sample_src/TestApp.xml"
--log_level all

A couple of things to note:
* If you set up an [[STRIDE_Runner#Environment_Variables | environment variable]] for the '''device''' option then it is not required in the option file. Note: Command line options override environment variables.
* ''stride --help'' provides [[STRIDE_Runner#Options | options information]]

If you haven't done so already, start <tt>TestApp</tt> running in a separate console window.

Now run stride as shown below and verify summary results:

> stride --options_file myoptions.txt --run "*"

The output should look like this:
<pre>
Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestParam"
> 0 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------------------
Summary: 21 passed, 3 failed, 0 in progress, 11 not in use.

Disconnecting from device...
Saving result file...

</pre>

=== Interpreting Results ===
Open '''TestApp.xml''' in your browser; this file is created in the directory from which you ran '''stride''' (or at the path specified via the <tt>--output</tt> command line option). If you were connected to the Internet when you ran the tests, the TestApp.xsl file is also generated in the directory. By opening TestApp.xml in a web browser, the xsl is automatically applied to create html in the browser. If you use Google Chrome, please see [[Browser Compatibility]].

If you're interested in the details of the tests, please see the test documentation contained in the test report.

=== Test Space Access ===

As part of the training, users implement exercises (test cases). As each exercise is completed, the results are expected to be uploaded to Test Space. Accessing Test Space (uploading, viewing, etc.) requires a user-name and password. Before working on a training module please confirm that your user account has been set up by [[Test_Space_Setup | logging in]].

Test Space will have expected results using [[Creating_And_Using_Baselines | baselines]]. This is used to automatically verify if the exercises have been implemented correctly (at least to some degree). For capturing test results a '''Training''' project has been created with the following '''Spaces''':

Sandbox - Results for training setup
TestBasic - Basics training results: TestBasic.cpp | h
TestParam - Parameters training results: TestParam.cpp | h
TestFixture - Fixturing training results: TestFixture.cpp | h
TestExpect - Expectations training results: TestExpect.cpp | h
TestRuntime - Runtime API training results: TestRuntime.cpp | h
TestFile - File IO training results: TestFile.cpp | h
TestDouble - Doubling training results: TestDouble.cpp | h

To publish your results using the [[STRIDE_Runner | STRIDE Runner]] the following command-line options should be used:

--testspace <nowiki>username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--space ''MODULENAME''
--name ''YOURNAME''

Notes:
* Concerning the ''MODULENAME'' option: use the name that corresponds to the training module currently underway (e.g. TestBasic, TestParam, etc.)
* ''YOURNAME'' should be set to your name (i.e. JohnD); omit spaces from this string
* If you access the Internet via an HTTP proxy please read [[STRIDE_Runner#Using_a_Proxy | '''this article''']]

To make it easier for now we recommend that you update your existing option file (<tt>myoptions.txt</tt>) with the following:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

=== Publish Test Space Results ===

To complete the setup publish your results to Test Space. Please make sure to use '''Sandbox''' as the space to upload your results to (see below).

> stride --options_file myoptions.txt --run "*" --space Sandbox --upload

Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestParam"
> 0 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------------------
Summary: 21 passed, 3 failed, 0 in progress, 11 not in use.
Disconnecting from device...
Saving result file...
Uploading to test space...

=== Confirming Setup ===

After you have run and uploaded the results of the training setup, you should confirm the correctness of your work. The training setup results will end up with a mix of passes and failures, so we use a [[Creating_And_Using_Baselines | Test Space baseline]] to get an aggregate comparison between expected and actual results.

To validate your setup:
# Navigate to Sandbox set in Test Space and look under the column labeled '''''SEQUENTIAL COMPARISON'''''
# Confirm that '''''none''''' is indicated for each of the sub-column. If so, you have completed the setup and are ready to move onto the training modules.

[[image:Training_Confirming_Setup.jpg|none|600px| Test Space baseline example]]

== Training ==
At this point you should be ready to start the actual training. There are '''7 separate training modules''' and we recommend the following order:

===Introductory===
* [[Training_Basics | '''Basics''']] - Covers basics of implementing and executing test cases
* [[Training_Parameters |'''Parameters''']] - How to pass parameters to a test
* [[Training_Fixturing | '''Fixturing''']] - Leveraging setup and teardown featuring

===Advanced===
* [[Training_Expectations | '''Expectations''']] - Validating code sequencing along with state data
* [[Training_Runtime_API | '''Runtime API''']] - Using the runtime services to dynamically create test suits / cases
* [[Training_File_IO | '''File IO''']] - Reading and writing files existing on the host
* [[Training_Doubling | '''Doubling''']] - Replacing a dependency with a stub, fake, or mock

== Training Confirmation ==

As you run and upload each test unit containing your worked training exercises, you should confirm the correctness of your work.

The correctly worked training test units will end up with a mix of passes and failures, so we use a [[Creating_And_Using_Baselines | Test Space baseline]] to get an aggregate comparison between expected and actual results

To validate your work:
# Navigate to your result set in Test Space and look under the column labeled '''''SEQUENTIAL COMPARISON'''''
# Confirm that '''''none''''' is indicated for each of the sub-column.

==Notes==
<references/>

[[Category: Training]]

Training Doubling

2013-02-02T00:34:16Z

Jeffs: Undo revision 13914 by Jeffs (Talk)

== Objectives ==

This Training Module is focused on explaining how to leverage '''Test Doubles''' in the context of executing a test. For an overview of '''intercepting''' existing functions please refer to [[Using_Test_Doubles | Test Doubles]]. The module covers following topics:
* How to apply pragmas for [[Function_Capturing | function intercepting]]
* How to decide what kind of ''mangling'' is required
** ''Definition'' verses ''Reference''
** ''Explicit'' verses ''Implicit''
* Setting and Resetting the Double implementation

There are two new files used -- '''TestDouble.cpp & TestDouble.h''' --- that implement two test Units:
* '''TestDouble_Reference'''
* '''TestDouble_Definition'''

All of the Test Units have test cases already implemented (used for reference) and have one test method that you are required to implement called '''Exercise'''. Currently the exercise methods return a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Double'' Test Units only

> stride --options_file myoptions.txt --run TestDouble_Reference --run TestDouble_Definition

Loading database...
Connecting to device...
Executing...
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------
Summary: 4 passed, 0 failed, 0 in progress, 2 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestDouble_Reference::Exercise'''
** Implement a new ''test double'' for the ''strlen()'' routine:
*** Add a ''NOTE'' capturing its name when it is called
*** Uses the real ''strlen()'' to return the length of the passed in string
** Use ''srEXPECT_EQ()'' to validate that ''sut_strcheck()'' returns correct length
** Make sure to restore the original routine

* '''TestDouble_Definition::Exercise'''
** Implement a new ''test double'' for the ''sut_strcpy()'' routine:
*** Log its name when it is called
*** Validate string passed to ''sut_strsave()'' is received correctly by the ''test double''
*** Remember that ''sut_strsave()'' calls ''sut_strcpy()'' with the string passed to it
*** Can use a test macro to validate the string is correctly passed to the double
*** Call the original function (''sut_strcpy()'') with a '''DIFFERENT STRING''' (i.e "Exercise Test String 2")
**** Call the original within the test double
**** Make sure to reset the original function within the mock before calling it (i.e. srDOUBLE_RESET)
** Call ''sut_strsave()'' with a string (i.e. "Exercise Test String 1")
** Use ''sut_strget()'' to retrieve a string
** Compare retrieved string with the '''DIFFERENT STRING''' inserted by the ''test double''
** NOTE - won't work if you wait until the end of the test to restore the original routine (i.e. srDOUBLE_SET)

* Execute ''Test Double'' Test Units only

> stride --options_file myoptions.txt --run TestDouble_Reference TestDouble_Definition

Loading database...
Connecting to device...
Executing...
test unit "TestDouble_Reference"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestDouble_Definition"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------
Summary: 6 passed, 0 failed, 0 in progress, 0 not in use.

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. If you have not added test space options to your options file (<tt>myoptions.txt</tt>) please see [[Training_Getting_Started#Test_Space_Access| testspace access]].

> stride --options_file myoptions.txt --run TestDouble_Reference TestDouble_Definition --space TestDouble --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===
* [[Using_Test_Doubles | Using Test Doubles]] - Outlines the basic steps to enable a ''double''
** Capture the f(x) that is going to be doubled
** Code the test logic to additionally set the Double and restore original f(x)

* [[Scl_function | Intercepting a function]] - Pragma specifics outline
** For this exercise, there are '''NO''' optional parameters when capturing a f(x) to be '''intercepted'''

* Details of the pragma parameters can be understood in the [[Intercept_Module#Interceptor | Intercept Module]] article. Please review in detail the following:
** ''context'' - This refers to incepting a callee or the called routine.
** ''name-mangling'' - How the function name is switched during the compiling process
** ''group-id'' - Used to associate a group of intercepted functions

* Some '''external links''' to review:
** [http://www.martinfowler.com/bliki/TestDouble.html Martin Fowler definition]
** [http://en.wikipedia.org/wiki/Test_Double Wikipedia definition]
** [http://msdn.microsoft.com/en-us/magazine/cc163358.aspx MSDN Magazine description]

=== Samples ===

* [[Test_Double_Sample | Test Double Sample]] that can be a useful reference. This reference example can be built and executed like all of samples using the Off-Target environment.

[[Category: Training]]

Training Doubling

2013-02-02T00:31:35Z

Jeffs: Reverted edits by Jeffs (Talk) to last revision by Marku

== Objectives ==

This Training Module is focused on explaining how to leverage '''Test Doubles''' in the context of executing a test. For an overview of '''intercepting''' existing functions please refer to [[Using_Test_Doubles | Test Doubles]]. The module covers following topics:
* How to apply pragmas for [[Function_Capturing | function intercepting]]
* How to decide what kind of ''mangling'' is required
** ''Definition'' verses ''Reference''
** ''Explicit'' verses ''Implicit''
* Setting and Resetting the Double implementation

There are two new files used -- '''TestDouble.cpp & TestDouble.h''' --- that implement two test Units:
* '''TestDouble_Reference'''
* '''TestDouble_Definition'''

All of the Test Units have test cases already implemented (used for reference) and have one test method that you are required to implement called '''Exercise'''. Currently the exercise methods return a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If not already done, create an [[Stride_Runner#Options | option file]] (myoptions.txt) using the following content (Windows example)

##### Command Line Options ######
--device "TCP:localhost:8000"
--database "%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb"
--output "%STRIDE_DIR%\SDK\Windows\sample_src\TestApp.xml"
--log_level all

* Execute ''Test Double'' Test Units only

> stride --options_file myoptions.txt --run TestDouble_Reference --run TestDouble_Definition

Loading database...
Connecting to device...
Executing...
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------
Summary: 4 passed, 0 failed, 0 in progress, 2 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestDouble_Reference::Exercise'''
** Implement a new ''test double'' for the ''strlen()'' routine:
*** Add a ''NOTE'' capturing its name when it is called
*** Uses the real ''strlen()'' to return the length of the passed in string
** Use ''srEXPECT_EQ()'' to validate that ''sut_strcheck()'' returns correct length
** Make sure to restore the original routine

* '''TestDouble_Definition::Exercise'''
** Implement a new ''test double'' for the ''sut_strcpy()'' routine:
*** Log its name when it is called
*** Validate string passed to ''sut_strsave()'' is received correctly by the ''test double''
*** Remember that ''sut_strsave()'' calls ''sut_strcpy()'' with the string passed to it
*** Can use a test macro to validate the string is correctly passed to the double
*** Call the original function (''sut_strcpy()'') with a '''DIFFERENT STRING''' (i.e "Exercise Test String 2")
**** Call the original within the test double
**** Make sure to reset the original function within the mock before calling it (i.e. srDOUBLE_RESET)
** Call ''sut_strsave()'' with a string (i.e. "Exercise Test String 1")
** Use ''sut_strget()'' to retrieve a string
** Compare retrieved string with the '''DIFFERENT STRING''' inserted by the ''test double''
** NOTE - won't work if you wait until the end of the test to restore the original routine (i.e. srDOUBLE_SET)

* Execute ''Test Double'' Test Units only

> stride --options_file myoptions.txt --run TestDouble_Reference TestDouble_Definition

Loading database...
Connecting to device...
Executing...
test unit "TestDouble_Reference"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestDouble_Definition"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------
Summary: 6 passed, 0 failed, 0 in progress, 0 not in use.

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following if not already done:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

> stride --options_file myoptions.txt --run TestDouble_Reference TestDouble_Definition --space TestDouble --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===
* [[Using_Test_Doubles | Using Test Doubles]] - Outlines the basic steps to enable a ''double''
** Capture the f(x) that is going to be doubled
** Code the test logic to additionally set the Double and restore original f(x)

* [[Scl_function | Intercepting a function]] - Pragma specifics outline
** For this exercise, there are '''NO''' optional parameters when capturing a f(x) to be '''intercepted'''

* Details of the pragma parameters can be understood in the [[Intercept_Module#Interceptor | Intercept Module]] article. Please review in detail the following:
** ''context'' - This refers to incepting a callee or the called routine.
** ''name-mangling'' - How the function name is switched during the compiling process
** ''group-id'' - Used to associate a group of intercepted functions

* Some '''external links''' to review:
** [http://www.martinfowler.com/bliki/TestDouble.html Martin Fowler definition]
** [http://en.wikipedia.org/wiki/Test_Double Wikipedia definition]
** [http://msdn.microsoft.com/en-us/magazine/cc163358.aspx MSDN Magazine description]

=== Samples ===

* [[Test_Double_Sample | Test Double Sample]] that can be a useful reference. This reference example can be built and executed like all of samples using the Off-Target environment.

[[Category: Training]]

Training Doubling

2013-02-02T00:30:19Z

Jeffs: /* Implement Exercise */

== Objectives ==

This Training Module is focused on explaining how to leverage '''Test Doubles''' in the context of executing a test. For an overview of '''intercepting''' existing functions please refer to [[Using_Test_Doubles | Test Doubles]]. The module covers following topics:
* How to apply pragmas for [[Function_Capturing | function intercepting]]
* How to decide what kind of ''mangling'' is required
** ''Definition'' verses ''Reference''
** ''Explicit'' verses ''Implicit''
* Setting and Resetting the Double implementation

There are two new files used -- '''TestDouble.cpp & TestDouble.h''' --- that implement two test Units:
* '''TestDouble_Reference'''
* '''TestDouble_Definition'''

All of the Test Units have test cases already implemented (used for reference) and have one test method that you are required to implement called '''Exercise'''. Currently the exercise methods return a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Double'' Test Units only

> stride --options_file myoptions.txt --run TestDouble_Reference --run TestDouble_Definition

Loading database...
Connecting to device...
Executing...
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------
Summary: 4 passed, 0 failed, 0 in progress, 2 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestDouble_Reference::Exercise'''
** Implement a new ''test double'' for the ''strlen()'' routine:
*** Add a ''NOTE'' capturing its name when it is called
*** Uses the real ''strlen()'' to return the length of the passed in string
** Use ''srEXPECT_EQ()'' to validate that ''sut_strcheck()'' returns correct length
** Make sure to restore the original routine

* '''TestDouble_Definition::Exercise'''
** Implement a new ''test double'' for the ''sut_strcpy()'' routine:
*** Log its name when it is called
*** Validate string passed to ''sut_strsave()'' is received correctly by the ''test double''
*** Remember that ''sut_strsave()'' calls ''sut_strcpy()'' with the string passed to it
*** Can use a test macro to validate the string is correctly passed to the double
*** Call the original function (''sut_strcpy()'') with a '''DIFFERENT STRING''' (i.e "Exercise Test String 2")
**** Call the original within the test double
**** Make sure to reset the original function within the mock before calling it (i.e. srDOUBLE_RESET)
** Call ''sut_strsave()'' with a string (i.e. "Exercise Test String 1")
** Use ''sut_strget()'' to retrieve a string
** Compare retrieved string with the '''DIFFERENT STRING''' inserted by the ''test double''
** NOTE - won't work if you wait until the end of the test to restore the original routine (i.e. srDOUBLE_SET)

* Execute ''Test Double'' Test Units only

> stride --options_file myoptions.txt --run TestDouble_Reference TestDouble_Definition

Loading database...
Connecting to device...
Executing...
test unit "TestDouble_Reference"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestDouble_Definition"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------
Summary: 6 passed, 0 failed, 0 in progress, 0 not in use.

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. If you have not added test space options to your options file (<tt>myoptions.txt</tt>) please see [[Training_Getting_Started#Test_Space_Access| testspace access]].

> stride --options_file myoptions.txt --run TestDouble_Reference TestDouble_Definition --space TestDouble --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===
* [[Using_Test_Doubles | Using Test Doubles]] - Outlines the basic steps to enable a ''double''
** Capture the f(x) that is going to be doubled
** Code the test logic to additionally set the Double and restore original f(x)

* [[Scl_function | Intercepting a function]] - Pragma specifics outline
** For this exercise, there are '''NO''' optional parameters when capturing a f(x) to be '''intercepted'''

* Details of the pragma parameters can be understood in the [[Intercept_Module#Interceptor | Intercept Module]] article. Please review in detail the following:
** ''context'' - This refers to incepting a callee or the called routine.
** ''name-mangling'' - How the function name is switched during the compiling process
** ''group-id'' - Used to associate a group of intercepted functions

* Some '''external links''' to review:
** [http://www.martinfowler.com/bliki/TestDouble.html Martin Fowler definition]
** [http://en.wikipedia.org/wiki/Test_Double Wikipedia definition]
** [http://msdn.microsoft.com/en-us/magazine/cc163358.aspx MSDN Magazine description]

=== Samples ===

* [[Test_Double_Sample | Test Double Sample]] that can be a useful reference. This reference example can be built and executed like all of samples using the Off-Target environment.

[[Category: Training]]

Training Getting Started

2013-02-02T00:17:49Z

Jeffs: /* Before Starting */

== Overview ==

Our training approach is based on a self-guided tour of the [[STRIDE_Overview#STRIDE_Testing_Features | STRIDE Testing Features]] using reference examples and assigned exercises. The set of examples and the implemented exercises will be built and executed using a standard desktop computer.

The ''software under test'', ''reference examples'', and ''exercises'' have been designed to be as '''''simple as possible while sufficiently demonstrating the topic at hand'''''. In particular, the ''software under test'' is '''very light''' on core application logic -- the focus instead is on the test code that leverages STRIDE to define and execute tests.

The user is expected to work through each of the training modules covering a specific testing feature. Once the exercises are completed (actual test cases implemented), the results are published to [[STRIDE_Overview#STRIDE_Test_Space | Test Space]] (and validated against a pre-created baseline).

The training collateral consists of the following:
# The [[STRIDE_Overview#STRIDE_Framework | STRIDE Framework]] used to implement and execute tests
#* The Framework is configured in an [[STRIDE_Off-Target_Environment | Off-Target Environment]]
# A set of specific [[#Training | Training Modules]] that will guide you through the exercises
# [[Main_Page | ''Wiki articles'']] that will provide background material and other technical information
# [[STRIDE_Overview#STRIDE_Test_Space | STRIDE Test Space]]

For more details concerning STRIDE refer to the following:
* ''[[STRIDE_Overview_Video | Overview screencast]]''
* ''[[What is Unique About STRIDE | What is Unique About STRIDE]]''
* ''[[Types of Testing Supported by STRIDE | Types of Testing Supported]]''
* ''[[Frequently Asked Questions About STRIDE | Frequently Asked Questions]]''

''For questions and support [mailto:training@s2technologies.com email us]''

== Before Starting ==
Before you can start setting up your environment you need the following 3 items:
* '''STRIDE Desktop Installation Package''' (one of the following)
**<tt>STRIDE_framework-windows_4.3.yy.zip</tt> (Windows desktop)
**<tt>STRIDE_framework-linux_4.3.yy.tgz</tt> (Linux desktop)

* '''Training Source files'''
:<tt>STRIDE_training_source.zip</tt>

* '''Test Space User Account URL and Logon Credentials'''
** Test Space URL (<tt>https://YourCompany.stridetestspace.com</tt>)
** User-name
** User-password

=== Desktop Setup ===
The training requires that you install the [[Desktop_Installation | Desktop Framework]] and that the [[STRIDE_Off-Target_Environment | Off-Target Environment]] is setup correctly and verified.

For an overview of the installation steps please refer to [[Installation_Overview | this]] article.

The following steps are required:
# Install your [[Desktop_Installation | desktop Framework package]]
# Read about the [[STRIDE_Off-Target_Environment | Off-Target Environment]]
# Install [[STRIDE_Off-Target_Environment#Host_Compiler | host C++ compiler]] for your desktop (if needed)
# Use the Off-Target SDK to [[Building_an_Off-Target_Test_App | build a Test App]]
# [[Building_an_Off-Target_Test_App#Diagnostics | Run STRIDE diagnostics]] with the Test App built

=== Training Setup ===
The following source code can be found in the '''STRIDE_training_source.zip''' file:

software_under_test.c | h
TestBasic.cpp | h
TestParam.cpp | h
TestFixture.cpp | h
TestExpect.cpp | h
TestRuntime.cpp | h
TestFile.cpp | h
TestDouble.cpp | h

The ''software under test'' is contained in the file '''software_under_test.c | h'''. All of the public functions use '''sut_''' as a prefix. All training modules test against this file. Although the test examples are contained in C++ files, most of the test logic is written in standard C.

* Extract STRIDE_training_source.zip into the directory <tt>%STRIDE_DIR%\SDK\Windows\sample_src</tt> (or <tt>$STRIDE_DIR/SDK/Posix/sample_src</tt> for Linux)
* Rebuild the TestApp following the instructions for [[Building_an_Off-Target_Test_App#Build_Steps | Building a TestApp]].
* List all [[Test_Units | Test Units]] within the generated database file using the following command:

'''Windows'''
> stride --database="%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb" --list

'''Linux'''
> stride --database="$STRIDE_DIR/SDK/Posix/out/TestApp.sidb" --list

The following remoted Functions and Test Units should be displayed:

Functions
sut_strcpy(char const * input, char * output) : void
strlen(char const * _Str) : size_t
Test Units
TestBasic()
TestDouble_Reference()
TestDouble_Definition()
TestExpect_Seq()
TestExpect_Data()
TestExpect_Misc()
TestFile()
TestFixture()
TestParam(int data1, int data2, char * szString)
TestRuntime_Static()
TestRuntime_Dynamic(int NumberOfTestCases)

=== Run Training Tests ===
Here we will run all tests in the <tt>TestApp.sidb</tt> database.<ref>Note that the S2 diagnostic tests are treated separately, and are not run unless the <tt>--diagnostics</tt> option is specified to <tt>stride</tt>.</ref>

* If you haven't done so already, [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using the SDK makefile
* Invoke TestApp found in the ''/out/bin'' directory
* Create an [[Stride_Runner#Options | option file]] (<tt>myoptions.txt</tt>) that includes the following content in <tt>%STRIDE_DIR%\SDK\Windows\myoptions.txt</tt> (or <tt>$STRIDE_DIR/SDK/Posix/myoptions.txt</tt> for Linux)
'''Windows'''
##### Command Line Options ######
--device "TCP:localhost:8000"
--database "%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb"
--output "%STRIDE_DIR%\SDK\Windows\sample_src\TestApp.xml"
--log_level all

'''Linux'''
##### Command Line Options #####
--device "TCP:localhost:8000"
--database "$STRIDE_DIR/SDK/Posix/out/TestApp.sidb"
--output "$STRIDE_DIR/SDK/Posix/sample_src/TestApp.xml"
--log_level all

A couple of things to note:
* If you set up an [[STRIDE_Runner#Environment_Variables | environment variable]] for the '''device''' option then it is not required in the option file. Note: Command line options override environment variables.
* ''stride --help'' provides [[STRIDE_Runner#Options | options information]]

If you haven't done so already, start <tt>TestApp</tt> running in a separate console window.

Now run stride as shown below and verify summary results (starting from the <tt>SDK\Windows</tt> or <tt>SDK/Posix</tt> directory):

> stride --options_file myoptions.txt --run "*"

The output should look like this:
<pre>
Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestParam"
> 0 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------------------
Summary: 21 passed, 3 failed, 0 in progress, 11 not in use.

Disconnecting from device...
Saving result file...

</pre>

=== Interpreting Results ===
Open '''TestApp.xml''' in your browser; this file is created in the directory from which you ran '''stride''' (or at the path specified via the <tt>--output</tt> command line option). If you were connected to the Internet when you ran the tests, the TestApp.xsl file is also generated in the directory. By opening TestApp.xml in a web browser, the xsl is automatically applied to create html in the browser. If you use Google Chrome, please see [[Browser Compatibility]].

If you're interested in the details of the tests, please see the test documentation contained in the test report.

=== Test Space Access ===

As part of the training, users implement exercises (test cases). As each exercise is completed, the results are expected to be uploaded to Test Space. Accessing Test Space (uploading, viewing, etc.) requires a user-name and password. Before working on a training module please confirm that your user account has been set up by [[Test_Space_Setup | logging in]].

Test Space will have expected results using [[Creating_And_Using_Baselines | baselines]]. This is used to automatically verify if the exercises have been implemented correctly (at least to some degree). For capturing test results a '''Training''' project has been created with the following '''Spaces''':

Sandbox - Results for training setup
TestBasic - Basics training results: TestBasic.cpp | h
TestParam - Parameters training results: TestParam.cpp | h
TestFixture - Fixturing training results: TestFixture.cpp | h
TestExpect - Expectations training results: TestExpect.cpp | h
TestRuntime - Runtime API training results: TestRuntime.cpp | h
TestFile - File IO training results: TestFile.cpp | h
TestDouble - Doubling training results: TestDouble.cpp | h

To publish your results using the [[STRIDE_Runner | STRIDE Runner]] the following command-line options should be used:

--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--space ''MODULENAME''
--name ''YOURNAME''

Notes:
* Concerning the ''MODULENAME'' option: use the name that corresponds to the training module currently underway (e.g. TestBasic, TestParam, etc.)
* ''YOURNAME'' should be set to your name (i.e. JohnD); omit spaces from this string
* If you access the Internet via an HTTP proxy please read [[STRIDE_Runner#Using_a_Proxy | '''this article''']]

To make it easier for now we recommend that you update your existing option file (<tt>myoptions.txt</tt>) with the following:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

=== Publish Test Space Results ===

To complete the setup publish your results to Test Space. Please make sure to use '''Sandbox''' as the space to upload your results to (see below).

> stride --options_file myoptions.txt --run "*" --space Sandbox --upload

Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestParam"
> 0 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------------------
Summary: 21 passed, 3 failed, 0 in progress, 11 not in use.
Disconnecting from device...
Saving result file...
Uploading to test space...

=== Confirming Setup ===

After you have run and uploaded the results of the training setup, you should confirm the correctness of your work. The training setup results will end up with a mix of passes and failures, so we use a [[Creating_And_Using_Baselines | Test Space baseline]] to get an aggregate comparison between expected and actual results.

To validate your setup:
# Navigate to Sandbox set in Test Space and look under the column labeled '''''SEQUENTIAL COMPARISON'''''
# Confirm that '''''none''''' is indicated for each of the sub-column. If so, you have completed the setup and are ready to move onto the training modules.

[[image:STRIDE_results_012813.JPG|none|600px| Test Space baseline example]]

== Training ==
At this point you should be ready to start the actual training. There are '''7 separate training modules''' and we recommend the following order:

===Introductory===
* [[Training_Basics | '''Basics''']] - Covers basics of implementing and executing test cases
* [[Training_Parameters |'''Parameters''']] - How to pass parameters to a test
* [[Training_Fixturing | '''Fixturing''']] - Leveraging setup and teardown featuring

===Advanced===
* [[Training_Expectations | '''Expectations''']] - Validating code sequencing along with state data
* [[Training_Runtime_API | '''Runtime API''']] - Using the runtime services to dynamically create test suits / cases
* [[Training_File_IO | '''File IO''']] - Reading and writing files existing on the host
* [[Training_Doubling | '''Doubling''']] - Replacing a dependency with a stub, fake, or mock

== Training Confirmation ==

As you run and upload each test unit containing your worked training exercises, you should confirm the correctness of your work.

The correctly worked training test units will end up with a mix of passes and failures, so we use a [[Creating_And_Using_Baselines | Test Space baseline]] to get an aggregate comparison between expected and actual results

To validate your work:
# Navigate to your result set in Test Space and look under the column labeled '''''SEQUENTIAL COMPARISON'''''
# Confirm that '''''none''''' is indicated for each of the sub-column.

==Notes==
<references/>

[[Category: Training]]

Training Getting Started

2013-02-02T00:15:55Z

Jeffs: /* Overview */

== Overview ==

Our training approach is based on a self-guided tour of the [[STRIDE_Overview#STRIDE_Testing_Features | STRIDE Testing Features]] using reference examples and assigned exercises. The set of examples and the implemented exercises will be built and executed using a standard desktop computer.

The ''software under test'', ''reference examples'', and ''exercises'' have been designed to be as '''''simple as possible while sufficiently demonstrating the topic at hand'''''. In particular, the ''software under test'' is '''very light''' on core application logic -- the focus instead is on the test code that leverages STRIDE to define and execute tests.

The user is expected to work through each of the training modules covering a specific testing feature. Once the exercises are completed (actual test cases implemented), the results are published to [[STRIDE_Overview#STRIDE_Test_Space | Test Space]] (and validated against a pre-created baseline).

The training collateral consists of the following:
# The [[STRIDE_Overview#STRIDE_Framework | STRIDE Framework]] used to implement and execute tests
#* The Framework is configured in an [[STRIDE_Off-Target_Environment | Off-Target Environment]]
# A set of specific [[#Training | Training Modules]] that will guide you through the exercises
# [[Main_Page | ''Wiki articles'']] that will provide background material and other technical information
# [[STRIDE_Overview#STRIDE_Test_Space | STRIDE Test Space]]

For more details concerning STRIDE refer to the following:
* ''[[STRIDE_Overview_Video | Overview screencast]]''
* ''[[What is Unique About STRIDE | What is Unique About STRIDE]]''
* ''[[Types of Testing Supported by STRIDE | Types of Testing Supported]]''
* ''[[Frequently Asked Questions About STRIDE | Frequently Asked Questions]]''

''For questions and support [mailto:training@s2technologies.com email us]''

== Before Starting ==
Before you can start setting up your environment you need the following 3 items:
* '''STRIDE Desktop Installation Package''' (one of the following)
:<tt>STRIDE_framework-windows_4.3.yy.zip</tt> (Windows desktop)
''or''
:<tt>STRIDE_framework-linux_4.3.yy.tgz</tt> (Linux desktop)

* '''Training Source files'''
:<tt>STRIDE_training_source.zip</tt>

* '''Test Space User Account URL and Logon Credentials'''
** Test Space URL (<tt>https://YourCompany.stridetestspace.com</tt>)
** User-name
** User-password

=== Desktop Setup ===
The training requires that you install the [[Desktop_Installation | Desktop Framework]] and that the [[STRIDE_Off-Target_Environment | Off-Target Environment]] is setup correctly and verified.

For an overview of the installation steps please refer to [[Installation_Overview | this]] article.

The following steps are required:
# Install your [[Desktop_Installation | desktop Framework package]]
# Read about the [[STRIDE_Off-Target_Environment | Off-Target Environment]]
# Install [[STRIDE_Off-Target_Environment#Host_Compiler | host C++ compiler]] for your desktop (if needed)
# Use the Off-Target SDK to [[Building_an_Off-Target_Test_App | build a Test App]]
# [[Building_an_Off-Target_Test_App#Diagnostics | Run STRIDE diagnostics]] with the Test App built

=== Training Setup ===
The following source code can be found in the '''STRIDE_training_source.zip''' file:

software_under_test.c | h
TestBasic.cpp | h
TestParam.cpp | h
TestFixture.cpp | h
TestExpect.cpp | h
TestRuntime.cpp | h
TestFile.cpp | h
TestDouble.cpp | h

The ''software under test'' is contained in the file '''software_under_test.c | h'''. All of the public functions use '''sut_''' as a prefix. All training modules test against this file. Although the test examples are contained in C++ files, most of the test logic is written in standard C.

* Extract STRIDE_training_source.zip into the directory <tt>%STRIDE_DIR%\SDK\Windows\sample_src</tt> (or <tt>$STRIDE_DIR/SDK/Posix/sample_src</tt> for Linux)
* Rebuild the TestApp following the instructions for [[Building_an_Off-Target_Test_App#Build_Steps | Building a TestApp]].
* List all [[Test_Units | Test Units]] within the generated database file using the following command:

'''Windows'''
> stride --database="%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb" --list

'''Linux'''
> stride --database="$STRIDE_DIR/SDK/Posix/out/TestApp.sidb" --list

The following remoted Functions and Test Units should be displayed:

Functions
sut_strcpy(char const * input, char * output) : void
strlen(char const * _Str) : size_t
Test Units
TestBasic()
TestDouble_Reference()
TestDouble_Definition()
TestExpect_Seq()
TestExpect_Data()
TestExpect_Misc()
TestFile()
TestFixture()
TestParam(int data1, int data2, char * szString)
TestRuntime_Static()
TestRuntime_Dynamic(int NumberOfTestCases)

=== Run Training Tests ===
Here we will run all tests in the <tt>TestApp.sidb</tt> database.<ref>Note that the S2 diagnostic tests are treated separately, and are not run unless the <tt>--diagnostics</tt> option is specified to <tt>stride</tt>.</ref>

* If you haven't done so already, [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using the SDK makefile
* Invoke TestApp found in the ''/out/bin'' directory
* Create an [[Stride_Runner#Options | option file]] (<tt>myoptions.txt</tt>) that includes the following content in <tt>%STRIDE_DIR%\SDK\Windows\myoptions.txt</tt> (or <tt>$STRIDE_DIR/SDK/Posix/myoptions.txt</tt> for Linux)
'''Windows'''
##### Command Line Options ######
--device "TCP:localhost:8000"
--database "%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb"
--output "%STRIDE_DIR%\SDK\Windows\sample_src\TestApp.xml"
--log_level all

'''Linux'''
##### Command Line Options #####
--device "TCP:localhost:8000"
--database "$STRIDE_DIR/SDK/Posix/out/TestApp.sidb"
--output "$STRIDE_DIR/SDK/Posix/sample_src/TestApp.xml"
--log_level all

A couple of things to note:
* If you set up an [[STRIDE_Runner#Environment_Variables | environment variable]] for the '''device''' option then it is not required in the option file. Note: Command line options override environment variables.
* ''stride --help'' provides [[STRIDE_Runner#Options | options information]]

If you haven't done so already, start <tt>TestApp</tt> running in a separate console window.

Now run stride as shown below and verify summary results (starting from the <tt>SDK\Windows</tt> or <tt>SDK/Posix</tt> directory):

> stride --options_file myoptions.txt --run "*"

The output should look like this:
<pre>
Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestParam"
> 0 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------------------
Summary: 21 passed, 3 failed, 0 in progress, 11 not in use.

Disconnecting from device...
Saving result file...

</pre>

=== Interpreting Results ===
Open '''TestApp.xml''' in your browser; this file is created in the directory from which you ran '''stride''' (or at the path specified via the <tt>--output</tt> command line option). If you were connected to the Internet when you ran the tests, the TestApp.xsl file is also generated in the directory. By opening TestApp.xml in a web browser, the xsl is automatically applied to create html in the browser. If you use Google Chrome, please see [[Browser Compatibility]].

If you're interested in the details of the tests, please see the test documentation contained in the test report.

=== Test Space Access ===

As part of the training, users implement exercises (test cases). As each exercise is completed, the results are expected to be uploaded to Test Space. Accessing Test Space (uploading, viewing, etc.) requires a user-name and password. Before working on a training module please confirm that your user account has been set up by [[Test_Space_Setup | logging in]].

Test Space will have expected results using [[Creating_And_Using_Baselines | baselines]]. This is used to automatically verify if the exercises have been implemented correctly (at least to some degree). For capturing test results a '''Training''' project has been created with the following '''Spaces''':

Sandbox - Results for training setup
TestBasic - Basics training results: TestBasic.cpp | h
TestParam - Parameters training results: TestParam.cpp | h
TestFixture - Fixturing training results: TestFixture.cpp | h
TestExpect - Expectations training results: TestExpect.cpp | h
TestRuntime - Runtime API training results: TestRuntime.cpp | h
TestFile - File IO training results: TestFile.cpp | h
TestDouble - Doubling training results: TestDouble.cpp | h

To publish your results using the [[STRIDE_Runner | STRIDE Runner]] the following command-line options should be used:

--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--space ''MODULENAME''
--name ''YOURNAME''

Notes:
* Concerning the ''MODULENAME'' option: use the name that corresponds to the training module currently underway (e.g. TestBasic, TestParam, etc.)
* ''YOURNAME'' should be set to your name (i.e. JohnD); omit spaces from this string
* If you access the Internet via an HTTP proxy please read [[STRIDE_Runner#Using_a_Proxy | '''this article''']]

To make it easier for now we recommend that you update your existing option file (<tt>myoptions.txt</tt>) with the following:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

=== Publish Test Space Results ===

To complete the setup publish your results to Test Space. Please make sure to use '''Sandbox''' as the space to upload your results to (see below).

> stride --options_file myoptions.txt --run "*" --space Sandbox --upload

Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestParam"
> 0 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------------------
Summary: 21 passed, 3 failed, 0 in progress, 11 not in use.
Disconnecting from device...
Saving result file...
Uploading to test space...

=== Confirming Setup ===

After you have run and uploaded the results of the training setup, you should confirm the correctness of your work. The training setup results will end up with a mix of passes and failures, so we use a [[Creating_And_Using_Baselines | Test Space baseline]] to get an aggregate comparison between expected and actual results.

To validate your setup:
# Navigate to Sandbox set in Test Space and look under the column labeled '''''SEQUENTIAL COMPARISON'''''
# Confirm that '''''none''''' is indicated for each of the sub-column. If so, you have completed the setup and are ready to move onto the training modules.

[[image:STRIDE_results_012813.JPG|none|600px| Test Space baseline example]]

== Training ==
At this point you should be ready to start the actual training. There are '''7 separate training modules''' and we recommend the following order:

===Introductory===
* [[Training_Basics | '''Basics''']] - Covers basics of implementing and executing test cases
* [[Training_Parameters |'''Parameters''']] - How to pass parameters to a test
* [[Training_Fixturing | '''Fixturing''']] - Leveraging setup and teardown featuring

===Advanced===
* [[Training_Expectations | '''Expectations''']] - Validating code sequencing along with state data
* [[Training_Runtime_API | '''Runtime API''']] - Using the runtime services to dynamically create test suits / cases
* [[Training_File_IO | '''File IO''']] - Reading and writing files existing on the host
* [[Training_Doubling | '''Doubling''']] - Replacing a dependency with a stub, fake, or mock

== Training Confirmation ==

As you run and upload each test unit containing your worked training exercises, you should confirm the correctness of your work.

The correctly worked training test units will end up with a mix of passes and failures, so we use a [[Creating_And_Using_Baselines | Test Space baseline]] to get an aggregate comparison between expected and actual results

To validate your work:
# Navigate to your result set in Test Space and look under the column labeled '''''SEQUENTIAL COMPARISON'''''
# Confirm that '''''none''''' is indicated for each of the sub-column.

==Notes==
<references/>

[[Category: Training]]

Building an Off-Target Test App

2013-02-01T23:26:56Z

Jeffs: /* Samples (Optional Not for Training Exercises) */

== Prerequisite ==
This article guides you through building a test application ('''TestApp''') for the purpose of running sample code using the [[STRIDE Off-Target Environment]].

It requires an installation of the STRIDE Framework desktop package. If not installed, please see [[Desktop Installation]] for more information.

It also requires that your desktop contains one of the following '''compilers''':
* For Windows Microsoft Visual Studio 2008 or later is required. If you don't already have Visual Studio, the free [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio_Express Visual C++ Express] can be used (download [http://www.microsoft.com/express/download/#webInstall here]).
* For Linux the [http://en.wikipedia.org/wiki/GNU_Compiler_Collection GNU Compiler Collection] (included by default in almost all Linux distros) is required.

== Building a TestApp ==

=== SDK Makefile ===
The SDK Makefile is set up so that all <tt>.c</tt> <tt>.cpp</tt> and <tt>.h</tt> files found in the directory <tt>SDK\Windows\sample_src</tt> (or <tt>SDK/Posix/sample_src</tt> for Linux) are included in the compile and link of the '''testapp''' target.

Further--as a pre-compilation step--any <tt>.h</tt> files found in <tt>sample_src</tt> are submitted to the [[STRIDE Build Tools]]. This will result in
* the detection of [[Test_Unit_Pragmas| test pragmas]] used to declare Test Units in these <tt>.h</tt> files
* the detection of [[Scl_function | function pragmas]] used to declare remoting of functions also found in <tt>.h</tt> files
* the inclusion of metadata into the <tt>sidb</tt> file created
* the generation of an [[Intercept Module]] required for executing tests

=== Build Steps ===
To begin, be sure that TestApp is not running then perform the following steps:

NOTE: ''If you experience any build problem please make sure to read [[Troubleshooting Build Problems]] for possible resolution.''

====Linux====
<ol>
<li>Build the test app
<source lang="bash">
cd $STRIDE_DIR/SDK/Posix/src
make testapp
</source>
</li>
<li>Note that the following artifacts are produced by the build:

;<tt>$STRIDE_DIR/SDK/Posix/out/bin/TestApp</tt>
: the test application
;<tt>$STRIDE_DIR/SDK/Posix/out/TestApp.sidb</tt>
: the STRIDE interface database file which contains metadata describing the interfaces remoted by the test app (along with other data)
</li>
</ol>

====Windows====
<ol>
<li>If using Microsoft Visual Studio, open a [http://msdn.microsoft.com/en-us/library/ms235639(v=vs.100).aspx Visual Studio Command Prompt] to ensure that the compiler and linker are on your PATH.
</li>
<li>Build the test app using the supplied GNU make. (You will get Makefile errors if you use the default make.)
<source lang="dos">
cd %STRIDE_DIR%\SDK\Windows\src
..\bin\make testapp
</source>
<li>Note that the following artifacts are produced by the build:

;<tt>%STRIDE_DIR%\SDK\Windows\out\bin\TestApp.exe</tt>
: the test application
;<tt>%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb</tt>
: the STRIDE interface database file which contains metadata describing the interfaces remoted by the test app (along with other data)
</li>
</ol>

NOTE: ''If you prefer to use Visual Studio to build/debug/run your testapp, we provide instructions [[STRIDE_Extensions_for_Visual_Studio|here]] about how to accomplish this.''

== Diagnostics ==
The test app we just built does not have any user tests in it. At this point it provides a starting point for test that we will subsequently add.

However, a set of diagnostic tests that verify operation of the STRIDE runtime itself are always built into the generated TestApp executable. If desired (we recommend you to do so) you could run them by doing the following:

<ol>
<li>Invoke the TestApp. In order to see TestApp's output, we recommend that you manually open a new console (or Windows equivalent):
;Linux
<source lang="bash">
$STRIDE_DIR/SDK/Posix/out/bin/TestApp
</source>
;Windows
<source lang="dos">
%STRIDE_DIR%\SDK\Windows\out\bin\TestApp
</source>
(...or launch from the file explorer)

<li> Note TestApp's output upon startup.
<pre>
--------------------------------------------------
STRIDE Test Console Application.
Enter 'Ctrl+C' to Quit.
--------------------------------------------------
Listening on TCP port 8000
starting up...
"_srThread" thread started.
"stride" thread started.
</pre>
</li>
<li>From a second console window, invoke <tt>[[STRIDE_Runner|stride]]</tt> as follows, to verify connectivity with the test app and STRIDE runtime operation:
 
;Linux
<source lang="bash">
stride --diagnostics --database="$STRIDE_DIR/SDK/Posix/out/TestApp.sidb" --device=TCP:localhost:8000 --run="*"
</source>
;Windows
<source lang="dos">
stride --diagnostics --database="%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb" --device=TCP:localhost:8000 --run="*"
</source>

As the tests run you will see output in both the TestApp (target) and stride (host) console windows.

The host console window output is shown here:
<pre>
Loading database...
Connecting to device...
runtime version: 4.3.0x
Executing diagnostics...
test unit "Link"
Loopback ............
Payload Fragmentation
Stub-Proxy Deadlock
Target Characteristics
> 4 passed, 0 failed, 0 in progress, 0 not in use.
test unit "Stat"
> 2 passed, 0 failed, 0 in progress, 0 not in use.
test unit "Time"
> 2 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 8 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...
</pre>
</li>
<li>Note the Summary results shown in the host output; all in use tests should pass.
</li>
<li>To exit TestApp, give the target window focus and enter Ctrl-C (or 'q' under Windows).
</ol>

== Samples (Optional, Not for Training Exercises) ==

The initial desktop installation of STRIDE does not set up any source code (with the exception of a set of system diagnostic tests) for automatic inclusion in a test application. The [[Desktop Installation | desktop framework]] distribution, however, comes with a set of [[Samples_Overview | Samples]].

To demonstrate how to build a sample, will will add the [[Test Intro Sample]] that provide an overview of STRIDE testing techniques. For an overview from a C++ perspective, please see [[Test Intro Cpp Sample]].

The following steps are applicable for '''all''' [[Samples_Overview | Samples]].

To begin, be sure that TestApp is not running, then copy the <tt>.c</tt> and <tt>.h</tt> files found in <tt>Samples/test_in_c_cpp/TestIntro</tt> to <tt>SDK/Windows/sample_src</tt> (or <tt>SDK/Posix/sample_src</tt> for Linux).

'''''Note:''' Only files in the sample_src directory will be picked up by the makefile. Files in any subdirectories will be igored.
''

Once the files have been copied to <tt>sample_src</tt>, simply build TestApp as described above. Note if you had previously copied other sample source to this directory, you should decide whether to remove those files first. When you complete the test app build, any source that is in this directory at the time of build will be included in the test app.

=== Running the Tests ===
Here we will run all tests in the <tt>TestApp.sidb</tt> database.<ref>Note that the S2 diagnostic tests are treated separately, and are not run unless the <tt>--diagnostics</tt> option is specified to <tt>stride</tt>.</ref>

# Run the build above TestApp in a console window.
# Invoke <tt>stride</tt> in a separate console window (different from the running TestApp) -- as shown below and verify Summary results.

Here are the command line parameters that we will submit to <tt>stride</tt>

<pre>
--database ./out/TestApp.sidb
--device TCP:localhost:8000
</pre>

A couple of things to note:
* If you setup an [[STRIDE_Runner#Environment_Variables | environment variable]] for the '''device''' option than it is not required in the option file. Note: Command line options override environment variables.
* You may want to create a text file named ''RunTestIntro.txt'' in the <tt>SDK\Windows</tt> (or <tt>SDK/Posix</tt> for Linux) directory as an option file to submit to <tt>stride</tt>.
* ''stride --help'' provides [[STRIDE_Runner#Options | options information]]

If you haven't done so already, start <tt>TestApp</tt> running in a separate console window.

Now run stride as follows (starting from the <tt>SDK\Windows</tt> or <tt>SDK/Posix</tt> directory):

<source lang="bash">
stride --options_file RunTestIntro.txt --run="*"
</source>

The output should look like this:
<pre>
Loading database...
Connecting to device...
Executing...
test unit "s2_testintro_flist"
> 2 passed, 1 failed, 0 in progress, 0 not in use.
test unit "s2_testintro_cclass"
> 1 passed, 1 failed, 0 in progress, 0 not in use.
test unit "s2_testintro_testdoubles"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "s2_testintro_testpoints"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "s2_testintro_parameters"
> 2 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 11 passed, 2 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...
</pre>

=== Interpreting Results ===
Open '''TestApp.xml''' in your browser; this file is created in the directory from which you ran '''stride''' (or the directory via the <tt>--output</tt> command line option). If you were connected to the Internet when you ran the tests, the TestApp.xsl file is also generated in the directory. By opening TestApp.xml in a web browser, the xsl is automatically applied to create html in the browser. If you use Google Chrome, please see [[Browser Compatibility]].

If you're interested in the details of the tests, please see to the test documentation contained in the test report.

==Notes==
<references/>

[[Category:Installation]]

Desktop Installation

2013-02-01T23:25:40Z

Jeffs: /* Perl Installation (Optional Not for Training Exercises) */

== Installation Packages ==
Files are installed by unzipping the provided package to your PC. Packages are available targeting the following operating systems (your version number may be different than that shown):
;Windows (x86)
:<tt>STRIDE_framework-windows_4.x.yy.zip</tt>
;Linux (x86)
:<tt>STRIDE_framework-linux_4.x.yy.tgz</tt>

Please see the appropriate installation instructions below.

== Windows Installation ==

=== Unpacking ===
The following installation example assumes the the installation package is located in your root directory and that the directory <tt>\stride</tt> exists. You can choose to install to a different location (all instructions below assume you are installing into <tt>\stride</tt>).

The example uses the open source [http://www.7-zip.org/ 7-Zip] utility to unzip the archive.

cd \stride
"\Program Files\7-Zip\7z" x ..\STRIDE_framework-windows_4.x.yy.zip

Once unzipped, files will have been installed under the <tt>\stride</tt> directory.

=== Verify Environment Variables ===

==== Updated PATH ====
As a final step, you will need to update your <tt>[http://en.wikipedia.org/wiki/Path_(variable) PATH]</tt> environment variable to include <tt>\stride\bin</tt>.
For instructions on modifying it, please see [http://support.microsoft.com/kb/310519 http://support.microsoft.com/kb/310519].

NOTE: ''Make sure to insert '''no spaces''' before and after the semicolon separators(;).''

==== Create/Update STRIDE_DIR====

Verify that the <tt>STRIDE_DIR</tt> environment variable exists and is set to the root installation directory (<tt>\stride</tt>). If this environment variable does not yet exist, you should create it as a user environment variable.

To confirm installation and display ''help'' run the following command in a console window:

stride -h

=== Uninstalling ===
To uninstall STRIDE simply:
* Remove any reference to <tt>\stride\bin</tt> in your <tt>PATH</tt> environment variable.
* Remove <tt>STRIDE_DIR</tt> environment variable.
* Remove <tt>\stride</tt> directory.

== Linux Installation ==

=== Unpacking ===
The following installation example assumes the the installation package is located in your home directory and that the directory <tt>~/stride</tt> exists. You can choose to install to a different location (all instructions below assume you are installing into <tt>~/stride</tt>).

cd ~/stride
tar -zxvf ../STRIDE_framework-linux_4.x.yy.tgz

Once unzipped, files will have been installed under the <tt>~/stride</tt> directory.

=== Verify Environment Variables ===

==== Updated PATH ====
As a final step, you will need to update your <tt>[http://en.wikipedia.org/wiki/Path_(variable) PATH]</tt> environment variable to include <tt>~/stride/bin</tt>.

If you use the bash shell, enter the following at a command prompt, or to automatically set at each login, add to your <tt>.bashrc</tt>:
export PATH=$PATH:~/stride/bin

For other shells, and more information, please see the following articles:
* [http://www.linuxheadquarters.com/howto/basic/path.shtml http://www.linuxheadquarters.com/howto/basic/path.shtml].
* [http://en.wikipedia.org/wiki/Environment_variable#UNIX http://en.wikipedia.org/wiki/Environment_variable]

==== Create/Update STRIDE_DIR====
Verify that the <tt>STRIDE_DIR</tt> environment variable exists and is set to the root installation directory (<tt>~/stride</tt>). If this environment variable does not yet exist, you should automatically set at each login, add to your <tt>.bashrc</tt>:
export STRIDE_DIR=~/stride

To confirm installation and display ''help'' run the following command in a console window:

stride -h

NOTE: ''In a 64-bit environment the above may fail with the following error: <code>"stride: /lib/ld-linux.so.2: bad ELF interpreter: No such file or directory."</code> To resolve this issue install the appropriate 32-bit compatibility libraries for your Linux distribution:''

* Debian / Ubuntu
sudo apt-get install ia32-libs
* Fedora / CentOS / RHEL
sudo yum -y install glibc.i686 libstdc++.i686

=== Uninstalling ===
To uninstall STRIDE simply:
* Remove any reference to <tt>~/stride/bin</tt> in your <tt>PATH</tt> environment variable.
* Remove <tt>STRIDE_DIR</tt> environment variable.
* Remove <tt>~/stride</tt> directory.

== Directories and Files ==

It's not necessary to understand the workings of the STRIDE framework to perform evaluation or training. The desktop package contains an [[STRIDE Off-Target Environment]] that utilizes a SDK that is set up with appropriate options and settings to enable "out of the box" functionality. A quick orientation to the Off-Target Environment's directories and files are shown below.

If you are interested in the details, consult the articles [[Posix SDK]] and [[Windows SDK]].

===<tt>bin</tt>===
This directory contains the [[Build Tools|STRIDE Build Tools]] and the [[STRIDE Runner]].

The build tools are invoked early on in the target software build process to generate special STRIDE artifacts that are used in subsequent build steps and later when running tests against the target. In an Off-Target Environment installation, these files are needed on the host computer since this is where we are building the target application. In a production environment, these files are needed only on the computer that performs the target software build.

The [[STRIDE Runner]] is the program you use to run tests from the host.

===<tt>lib</tt>===
This directory contains a set of STRIDE specific core scripting libraries along with prebuild binaries intended to be used for [[Test Modules Overview|testing in scripts]].

===<tt>Samples</tt>===
The Samples directory contains a number of sub-directories, each containing the source for a [[Samples|sample test]].

===<tt>SDK</tt>===
This directory contains the sub-directories <tt>GRS</tt>, <tt>Runtime</tt>, and <tt>SLAP</tt>, which contain source code that comprises the STRIDE Runtime. These sources are built into the STRIDE Runtime library as a dependency of the <tt>testapp</tt> target. (See <tt>src</tt> directory below.)

In addition, there is a directory named either <tt>Posix</tt> or <tt>Windows</tt>, depending on your host operating system.

'''<tt>[[Windows_SDK| SDK\Windows]](</tt> or <tt>[[Posix_SDK| SDK/Posix]]</tt> )'''

This directory (<tt>Windows</tt> or <tt>Posix</tt>) contains the following directories:

*<tt>bin</tt> ''(Windows only)''
: Contains GNU make program files (GNU make is already present on Linux systems)
*<tt>sample_src</tt>
: This directory is originally empty. The sandbox is set up so that any files in this directory are included in the TestApp build.
*<tt>settings</tt>
: This directory contains a set of <tt>stride.XXX.s2scompile</tt> files, where <tt>XXX</tt> coresponds to the target CPU architecture (i.e. X86, ARM...). These files, used by the [[s2scompile|STRIDE Compiler]], specify target CPU characteristics (endian-ness, data sizes and alignments). On Windows, this directory also contains a set of files for [[STRIDE_Extensions_for_Visual_Studio|use in building target apps with Visual Studio]].
*<tt>src</tt>
: This directory contains the Makefile used to produce the sandbox TestApp as well as the TestApp sources.
*''<tt>out</tt>''
: This directory (and several sub-directories) is created as part of the make process. All of the make targets are written to this directory and its sub-directories.

== Perl Installation (Optional, Not for Training Exercises) ==
If you intend to use [[Test Modules Overview|STRIDE Script modules]] for testing in script, you will need a recent version of Perl (x86 with threads support) installed. This is '''NOT''' required if only tests in C/C++ will be run or to complete the training.

As of this writing, we support only the 32-bit versions 5.8.9, 5.10.x, 5.12.x and 5.14.x of Perl.

=== Windows ===
It is required to use the standard 32-bit Perl distributions from [http://www.activestate.com/activeperl ActiveState].

The following additional (non-standard) Perl packages are also required for full functionality of STRIDE tests in perl:

* [http://search.cpan.org/~smueller/Class-ISA-0.36/lib/Class/ISA.pm Class::ISA]
* [http://search.cpan.org/~andrewf/Pod-POM-0.27/lib/Pod/POM.pm Pod::POM]
* [http://search.cpan.org/~andk/Devel-Symdump-2.08/lib/Devel/Symdump.pm Devel::Symdump]

You can easily install these packages using the [http://docs.activestate.com/activeperl/5.10/faq/ActivePerl-faq2.html ppm tool]. If you access the Internet via a proxy make sure to read [http://docs.activestate.com/activeperl/5.10/faq/ActivePerl-faq2.html#ppm_and_proxies this]. Simple command-line installation of PACKAGE_NAME (the package to install) typically just requires typing:

ppm install PACKAGE_NAME

=== Linux ===
We recommend you to use the standard 32-bit Perl distribution that comes with your Linux version. In case you need to manually build from source make sure to configure "shared library" (<tt>-Duseshrplib</tt>), "thread support" (<tt>-Duseithreads</tt>) and no "64-bit support" (<tt>-Uuse64bitint -Uuse64bitall</tt>).

The following additional (non-standard) Perl packages are also required for full functionality of STRIDE tests in perl:

* [http://search.cpan.org/~ingy/YAML-LibYAML-0.38/lib/YAML/XS.pm YAML::XS]
* [http://search.cpan.org/~smueller/Class-ISA-0.36/lib/Class/ISA.pm Class::ISA]
* [http://search.cpan.org/~andrewf/Pod-POM-0.27/lib/Pod/POM.pm Pod::POM]
* [http://search.cpan.org/~andk/Devel-Symdump-2.08/lib/Devel/Symdump.pm Devel::Symdump]

If your perl is installed in a system directory (<tt>/usr/bin/perl</tt>, for instance), you will need root access to install shared modules. The simplest method for installing packages is via the [http://www.perl.com/doc/manual/html/lib/CPAN.html CPAN shell]. If you access the Internet via a proxy make sure to set the appropriate [http://search.cpan.org/dist/CPAN/lib/CPAN.pm#Config_Variables CPAN config variables]. To start the shell in interactive mode:

sudo perl -MCPAN -eshell

Once in the shell, search for and install the latest stable version of PACKAGE_NAME (the package to install):

install PACKAGE_NAME

The STRIDE perl packages also need to load your system's '''libperl.so''' (shared object file) at runtime. Depending on your system, this file should be loadable from a perl CORE directory or from one of the shared system directories. If you '''DO NOT''' have this shared library on your system, you might need to install a ''libperl-dev'', ''perl-devel'' or ''perl-libs'' package in order to get it. Here is how you can do that on the console of some Linux distributions:

* Debian / Ubuntu
sudo apt-get install libperl-dev
* Fedora / CentOS / RHEL
sudo yum -y install perl-devel

=== Validation ===
Once you have installed Perl we recommend you to run the following command in a console window:

<source lang="bash">
stride --device NULL --diagnostics Perl --output PerlCheck
</source>

If everything was properly set up you should get the following output:

<pre>
Loading database...
Executing diagnostics...
script "diagnostics.pl"
> 2 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 2 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...
</pre>

In addition a report file with name <tt>PerlCheck.xml</tt> will be created in the current directory. If interested in the details you could open that report file in a browser of your choice.

[[Category:Installation]]

Desktop Installation

2013-02-01T23:25:03Z

Jeffs: /* Perl Installation (Optional) */

== Installation Packages ==
Files are installed by unzipping the provided package to your PC. Packages are available targeting the following operating systems (your version number may be different than that shown):
;Windows (x86)
:<tt>STRIDE_framework-windows_4.x.yy.zip</tt>
;Linux (x86)
:<tt>STRIDE_framework-linux_4.x.yy.tgz</tt>

Please see the appropriate installation instructions below.

== Windows Installation ==

=== Unpacking ===
The following installation example assumes the the installation package is located in your root directory and that the directory <tt>\stride</tt> exists. You can choose to install to a different location (all instructions below assume you are installing into <tt>\stride</tt>).

The example uses the open source [http://www.7-zip.org/ 7-Zip] utility to unzip the archive.

cd \stride
"\Program Files\7-Zip\7z" x ..\STRIDE_framework-windows_4.x.yy.zip

Once unzipped, files will have been installed under the <tt>\stride</tt> directory.

=== Verify Environment Variables ===

==== Updated PATH ====
As a final step, you will need to update your <tt>[http://en.wikipedia.org/wiki/Path_(variable) PATH]</tt> environment variable to include <tt>\stride\bin</tt>.
For instructions on modifying it, please see [http://support.microsoft.com/kb/310519 http://support.microsoft.com/kb/310519].

NOTE: ''Make sure to insert '''no spaces''' before and after the semicolon separators(;).''

==== Create/Update STRIDE_DIR====

Verify that the <tt>STRIDE_DIR</tt> environment variable exists and is set to the root installation directory (<tt>\stride</tt>). If this environment variable does not yet exist, you should create it as a user environment variable.

To confirm installation and display ''help'' run the following command in a console window:

stride -h

=== Uninstalling ===
To uninstall STRIDE simply:
* Remove any reference to <tt>\stride\bin</tt> in your <tt>PATH</tt> environment variable.
* Remove <tt>STRIDE_DIR</tt> environment variable.
* Remove <tt>\stride</tt> directory.

== Linux Installation ==

=== Unpacking ===
The following installation example assumes the the installation package is located in your home directory and that the directory <tt>~/stride</tt> exists. You can choose to install to a different location (all instructions below assume you are installing into <tt>~/stride</tt>).

cd ~/stride
tar -zxvf ../STRIDE_framework-linux_4.x.yy.tgz

Once unzipped, files will have been installed under the <tt>~/stride</tt> directory.

=== Verify Environment Variables ===

==== Updated PATH ====
As a final step, you will need to update your <tt>[http://en.wikipedia.org/wiki/Path_(variable) PATH]</tt> environment variable to include <tt>~/stride/bin</tt>.

If you use the bash shell, enter the following at a command prompt, or to automatically set at each login, add to your <tt>.bashrc</tt>:
export PATH=$PATH:~/stride/bin

For other shells, and more information, please see the following articles:
* [http://www.linuxheadquarters.com/howto/basic/path.shtml http://www.linuxheadquarters.com/howto/basic/path.shtml].
* [http://en.wikipedia.org/wiki/Environment_variable#UNIX http://en.wikipedia.org/wiki/Environment_variable]

==== Create/Update STRIDE_DIR====
Verify that the <tt>STRIDE_DIR</tt> environment variable exists and is set to the root installation directory (<tt>~/stride</tt>). If this environment variable does not yet exist, you should automatically set at each login, add to your <tt>.bashrc</tt>:
export STRIDE_DIR=~/stride

To confirm installation and display ''help'' run the following command in a console window:

stride -h

NOTE: ''In a 64-bit environment the above may fail with the following error: <code>"stride: /lib/ld-linux.so.2: bad ELF interpreter: No such file or directory."</code> To resolve this issue install the appropriate 32-bit compatibility libraries for your Linux distribution:''

* Debian / Ubuntu
sudo apt-get install ia32-libs
* Fedora / CentOS / RHEL
sudo yum -y install glibc.i686 libstdc++.i686

=== Uninstalling ===
To uninstall STRIDE simply:
* Remove any reference to <tt>~/stride/bin</tt> in your <tt>PATH</tt> environment variable.
* Remove <tt>STRIDE_DIR</tt> environment variable.
* Remove <tt>~/stride</tt> directory.

== Directories and Files ==

It's not necessary to understand the workings of the STRIDE framework to perform evaluation or training. The desktop package contains an [[STRIDE Off-Target Environment]] that utilizes a SDK that is set up with appropriate options and settings to enable "out of the box" functionality. A quick orientation to the Off-Target Environment's directories and files are shown below.

If you are interested in the details, consult the articles [[Posix SDK]] and [[Windows SDK]].

===<tt>bin</tt>===
This directory contains the [[Build Tools|STRIDE Build Tools]] and the [[STRIDE Runner]].

The build tools are invoked early on in the target software build process to generate special STRIDE artifacts that are used in subsequent build steps and later when running tests against the target. In an Off-Target Environment installation, these files are needed on the host computer since this is where we are building the target application. In a production environment, these files are needed only on the computer that performs the target software build.

The [[STRIDE Runner]] is the program you use to run tests from the host.

===<tt>lib</tt>===
This directory contains a set of STRIDE specific core scripting libraries along with prebuild binaries intended to be used for [[Test Modules Overview|testing in scripts]].

===<tt>Samples</tt>===
The Samples directory contains a number of sub-directories, each containing the source for a [[Samples|sample test]].

===<tt>SDK</tt>===
This directory contains the sub-directories <tt>GRS</tt>, <tt>Runtime</tt>, and <tt>SLAP</tt>, which contain source code that comprises the STRIDE Runtime. These sources are built into the STRIDE Runtime library as a dependency of the <tt>testapp</tt> target. (See <tt>src</tt> directory below.)

In addition, there is a directory named either <tt>Posix</tt> or <tt>Windows</tt>, depending on your host operating system.

'''<tt>[[Windows_SDK| SDK\Windows]](</tt> or <tt>[[Posix_SDK| SDK/Posix]]</tt> )'''

This directory (<tt>Windows</tt> or <tt>Posix</tt>) contains the following directories:

*<tt>bin</tt> ''(Windows only)''
: Contains GNU make program files (GNU make is already present on Linux systems)
*<tt>sample_src</tt>
: This directory is originally empty. The sandbox is set up so that any files in this directory are included in the TestApp build.
*<tt>settings</tt>
: This directory contains a set of <tt>stride.XXX.s2scompile</tt> files, where <tt>XXX</tt> coresponds to the target CPU architecture (i.e. X86, ARM...). These files, used by the [[s2scompile|STRIDE Compiler]], specify target CPU characteristics (endian-ness, data sizes and alignments). On Windows, this directory also contains a set of files for [[STRIDE_Extensions_for_Visual_Studio|use in building target apps with Visual Studio]].
*<tt>src</tt>
: This directory contains the Makefile used to produce the sandbox TestApp as well as the TestApp sources.
*''<tt>out</tt>''
: This directory (and several sub-directories) is created as part of the make process. All of the make targets are written to this directory and its sub-directories.

== Perl Installation (Optional Not for Training Exercises) ==
If you intend to use [[Test Modules Overview|STRIDE Script modules]] for testing in script, you will need a recent version of Perl (x86 with threads support) installed. This is '''NOT''' required if only tests in C/C++ will be run or to complete the training.

As of this writing, we support only the 32-bit versions 5.8.9, 5.10.x, 5.12.x and 5.14.x of Perl.

=== Windows ===
It is required to use the standard 32-bit Perl distributions from [http://www.activestate.com/activeperl ActiveState].

The following additional (non-standard) Perl packages are also required for full functionality of STRIDE tests in perl:

* [http://search.cpan.org/~smueller/Class-ISA-0.36/lib/Class/ISA.pm Class::ISA]
* [http://search.cpan.org/~andrewf/Pod-POM-0.27/lib/Pod/POM.pm Pod::POM]
* [http://search.cpan.org/~andk/Devel-Symdump-2.08/lib/Devel/Symdump.pm Devel::Symdump]

You can easily install these packages using the [http://docs.activestate.com/activeperl/5.10/faq/ActivePerl-faq2.html ppm tool]. If you access the Internet via a proxy make sure to read [http://docs.activestate.com/activeperl/5.10/faq/ActivePerl-faq2.html#ppm_and_proxies this]. Simple command-line installation of PACKAGE_NAME (the package to install) typically just requires typing:

ppm install PACKAGE_NAME

=== Linux ===
We recommend you to use the standard 32-bit Perl distribution that comes with your Linux version. In case you need to manually build from source make sure to configure "shared library" (<tt>-Duseshrplib</tt>), "thread support" (<tt>-Duseithreads</tt>) and no "64-bit support" (<tt>-Uuse64bitint -Uuse64bitall</tt>).

The following additional (non-standard) Perl packages are also required for full functionality of STRIDE tests in perl:

* [http://search.cpan.org/~ingy/YAML-LibYAML-0.38/lib/YAML/XS.pm YAML::XS]
* [http://search.cpan.org/~smueller/Class-ISA-0.36/lib/Class/ISA.pm Class::ISA]
* [http://search.cpan.org/~andrewf/Pod-POM-0.27/lib/Pod/POM.pm Pod::POM]
* [http://search.cpan.org/~andk/Devel-Symdump-2.08/lib/Devel/Symdump.pm Devel::Symdump]

If your perl is installed in a system directory (<tt>/usr/bin/perl</tt>, for instance), you will need root access to install shared modules. The simplest method for installing packages is via the [http://www.perl.com/doc/manual/html/lib/CPAN.html CPAN shell]. If you access the Internet via a proxy make sure to set the appropriate [http://search.cpan.org/dist/CPAN/lib/CPAN.pm#Config_Variables CPAN config variables]. To start the shell in interactive mode:

sudo perl -MCPAN -eshell

Once in the shell, search for and install the latest stable version of PACKAGE_NAME (the package to install):

install PACKAGE_NAME

The STRIDE perl packages also need to load your system's '''libperl.so''' (shared object file) at runtime. Depending on your system, this file should be loadable from a perl CORE directory or from one of the shared system directories. If you '''DO NOT''' have this shared library on your system, you might need to install a ''libperl-dev'', ''perl-devel'' or ''perl-libs'' package in order to get it. Here is how you can do that on the console of some Linux distributions:

* Debian / Ubuntu
sudo apt-get install libperl-dev
* Fedora / CentOS / RHEL
sudo yum -y install perl-devel

=== Validation ===
Once you have installed Perl we recommend you to run the following command in a console window:

<source lang="bash">
stride --device NULL --diagnostics Perl --output PerlCheck
</source>

If everything was properly set up you should get the following output:

<pre>
Loading database...
Executing diagnostics...
script "diagnostics.pl"
> 2 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 2 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...
</pre>

In addition a report file with name <tt>PerlCheck.xml</tt> will be created in the current directory. If interested in the details you could open that report file in a browser of your choice.

[[Category:Installation]]

Building an Off-Target Test App

2013-02-01T23:24:12Z

Jeffs: /* Samples */

== Prerequisite ==
This article guides you through building a test application ('''TestApp''') for the purpose of running sample code using the [[STRIDE Off-Target Environment]].

It requires an installation of the STRIDE Framework desktop package. If not installed, please see [[Desktop Installation]] for more information.

It also requires that your desktop contains one of the following '''compilers''':
* For Windows Microsoft Visual Studio 2008 or later is required. If you don't already have Visual Studio, the free [http://en.wikipedia.org/wiki/Microsoft_Visual_Studio_Express Visual C++ Express] can be used (download [http://www.microsoft.com/express/download/#webInstall here]).
* For Linux the [http://en.wikipedia.org/wiki/GNU_Compiler_Collection GNU Compiler Collection] (included by default in almost all Linux distros) is required.

== Building a TestApp ==

=== SDK Makefile ===
The SDK Makefile is set up so that all <tt>.c</tt> <tt>.cpp</tt> and <tt>.h</tt> files found in the directory <tt>SDK\Windows\sample_src</tt> (or <tt>SDK/Posix/sample_src</tt> for Linux) are included in the compile and link of the '''testapp''' target.

Further--as a pre-compilation step--any <tt>.h</tt> files found in <tt>sample_src</tt> are submitted to the [[STRIDE Build Tools]]. This will result in
* the detection of [[Test_Unit_Pragmas| test pragmas]] used to declare Test Units in these <tt>.h</tt> files
* the detection of [[Scl_function | function pragmas]] used to declare remoting of functions also found in <tt>.h</tt> files
* the inclusion of metadata into the <tt>sidb</tt> file created
* the generation of an [[Intercept Module]] required for executing tests

=== Build Steps ===
To begin, be sure that TestApp is not running then perform the following steps:

NOTE: ''If you experience any build problem please make sure to read [[Troubleshooting Build Problems]] for possible resolution.''

====Linux====
<ol>
<li>Build the test app
<source lang="bash">
cd $STRIDE_DIR/SDK/Posix/src
make testapp
</source>
</li>
<li>Note that the following artifacts are produced by the build:

;<tt>$STRIDE_DIR/SDK/Posix/out/bin/TestApp</tt>
: the test application
;<tt>$STRIDE_DIR/SDK/Posix/out/TestApp.sidb</tt>
: the STRIDE interface database file which contains metadata describing the interfaces remoted by the test app (along with other data)
</li>
</ol>

====Windows====
<ol>
<li>If using Microsoft Visual Studio, open a [http://msdn.microsoft.com/en-us/library/ms235639(v=vs.100).aspx Visual Studio Command Prompt] to ensure that the compiler and linker are on your PATH.
</li>
<li>Build the test app using the supplied GNU make. (You will get Makefile errors if you use the default make.)
<source lang="dos">
cd %STRIDE_DIR%\SDK\Windows\src
..\bin\make testapp
</source>
<li>Note that the following artifacts are produced by the build:

;<tt>%STRIDE_DIR%\SDK\Windows\out\bin\TestApp.exe</tt>
: the test application
;<tt>%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb</tt>
: the STRIDE interface database file which contains metadata describing the interfaces remoted by the test app (along with other data)
</li>
</ol>

NOTE: ''If you prefer to use Visual Studio to build/debug/run your testapp, we provide instructions [[STRIDE_Extensions_for_Visual_Studio|here]] about how to accomplish this.''

== Diagnostics ==
The test app we just built does not have any user tests in it. At this point it provides a starting point for test that we will subsequently add.

However, a set of diagnostic tests that verify operation of the STRIDE runtime itself are always built into the generated TestApp executable. If desired (we recommend you to do so) you could run them by doing the following:

<ol>
<li>Invoke the TestApp. In order to see TestApp's output, we recommend that you manually open a new console (or Windows equivalent):
;Linux
<source lang="bash">
$STRIDE_DIR/SDK/Posix/out/bin/TestApp
</source>
;Windows
<source lang="dos">
%STRIDE_DIR%\SDK\Windows\out\bin\TestApp
</source>
(...or launch from the file explorer)

<li> Note TestApp's output upon startup.
<pre>
--------------------------------------------------
STRIDE Test Console Application.
Enter 'Ctrl+C' to Quit.
--------------------------------------------------
Listening on TCP port 8000
starting up...
"_srThread" thread started.
"stride" thread started.
</pre>
</li>
<li>From a second console window, invoke <tt>[[STRIDE_Runner|stride]]</tt> as follows, to verify connectivity with the test app and STRIDE runtime operation:
 
;Linux
<source lang="bash">
stride --diagnostics --database="$STRIDE_DIR/SDK/Posix/out/TestApp.sidb" --device=TCP:localhost:8000 --run="*"
</source>
;Windows
<source lang="dos">
stride --diagnostics --database="%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb" --device=TCP:localhost:8000 --run="*"
</source>

As the tests run you will see output in both the TestApp (target) and stride (host) console windows.

The host console window output is shown here:
<pre>
Loading database...
Connecting to device...
runtime version: 4.3.0x
Executing diagnostics...
test unit "Link"
Loopback ............
Payload Fragmentation
Stub-Proxy Deadlock
Target Characteristics
> 4 passed, 0 failed, 0 in progress, 0 not in use.
test unit "Stat"
> 2 passed, 0 failed, 0 in progress, 0 not in use.
test unit "Time"
> 2 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 8 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...
</pre>
</li>
<li>Note the Summary results shown in the host output; all in use tests should pass.
</li>
<li>To exit TestApp, give the target window focus and enter Ctrl-C (or 'q' under Windows).
</ol>

== Samples (Optional Not for Training Exercises) ==

The initial desktop installation of STRIDE does not set up any source code (with the exception of a set of system diagnostic tests) for automatic inclusion in a test application. The [[Desktop Installation | desktop framework]] distribution, however, comes with a set of [[Samples_Overview | Samples]].

To demonstrate how to build a sample, will will add the [[Test Intro Sample]] that provide an overview of STRIDE testing techniques. For an overview from a C++ perspective, please see [[Test Intro Cpp Sample]].

The following steps are applicable for '''all''' [[Samples_Overview | Samples]].

To begin, be sure that TestApp is not running, then copy the <tt>.c</tt> and <tt>.h</tt> files found in <tt>Samples/test_in_c_cpp/TestIntro</tt> to <tt>SDK/Windows/sample_src</tt> (or <tt>SDK/Posix/sample_src</tt> for Linux).

'''''Note:''' Only files in the sample_src directory will be picked up by the makefile. Files in any subdirectories will be igored.
''

Once the files have been copied to <tt>sample_src</tt>, simply build TestApp as described above. Note if you had previously copied other sample source to this directory, you should decide whether to remove those files first. When you complete the test app build, any source that is in this directory at the time of build will be included in the test app.

=== Running the Tests ===
Here we will run all tests in the <tt>TestApp.sidb</tt> database.<ref>Note that the S2 diagnostic tests are treated separately, and are not run unless the <tt>--diagnostics</tt> option is specified to <tt>stride</tt>.</ref>

# Run the build above TestApp in a console window.
# Invoke <tt>stride</tt> in a separate console window (different from the running TestApp) -- as shown below and verify Summary results.

Here are the command line parameters that we will submit to <tt>stride</tt>

<pre>
--database ./out/TestApp.sidb
--device TCP:localhost:8000
</pre>

A couple of things to note:
* If you setup an [[STRIDE_Runner#Environment_Variables | environment variable]] for the '''device''' option than it is not required in the option file. Note: Command line options override environment variables.
* You may want to create a text file named ''RunTestIntro.txt'' in the <tt>SDK\Windows</tt> (or <tt>SDK/Posix</tt> for Linux) directory as an option file to submit to <tt>stride</tt>.
* ''stride --help'' provides [[STRIDE_Runner#Options | options information]]

If you haven't done so already, start <tt>TestApp</tt> running in a separate console window.

Now run stride as follows (starting from the <tt>SDK\Windows</tt> or <tt>SDK/Posix</tt> directory):

<source lang="bash">
stride --options_file RunTestIntro.txt --run="*"
</source>

The output should look like this:
<pre>
Loading database...
Connecting to device...
Executing...
test unit "s2_testintro_flist"
> 2 passed, 1 failed, 0 in progress, 0 not in use.
test unit "s2_testintro_cclass"
> 1 passed, 1 failed, 0 in progress, 0 not in use.
test unit "s2_testintro_testdoubles"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "s2_testintro_testpoints"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "s2_testintro_parameters"
> 2 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 11 passed, 2 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...
</pre>

=== Interpreting Results ===
Open '''TestApp.xml''' in your browser; this file is created in the directory from which you ran '''stride''' (or the directory via the <tt>--output</tt> command line option). If you were connected to the Internet when you ran the tests, the TestApp.xsl file is also generated in the directory. By opening TestApp.xml in a web browser, the xsl is automatically applied to create html in the browser. If you use Google Chrome, please see [[Browser Compatibility]].

If you're interested in the details of the tests, please see to the test documentation contained in the test report.

==Notes==
<references/>

[[Category:Installation]]

Desktop Installation

2013-02-01T23:18:53Z

Jeffs: /* Perl Installation (Optional) */

== Installation Packages ==
Files are installed by unzipping the provided package to your PC. Packages are available targeting the following operating systems (your version number may be different than that shown):
;Windows (x86)
:<tt>STRIDE_framework-windows_4.x.yy.zip</tt>
;Linux (x86)
:<tt>STRIDE_framework-linux_4.x.yy.tgz</tt>

Please see the appropriate installation instructions below.

== Windows Installation ==

=== Unpacking ===
The following installation example assumes the the installation package is located in your root directory and that the directory <tt>\stride</tt> exists. You can choose to install to a different location (all instructions below assume you are installing into <tt>\stride</tt>).

The example uses the open source [http://www.7-zip.org/ 7-Zip] utility to unzip the archive.

cd \stride
"\Program Files\7-Zip\7z" x ..\STRIDE_framework-windows_4.x.yy.zip

Once unzipped, files will have been installed under the <tt>\stride</tt> directory.

=== Verify Environment Variables ===

==== Updated PATH ====
As a final step, you will need to update your <tt>[http://en.wikipedia.org/wiki/Path_(variable) PATH]</tt> environment variable to include <tt>\stride\bin</tt>.
For instructions on modifying it, please see [http://support.microsoft.com/kb/310519 http://support.microsoft.com/kb/310519].

NOTE: ''Make sure to insert '''no spaces''' before and after the semicolon separators(;).''

==== Create/Update STRIDE_DIR====

Verify that the <tt>STRIDE_DIR</tt> environment variable exists and is set to the root installation directory (<tt>\stride</tt>). If this environment variable does not yet exist, you should create it as a user environment variable.

To confirm installation and display ''help'' run the following command in a console window:

stride -h

=== Uninstalling ===
To uninstall STRIDE simply:
* Remove any reference to <tt>\stride\bin</tt> in your <tt>PATH</tt> environment variable.
* Remove <tt>STRIDE_DIR</tt> environment variable.
* Remove <tt>\stride</tt> directory.

== Linux Installation ==

=== Unpacking ===
The following installation example assumes the the installation package is located in your home directory and that the directory <tt>~/stride</tt> exists. You can choose to install to a different location (all instructions below assume you are installing into <tt>~/stride</tt>).

cd ~/stride
tar -zxvf ../STRIDE_framework-linux_4.x.yy.tgz

Once unzipped, files will have been installed under the <tt>~/stride</tt> directory.

=== Verify Environment Variables ===

==== Updated PATH ====
As a final step, you will need to update your <tt>[http://en.wikipedia.org/wiki/Path_(variable) PATH]</tt> environment variable to include <tt>~/stride/bin</tt>.

If you use the bash shell, enter the following at a command prompt, or to automatically set at each login, add to your <tt>.bashrc</tt>:
export PATH=$PATH:~/stride/bin

For other shells, and more information, please see the following articles:
* [http://www.linuxheadquarters.com/howto/basic/path.shtml http://www.linuxheadquarters.com/howto/basic/path.shtml].
* [http://en.wikipedia.org/wiki/Environment_variable#UNIX http://en.wikipedia.org/wiki/Environment_variable]

==== Create/Update STRIDE_DIR====
Verify that the <tt>STRIDE_DIR</tt> environment variable exists and is set to the root installation directory (<tt>~/stride</tt>). If this environment variable does not yet exist, you should automatically set at each login, add to your <tt>.bashrc</tt>:
export STRIDE_DIR=~/stride

To confirm installation and display ''help'' run the following command in a console window:

stride -h

NOTE: ''In a 64-bit environment the above may fail with the following error: <code>"stride: /lib/ld-linux.so.2: bad ELF interpreter: No such file or directory."</code> To resolve this issue install the appropriate 32-bit compatibility libraries for your Linux distribution:''

* Debian / Ubuntu
sudo apt-get install ia32-libs
* Fedora / CentOS / RHEL
sudo yum -y install glibc.i686 libstdc++.i686

=== Uninstalling ===
To uninstall STRIDE simply:
* Remove any reference to <tt>~/stride/bin</tt> in your <tt>PATH</tt> environment variable.
* Remove <tt>STRIDE_DIR</tt> environment variable.
* Remove <tt>~/stride</tt> directory.

== Directories and Files ==

It's not necessary to understand the workings of the STRIDE framework to perform evaluation or training. The desktop package contains an [[STRIDE Off-Target Environment]] that utilizes a SDK that is set up with appropriate options and settings to enable "out of the box" functionality. A quick orientation to the Off-Target Environment's directories and files are shown below.

If you are interested in the details, consult the articles [[Posix SDK]] and [[Windows SDK]].

===<tt>bin</tt>===
This directory contains the [[Build Tools|STRIDE Build Tools]] and the [[STRIDE Runner]].

The build tools are invoked early on in the target software build process to generate special STRIDE artifacts that are used in subsequent build steps and later when running tests against the target. In an Off-Target Environment installation, these files are needed on the host computer since this is where we are building the target application. In a production environment, these files are needed only on the computer that performs the target software build.

The [[STRIDE Runner]] is the program you use to run tests from the host.

===<tt>lib</tt>===
This directory contains a set of STRIDE specific core scripting libraries along with prebuild binaries intended to be used for [[Test Modules Overview|testing in scripts]].

===<tt>Samples</tt>===
The Samples directory contains a number of sub-directories, each containing the source for a [[Samples|sample test]].

===<tt>SDK</tt>===
This directory contains the sub-directories <tt>GRS</tt>, <tt>Runtime</tt>, and <tt>SLAP</tt>, which contain source code that comprises the STRIDE Runtime. These sources are built into the STRIDE Runtime library as a dependency of the <tt>testapp</tt> target. (See <tt>src</tt> directory below.)

In addition, there is a directory named either <tt>Posix</tt> or <tt>Windows</tt>, depending on your host operating system.

'''<tt>[[Windows_SDK| SDK\Windows]](</tt> or <tt>[[Posix_SDK| SDK/Posix]]</tt> )'''

This directory (<tt>Windows</tt> or <tt>Posix</tt>) contains the following directories:

*<tt>bin</tt> ''(Windows only)''
: Contains GNU make program files (GNU make is already present on Linux systems)
*<tt>sample_src</tt>
: This directory is originally empty. The sandbox is set up so that any files in this directory are included in the TestApp build.
*<tt>settings</tt>
: This directory contains a set of <tt>stride.XXX.s2scompile</tt> files, where <tt>XXX</tt> coresponds to the target CPU architecture (i.e. X86, ARM...). These files, used by the [[s2scompile|STRIDE Compiler]], specify target CPU characteristics (endian-ness, data sizes and alignments). On Windows, this directory also contains a set of files for [[STRIDE_Extensions_for_Visual_Studio|use in building target apps with Visual Studio]].
*<tt>src</tt>
: This directory contains the Makefile used to produce the sandbox TestApp as well as the TestApp sources.
*''<tt>out</tt>''
: This directory (and several sub-directories) is created as part of the make process. All of the make targets are written to this directory and its sub-directories.

== Perl Installation (Optional) ==
If you intend to use [[Test Modules Overview|STRIDE Script modules]] for testing in script, you will need a recent version of Perl (x86 with threads support) installed. This is '''NOT''' required if only tests in C/C++ will be run or to complete the training.

As of this writing, we support only the 32-bit versions 5.8.9, 5.10.x, 5.12.x and 5.14.x of Perl.

=== Windows ===
It is required to use the standard 32-bit Perl distributions from [http://www.activestate.com/activeperl ActiveState].

The following additional (non-standard) Perl packages are also required for full functionality of STRIDE tests in perl:

* [http://search.cpan.org/~smueller/Class-ISA-0.36/lib/Class/ISA.pm Class::ISA]
* [http://search.cpan.org/~andrewf/Pod-POM-0.27/lib/Pod/POM.pm Pod::POM]
* [http://search.cpan.org/~andk/Devel-Symdump-2.08/lib/Devel/Symdump.pm Devel::Symdump]

You can easily install these packages using the [http://docs.activestate.com/activeperl/5.10/faq/ActivePerl-faq2.html ppm tool]. If you access the Internet via a proxy make sure to read [http://docs.activestate.com/activeperl/5.10/faq/ActivePerl-faq2.html#ppm_and_proxies this]. Simple command-line installation of PACKAGE_NAME (the package to install) typically just requires typing:

ppm install PACKAGE_NAME

=== Linux ===
We recommend you to use the standard 32-bit Perl distribution that comes with your Linux version. In case you need to manually build from source make sure to configure "shared library" (<tt>-Duseshrplib</tt>), "thread support" (<tt>-Duseithreads</tt>) and no "64-bit support" (<tt>-Uuse64bitint -Uuse64bitall</tt>).

The following additional (non-standard) Perl packages are also required for full functionality of STRIDE tests in perl:

* [http://search.cpan.org/~ingy/YAML-LibYAML-0.38/lib/YAML/XS.pm YAML::XS]
* [http://search.cpan.org/~smueller/Class-ISA-0.36/lib/Class/ISA.pm Class::ISA]
* [http://search.cpan.org/~andrewf/Pod-POM-0.27/lib/Pod/POM.pm Pod::POM]
* [http://search.cpan.org/~andk/Devel-Symdump-2.08/lib/Devel/Symdump.pm Devel::Symdump]

If your perl is installed in a system directory (<tt>/usr/bin/perl</tt>, for instance), you will need root access to install shared modules. The simplest method for installing packages is via the [http://www.perl.com/doc/manual/html/lib/CPAN.html CPAN shell]. If you access the Internet via a proxy make sure to set the appropriate [http://search.cpan.org/dist/CPAN/lib/CPAN.pm#Config_Variables CPAN config variables]. To start the shell in interactive mode:

sudo perl -MCPAN -eshell

Once in the shell, search for and install the latest stable version of PACKAGE_NAME (the package to install):

install PACKAGE_NAME

The STRIDE perl packages also need to load your system's '''libperl.so''' (shared object file) at runtime. Depending on your system, this file should be loadable from a perl CORE directory or from one of the shared system directories. If you '''DO NOT''' have this shared library on your system, you might need to install a ''libperl-dev'', ''perl-devel'' or ''perl-libs'' package in order to get it. Here is how you can do that on the console of some Linux distributions:

* Debian / Ubuntu
sudo apt-get install libperl-dev
* Fedora / CentOS / RHEL
sudo yum -y install perl-devel

=== Validation ===
Once you have installed Perl we recommend you to run the following command in a console window:

<source lang="bash">
stride --device NULL --diagnostics Perl --output PerlCheck
</source>

If everything was properly set up you should get the following output:

<pre>
Loading database...
Executing diagnostics...
script "diagnostics.pl"
> 2 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 2 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...
</pre>

In addition a report file with name <tt>PerlCheck.xml</tt> will be created in the current directory. If interested in the details you could open that report file in a browser of your choice.

[[Category:Installation]]

Training Test Macros

2013-02-01T19:04:48Z

Jeffs: /* Run and Publish Results */

== Objectives ==

This Training Module focuses on the basics of writing and executing tests. It covers the following topics:
* The C++ class [[Test_Units | Test Unit packaging]] option
* How to leverage [[Test_Macros | Test Macros]]
* Using [[Test_Macros#Notes | Notes]] and [[Test_Log | Test Logs]], what is the difference?
* Creating [[Test_Documentation_in_C/C%2B%2B | Test Documentation]] via the build process
* Executing Tests using the [[STRIDE_Runner | Runner]]

The test unit is implemented in two source files: '''TestBasic.cpp''' and '''TestBasic.h'''. The comments and descriptions are contained in '''TestBasic.h'''.

The Test Unit has test cases already implemented (used for reference) and has a test method that you are required to implement (called '''Exercise'''). Currently this method is empty and returns a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using the SDK makefile
* Invoke TestApp found in the ''/out/bin'' directory
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute the ''Test Basic'' Test Units

> stride --options_file myoptions.txt --run TestBasic
Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
--------------------------------------------------------------
Summary: 3 passed, 1 failed, 0 in progress, 1 not in use.

Disconnecting from device...
Saving result file...

You can also review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the sample_src directory (based on the output option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercises ===

Now edit the training source code to complete the following exercises:

;Exercise()
* '''Assignment 1:''' Add an ''srNOTE'' to <tt>Exercise()</tt> that will add a simple message to the test report (e.g. "Exercise ...")
** ''Hint:'' You will probably want to read about [[Test Macros]]
** ''Hint:'' srTestCaseSetStatus(srTEST_CASE_DEFAULT, srTEST_NOTINUSE, 0) is a placeholder to be removed when you add your test code.
* '''Assignment 2:''' Within <tt>Exercise()</tt> validate that <tt>sut_mult(1,1)</tt> does NOT equal <tt>sut_add(1,1)</tt>
** ''Hint:'' You will use a [[Test Macros | Test Macro]].

=== Check Results ===

* Before you rebuild <tt>TestApp.exe</tt>, you will need to shut it down by entering 'q' in its console window or closing the window directly.

* After rebuilding, invoke <tt>TestApp.exe</tt> once again.

* Execute only ''TestBasic'' using the stride runner:

> stride --options_file myoptions.txt --run TestBasic

Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 4 passed, 1 failed, 0 in progress, 0 not in use.
--------------------------------------------------------------
Summary: 4 passed, 1 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

* If you have TestApp.xml already open in your browser, you can simply refresh (F5), to view the latest test results.

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space(--space TestBasic). If you have not added test space options to your options file (<tt>myoptions.txt</tt>) please see [[Training_Getting_Started#Test_Space_Access| testspace access]].

> stride --options_file myoptions.txt --run TestBasic --space TestBasic --upload

Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 4 passed, 1 failed, 0 in progress, 0 not in use.
------------------------------------------------------------
Summary: 4 passed, 1 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...
Uploading to test space...

=== Validate Uploaded Results ===

Navigate to Test Space using your browser, then validate your results against the pre-configured baseline. For details, please see [[Training_Getting_Started#Confirming_Training_Exercise_Results | Confirming Training Exercise Results]]

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

You can view the results on test space by pointing your browser at '''<tt><nowiki>https://yourcompany.stridetestspace.com</nowiki></tt>'''. Log in using the credentials you supplied earlier in the options file.

== Reference ==
The following reference information is related to Test Unit basics.

=== Wiki ===

* [[Test_Units_Overview | Test Units Overview]] - Provides a general overview Test Units (i.e. writing tests in C and C++)
* [[Test_Units |Test Unit Packaging]] - Discusses the three types of packaging that can be used for Test Units (we prefer Test Classes even for c programmers)
* [[Test_Macros|Test Macros]] - Optional macros that provide shortcuts for testing assertions and automatic report annotation (you will want to use these).
* [[Test_Macros#Note_Macros|Notes]] - Used to add logging information to your '''test logic''' (automatically added to test reports)
* [[Test_Log|Test Logs]] - Used to add logging information to your '''source code''' (added to test reports if enabled)
* [[Tracing|Tracing]] - The Runner allows tracing on logs and test points.

=== Samples ===

* [[Test_Class_Sample|Test Class Sample]] - Specifically the [[Test_Class_Sample#Basic|Basic Simple]] Test Unit
* [[Test_Macros_Sample|Test Macro Sample]] - This sample covers simple uses of each of the Test Macros.

Note - Other Test Unit Samples related to packaging that are useful are the following:

* [[Test_CClass_Sample|Test C Class Sample]]
* [[Test_Function_List_Sample|Test Function List Sample]]

[[Category: Training]]

Training Expectations

2013-02-01T19:04:11Z

Jeffs: /* Run and Publish Results */

== Objectives ==

This Training Module is focused on ''Test Points'' and how to validate them using ''Expectations''. The module covers the following topics:
* Presentation of a [[Expectations | validation]] technique based on ''code sequencing'' and ''state data''
* Overview of [[Source_Instrumentation_Overview#Instrumentation | source instrumentation]]
* Review of [[Test_Point_Testing_in_C/C%2B%2B | expectation tables and predicates]]
* Example use cases such as concurrent validation, using trigger conditions, etc.

There are two test files used -- '''TestExpect.cpp & TestExpect.h''' -- that implement three Test Units:
* '''TestExpect_Seq'''
* '''TestExpect_Data'''
* '''TestExpect_Misc'''

The first two Test Units have two test methods already implemented and have one method each that you are required to implement called '''Exercise'''. The third Test Unit has four test methods already implemented and also has one method to implement as an exercise. Currently the ''exercise methods'' return a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Expectations'' Test Units only

> stride --options_file myoptions.txt --run TestExpect_Seq --run TestExpect_Data --run TestExpect_Misc

Loading database...
Connecting to device...
Executing...
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
-----------------------------------------------------------
Summary: 7 passed, 1 failed, 0 in progress, 3 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestExpect_Seq::Exercise'''
** Validate '''ALL''' upper case Test Points (A - I)
** Use ''Unordered'' and ''Nonstrict'' sequencing
** Use ''sut_DoSequencing(SEQ_1)'' to generate part of the sequence
** Use ''sut_start_thread(SEQ_3)'' to generate the rest of the sequence
** Add a ''NOTE_INFO(..)'' to log that the test is executing

* '''TestExpect_Data::Exercise'''
** Validate the following Test Points '''{D, G, F, H}'''
** Check that '''F''' occurs 2 times
** Use ''Unordered'' sequencing
** Write a new custom predicate that validates data for both '''D''' and '''H''' Test Points
*** Add ''NOTE_INFO(..)'' to capture content of the Test Points
*** Add extra check for '''D''' that the status is '''GOOD'''
*** Confirm that data fields '''d1''' and '''d2''' are as expected
*** Pass the expected data fields (for both Test Points) as part of the '''user''' data within the setup
*** Use ''sut_start_thread(SEQ_3)'' to generate the sequence
** Write another custom predicate for validating data for '''G'''
*** Add ''NOTE_INFO(..)'' to capture content of the Test Point
*** Validate the expected string using '''user''' data
** Add a ''NOTE_INFO(..)'' to log that the test is executing

* '''TestExpect_Misc::Exercise'''
** Validate 2 sequences using a Trigger in between
*** Sequence 1 = '''D''' '''E''' and '''A'''
*** Trigger = '''C'''
*** Sequence 2 = '''F''' and '''F''' (2 occurrences)
** Use ''ANY AT ALL'' special member with trigger
** Use ''Ordered'' and ''Strict'' sequencing
** Use ''sut_start_thread(SEQ_2)'' to generate the expected sequences

* Execute ''Test Expectations'' Test Units only

> stride --options_file myoptions.txt --run TestExpect_Seq --run TestExpect_Data --run TestExpect_Misc

Loading database...
Connecting to device...
Executing...
test unit "TestExpect_Seq"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestExpect_Data"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestExpect_Misc"
> 4 passed, 1 failed, 0 in progress, 0 not in use.
-----------------------------------------------------------
Summary: 10 passed, 1 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. If you have not added test space options to your options file (<tt>myoptions.txt</tt>) please see [[Training_Getting_Started#Test_Space_Access| testspace access]].

> stride --options_file myoptions.txt --run TestExpect_Seq --run TestExpect_Data --run TestExpect_Misc --space TestExpect --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===

* [[Source_Instrumentation_Overview | Instrumentation Overview]] providing high-level concepts of this validation technique
* [[Test_Point |Test Point]] Macro definition
* [[Expectations |Expectations]] definition and how to set your ''Expectations''

=== Samples ===

* [[Test_Point_Sample | Test Point Sample]] - Demonstrates simple technique to monitor and test activity occurring in another thread

[[Category: Training]]

Training Runtime API

2013-02-01T19:03:34Z

Jeffs: /* Run and Publish Results */

== Objectives ==

This Training Module is focused leveraging the [[Runtime_Test_Services | Runtime APIs]] in the context of writing a test. The module covers the following topics:
* How to set [[Runtime_Test_Services#srTestCaseSetStatus | test status]]
* Setting [[Runtime_Test_Services#srTestCaseSetName | test case name]] and [[Runtime_Test_Services#srTestCaseSetDescription | description]] directly via the API
* Setting [[Runtime_Test_Services#srTestSuiteSetName | test suite name]] and [[Runtime_Test_Services#srTestSuiteSetDescription | description]] directly via the API
* Dynamically creating [[Runtime_Test_Services#srTestSuiteAddSuite | test suites]] and [[Runtime_Test_Services#srTestSuiteAddCase | test cases]]

There are two new files used -- '''TestRuntime.cpp & TestRuntime.h''' --- that implement two test Units:
* '''TestRuntime_Static'''
* '''TestRuntime_Dynamic'''

All of the Test Units have test cases already implemented (used for reference) and have one test method that you are required to implement: one is called '''Exercise''' and the other is called '''dynamic_Exercise'''. Currently the exercise methods return a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Runtime APIs'' Test Units only

> stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic

Loading database...
Connecting to device...
Executing...
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
------------------------------------------------------------
Summary: 6 passed, 0 failed, 0 in progress, 2 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestRuntime_Static::Exercise'''
** Validate the ''sut_mult()'' routine using some simple data input
*** ''Hint:'' You will probably want to read about [[Runtime Test Services]]
** Set the ''test method'' name to '''Mult'''
** Use direct Runtime APIs to:
*** Set the test case description
*** Capture logging information via comments
*** Set the status of the test case

* '''TestRuntime_Dynamic->Exercise'''
** Pass in '''5''' via command line for the number of test cases to generate
** Check using '''srEXIT_XX''' that the number of test cases has been passed in correctly
** Add a new Test Suite using '''Exercise''' for the name
** Write a loop generating a dynamic test case using ''NumberOfTestCases''
*** Test Case name shall be '''Test_n''' where "n" is the loop count as each case must have a unique name.
*** Add a description for each test case
*** Validate ''sut_foo()'' using the loop count

* note- The ''dynamic'' Test Unit is using ''CClass'' packaging

* Execute ''Test Runtime API'' Test Units only (NOTE - requires passing parameter)

> stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic(5)

Loading database...
Connecting to device...
Executing...
test unit "TestRuntime_Static"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestRuntime_Dynamic"
> 10 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 13 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. If you have not added test space options to your options file (<tt>myoptions.txt</tt>) please see [[Training_Getting_Started#Test_Space_Access| testspace access]].

> stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic --space TestRuntime --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===

* [[Runtime_Test_Services#C_Test_Functions | Runtime Test Services]] with special attention to the following:
** Review srTestCaseSetDescription()
** Review srTestCaseSetStatus()
** Review srTestCaseSetName()
** Review srTestCaseAddComment()
** Review srTestSuiteAddCase()
** Review srTestSuiteAddSuite()
** Review srTestSuiteSetDescription()
** Review srTestSuiteSetName()

=== Samples ===
* [[Test_Class_Sample | Test Class Sample]] - Specifically the [[Test_Class_Sample#Runtime_Services |Runtime Services]] Test Units
* [[Test_CClass_Sample | Test C Class Sample]] - Specifically the [[Test_CClass_Sample#Runtime_Services | Runtime Services]] Test Units
* [[Test_Function_List_Sample | Test Function List Sample]] - Specifically the [[Test_Function_List_Sample#Runtime_Services | Runtime Services]] Test Units

[[Category: Training]]

Training Runtime API

2013-02-01T19:03:21Z

Jeffs: /* Run and Publish Results */

Training Doubling

2013-02-01T19:02:52Z

Jeffs: /* Run and Publish Results */

== Objectives ==

This Training Module is focused on explaining how to leverage '''Test Doubles''' in the context of executing a test. For an overview of '''intercepting''' existing functions please refer to [[Using_Test_Doubles | Test Doubles]]. The module covers following topics:
* How to apply pragmas for [[Function_Capturing | function intercepting]]
* How to decide what kind of ''mangling'' is required
** ''Definition'' verses ''Reference''
** ''Explicit'' verses ''Implicit''
* Setting and Resetting the Double implementation

There are two new files used -- '''TestDouble.cpp & TestDouble.h''' --- that implement two test Units:
* '''TestDouble_Reference'''
* '''TestDouble_Definition'''

All of the Test Units have test cases already implemented (used for reference) and have one test method that you are required to implement called '''Exercise'''. Currently the exercise methods return a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Double'' Test Units only

> stride --options_file myoptions.txt --run TestDouble_Reference --run TestDouble_Definition

Loading database...
Connecting to device...
Executing...
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------
Summary: 4 passed, 0 failed, 0 in progress, 2 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestDouble_Reference::Exercise'''
** Implement a new ''test double'' for the ''strlen()'' routine:
*** Add a ''NOTE'' capturing its name when it is called
*** Uses the real ''strlen()'' to return the length of the passed in string
** Use ''srEXPECT_EQ()'' to validate that ''sut_strcheck()'' returns correct length
** Make sure to restore the original routine

* '''TestDouble_Definition::Exercise'''
** Implement a new ''test double'' for the ''sut_strcpy()'' routine:
*** Log its name when it is called
*** Validate string passed to ''sut_strsave()'' is received correctly by the ''test double''
*** Remember that ''sut_strsave()'' calls ''sut_strcpy()'' with the string passed to it
*** Can use a test macro to validate the string is correctly passed to the double
*** Call the original function (''sut_strcpy()'') with a '''DIFFERENT STRING''' (i.e "Exercise Test String 2")
**** Call the original within the test double
**** Make sure to reset the original function within the mock before calling it (i.e. srDOUBLE_RESET)
** Call ''sut_strsave()'' with a string (i.e. "Exercise Test String 1")
** Use ''sut_strget()'' to retrieve a string
** Compare retrieved string with the '''DIFFERENT STRING''' inserted by the ''test double''
** NOTE - won't work if you wait until the end of the test to restore the original routine (i.e. srDOUBLE_SET)

* Execute ''Test Double'' Test Units only

> stride --options_file myoptions.txt --run TestDouble_Reference TestDouble_Definition

Loading database...
Connecting to device...
Executing...
test unit "TestDouble_Reference"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestDouble_Definition"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------
Summary: 6 passed, 0 failed, 0 in progress, 0 not in use.

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. If you have not added test space options to your options file (<tt>myoptions.txt</tt>) please see [[Training_Getting_Started#Test_Space_Access| testspace access]].

> stride --options_file myoptions.txt --run TestDouble_Reference TestDouble_Definition --space TestDouble --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===
* [[Using_Test_Doubles | Using Test Doubles]] - Outlines the basic steps to enable a ''double''
** Capture the f(x) that is going to be doubled
** Code the test logic to additionally set the Double and restore original f(x)

* [[Scl_function | Intercepting a function]] - Pragma specifics outline
** For this exercise, there are '''NO''' optional parameters when capturing a f(x) to be '''intercepted'''

* Details of the pragma parameters can be understood in the [[Intercept_Module#Interceptor | Intercept Module]] article. Please review in detail the following:
** ''context'' - This refers to incepting a callee or the called routine.
** ''name-mangling'' - How the function name is switched during the compiling process
** ''group-id'' - Used to associate a group of intercepted functions

* Some '''external links''' to review:
** [http://www.martinfowler.com/bliki/TestDouble.html Martin Fowler definition]
** [http://en.wikipedia.org/wiki/Test_Double Wikipedia definition]
** [http://msdn.microsoft.com/en-us/magazine/cc163358.aspx MSDN Magazine description]

=== Samples ===

* [[Test_Double_Sample | Test Double Sample]] that can be a useful reference. This reference example can be built and executed like all of samples using the Off-Target environment.

[[Category: Training]]

Training File IO

2013-02-01T19:02:35Z

Jeffs: /* Run and Publish Results */

== Objectives ==

This Training Module is focused on leveraging [[File_Transfer_Services | File IO]] from the context of a Test Unit executing on a target. The module covers the following:
* Accessing host based files
* Read and Writing content to / from host files

There are two test files used -- '''TestFile.cpp & TestFile.h''' -- that implement one Test Unit:
* '''TestFile'''

The Test Unit has test cases already implemented (used for reference) and has one test method that you are required to implement (called '''Exercise'''). Currently this method is empty and returns a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test File'' Test Unit only

> stride --options_file myoptions.txt --run TestFile

Loading database...
Connecting to device...
Executing...
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
--------------------------------------------------------------
Summary: 0 passed, 0 failed, 0 in progress, 1 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* Read data content ''TestFileInput.dat'' file created from the previous test method (SDK\Windows or SDK/Posix).
** ''Hint:'' You will probably want to read about [[File Transfer Services]]
* Make sure to Exit the test if an error occurs when opening file(s) (i.e. use '''srEXIT_XX''')
* Take a summation of the data content using ''sut_add()''
* Validate summation with content in ''TestFileSum.dat'' file
* Add a ''NOTE'' capturing important information such as ''bytes read'', ''summation'', etc. _

* Execute only ''TestFile''

> stride --options_file myoptions.txt --run TestFile

Loading database...
Connecting to device...
Executing...
test unit "TestFile"
> 1 passed, 0 failed, 0 in progress, 0 not in use.
--------------------------------------------------------------
Summary: 1 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. If you have not added test space options to your options file (<tt>myoptions.txt</tt>) please see [[Training_Getting_Started#Test_Space_Access| testspace access]].

> stride --options_file myoptions.txt --run TestFile --space TestFile --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===
* [[File_Transfer_Services | File Transfer Services]] - important information for using file operations on host file from test code executing on the target
** Review ''srFileOpen()'' and ''srFileClose()''
** Review ''srFileRead()'' and ''srFileWrite()''
** Other references as needed

=== Samples ===
* [[File_Services_Sample | File Services Sample]] - presents a few basic examples of how to use the File Transfer Services API to interact with the host file system.

[[Category: Training]]

Training Doubling

2013-02-01T19:02:00Z

Jeffs: /* Run and Publish Results */

== Objectives ==

This Training Module is focused on explaining how to leverage '''Test Doubles''' in the context of executing a test. For an overview of '''intercepting''' existing functions please refer to [[Using_Test_Doubles | Test Doubles]]. The module covers following topics:
* How to apply pragmas for [[Function_Capturing | function intercepting]]
* How to decide what kind of ''mangling'' is required
** ''Definition'' verses ''Reference''
** ''Explicit'' verses ''Implicit''
* Setting and Resetting the Double implementation

There are two new files used -- '''TestDouble.cpp & TestDouble.h''' --- that implement two test Units:
* '''TestDouble_Reference'''
* '''TestDouble_Definition'''

All of the Test Units have test cases already implemented (used for reference) and have one test method that you are required to implement called '''Exercise'''. Currently the exercise methods return a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Double'' Test Units only

> stride --options_file myoptions.txt --run TestDouble_Reference --run TestDouble_Definition

Loading database...
Connecting to device...
Executing...
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------
Summary: 4 passed, 0 failed, 0 in progress, 2 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestDouble_Reference::Exercise'''
** Implement a new ''test double'' for the ''strlen()'' routine:
*** Add a ''NOTE'' capturing its name when it is called
*** Uses the real ''strlen()'' to return the length of the passed in string
** Use ''srEXPECT_EQ()'' to validate that ''sut_strcheck()'' returns correct length
** Make sure to restore the original routine

* '''TestDouble_Definition::Exercise'''
** Implement a new ''test double'' for the ''sut_strcpy()'' routine:
*** Log its name when it is called
*** Validate string passed to ''sut_strsave()'' is received correctly by the ''test double''
*** Remember that ''sut_strsave()'' calls ''sut_strcpy()'' with the string passed to it
*** Can use a test macro to validate the string is correctly passed to the double
*** Call the original function (''sut_strcpy()'') with a '''DIFFERENT STRING''' (i.e "Exercise Test String 2")
**** Call the original within the test double
**** Make sure to reset the original function within the mock before calling it (i.e. srDOUBLE_RESET)
** Call ''sut_strsave()'' with a string (i.e. "Exercise Test String 1")
** Use ''sut_strget()'' to retrieve a string
** Compare retrieved string with the '''DIFFERENT STRING''' inserted by the ''test double''
** NOTE - won't work if you wait until the end of the test to restore the original routine (i.e. srDOUBLE_SET)

* Execute ''Test Double'' Test Units only

> stride --options_file myoptions.txt --run TestDouble_Reference TestDouble_Definition

Loading database...
Connecting to device...
Executing...
test unit "TestDouble_Reference"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestDouble_Definition"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------
Summary: 6 passed, 0 failed, 0 in progress, 0 not in use.

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. If you have not added test space options to your options file (<tt>myoptions.txt</tt>) please see [[Training_Getting_Started#Test_Space_Access| testspace access]].

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

> stride --options_file myoptions.txt --run TestDouble_Reference TestDouble_Definition --space TestDouble --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===
* [[Using_Test_Doubles | Using Test Doubles]] - Outlines the basic steps to enable a ''double''
** Capture the f(x) that is going to be doubled
** Code the test logic to additionally set the Double and restore original f(x)

* [[Scl_function | Intercepting a function]] - Pragma specifics outline
** For this exercise, there are '''NO''' optional parameters when capturing a f(x) to be '''intercepted'''

* Details of the pragma parameters can be understood in the [[Intercept_Module#Interceptor | Intercept Module]] article. Please review in detail the following:
** ''context'' - This refers to incepting a callee or the called routine.
** ''name-mangling'' - How the function name is switched during the compiling process
** ''group-id'' - Used to associate a group of intercepted functions

* Some '''external links''' to review:
** [http://www.martinfowler.com/bliki/TestDouble.html Martin Fowler definition]
** [http://en.wikipedia.org/wiki/Test_Double Wikipedia definition]
** [http://msdn.microsoft.com/en-us/magazine/cc163358.aspx MSDN Magazine description]

=== Samples ===

* [[Test_Double_Sample | Test Double Sample]] that can be a useful reference. This reference example can be built and executed like all of samples using the Off-Target environment.

[[Category: Training]]

Training Runtime API

2013-02-01T18:46:38Z

Jeffs: /* Implement Exercise */

== Objectives ==

This Training Module is focused leveraging the [[Runtime_Test_Services | Runtime APIs]] in the context of writing a test. The module covers the following topics:
* How to set [[Runtime_Test_Services#srTestCaseSetStatus | test status]]
* Setting [[Runtime_Test_Services#srTestCaseSetName | test case name]] and [[Runtime_Test_Services#srTestCaseSetDescription | description]] directly via the API
* Setting [[Runtime_Test_Services#srTestSuiteSetName | test suite name]] and [[Runtime_Test_Services#srTestSuiteSetDescription | description]] directly via the API
* Dynamically creating [[Runtime_Test_Services#srTestSuiteAddSuite | test suites]] and [[Runtime_Test_Services#srTestSuiteAddCase | test cases]]

There are two new files used -- '''TestRuntime.cpp & TestRuntime.h''' --- that implement two test Units:
* '''TestRuntime_Static'''
* '''TestRuntime_Dynamic'''

All of the Test Units have test cases already implemented (used for reference) and have one test method that you are required to implement: one is called '''Exercise''' and the other is called '''dynamic_Exercise'''. Currently the exercise methods return a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Runtime APIs'' Test Units only

> stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic

Loading database...
Connecting to device...
Executing...
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
------------------------------------------------------------
Summary: 6 passed, 0 failed, 0 in progress, 2 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestRuntime_Static::Exercise'''
** Validate the ''sut_mult()'' routine using some simple data input
*** ''Hint:'' You will probably want to read about [[Runtime Test Services]]
** Set the ''test method'' name to '''Mult'''
** Use direct Runtime APIs to:
*** Set the test case description
*** Capture logging information via comments
*** Set the status of the test case

* '''TestRuntime_Dynamic->Exercise'''
** Pass in '''5''' via command line for the number of test cases to generate
** Check using '''srEXIT_XX''' that the number of test cases has been passed in correctly
** Add a new Test Suite using '''Exercise''' for the name
** Write a loop generating a dynamic test case using ''NumberOfTestCases''
*** Test Case name shall be '''Test_n''' where "n" is the loop count as each case must have a unique name.
*** Add a description for each test case
*** Validate ''sut_foo()'' using the loop count

* note- The ''dynamic'' Test Unit is using ''CClass'' packaging

* Execute ''Test Runtime API'' Test Units only (NOTE - requires passing parameter)

> stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic(5)

Loading database...
Connecting to device...
Executing...
test unit "TestRuntime_Static"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestRuntime_Dynamic"
> 10 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 13 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following if not already done:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

> stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic --space TestRuntime --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===

* [[Runtime_Test_Services#C_Test_Functions | Runtime Test Services]] with special attention to the following:
** Review srTestCaseSetDescription()
** Review srTestCaseSetStatus()
** Review srTestCaseSetName()
** Review srTestCaseAddComment()
** Review srTestSuiteAddCase()
** Review srTestSuiteAddSuite()
** Review srTestSuiteSetDescription()
** Review srTestSuiteSetName()

=== Samples ===
* [[Test_Class_Sample | Test Class Sample]] - Specifically the [[Test_Class_Sample#Runtime_Services |Runtime Services]] Test Units
* [[Test_CClass_Sample | Test C Class Sample]] - Specifically the [[Test_CClass_Sample#Runtime_Services | Runtime Services]] Test Units
* [[Test_Function_List_Sample | Test Function List Sample]] - Specifically the [[Test_Function_List_Sample#Runtime_Services | Runtime Services]] Test Units

[[Category: Training]]

Training File IO

2013-02-01T17:30:37Z

Jeffs: /* Implement Exercise */

== Objectives ==

This Training Module is focused on leveraging [[File_Transfer_Services | File IO]] from the context of a Test Unit executing on a target. The module covers the following:
* Accessing host based files
* Read and Writing content to / from host files

There are two test files used -- '''TestFile.cpp & TestFile.h''' -- that implement one Test Unit:
* '''TestFile'''

The Test Unit has test cases already implemented (used for reference) and has one test method that you are required to implement (called '''Exercise'''). Currently this method is empty and returns a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test File'' Test Unit only

> stride --options_file myoptions.txt --run TestFile

Loading database...
Connecting to device...
Executing...
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
--------------------------------------------------------------
Summary: 0 passed, 0 failed, 0 in progress, 1 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* Read data content ''TestFileInput.dat'' file created from the previous test method (SDK\Windows or SDK/Posix).
** ''Hint:'' You will probably want to read about [[File Transfer Services]]
* Make sure to Exit the test if an error occurs when opening file(s) (i.e. use '''srEXIT_XX''')
* Take a summation of the data content using ''sut_add()''
* Validate summation with content in ''TestFileSum.dat'' file
* Add a ''NOTE'' capturing important information such as ''bytes read'', ''summation'', etc. _

* Execute only ''TestFile''

> stride --options_file myoptions.txt --run TestFile

Loading database...
Connecting to device...
Executing...
test unit "TestFile"
> 1 passed, 0 failed, 0 in progress, 0 not in use.
--------------------------------------------------------------
Summary: 1 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following if not already done:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

> stride --options_file myoptions.txt --run TestFile --space TestFile --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===
* [[File_Transfer_Services | File Transfer Services]] - important information for using file operations on host file from test code executing on the target
** Review ''srFileOpen()'' and ''srFileClose()''
** Review ''srFileRead()'' and ''srFileWrite()''
** Other references as needed

=== Samples ===
* [[File_Services_Sample | File Services Sample]] - presents a few basic examples of how to use the File Transfer Services API to interact with the host file system.

[[Category: Training]]

Training Fixturing

2013-01-31T21:25:32Z

Jeffs: /* Run and Publish Results */

== Objectives ==

This Training Module focuses on '''Fixturing''' in the context of the STRIDE Test System. For a high-level overview refer to the following wiki section on [[What_is_Unique_About_STRIDE#Fixturing | fixturing]]. This module covers the following topics:
* Startup logic at the beginning of a Test Unit
* [[Test_Unit_Pragmas#Fixturing_Pragmas | Setup]] logic for each test method
* [[Test_Unit_Pragmas#Fixturing_Pragmas | Teardown]] logic for each test method

There are two test files used -- '''TestFixture.cpp & TestFixture.h'''. These implement one Test Unit:
* '''TestFixture'''

The Test Unit has one test case already implemented (used for reference) and has '''two test methods''' that you are required to implement (called '''Exercise1''' and '''Exercise2'''). Currently the ''Exercise1'' method is empty and returns a ''NOT IN USE'' status. The ''Exercise2'' method does not yet exist.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Fixture'' Test Unit only

> stride -options_file myoptions.txt --run TestFixture

Loading database...
Connecting to device...
Executing...
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
--------------------------------------------------------------
Summary: 1 passed, 0 failed, 0 in progress, 1 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestFixture::Exercise1'''
** Add a ''NOTE'' that will display the ''name'' of the test.
** Validate that '''Sequence2''' is stored when starting the Thread by checking that "Sequence2" equals MyString[0].
*** ''HINT'': The method to focus on in SUT is DoSequencing()
*** ''HINT'': m_sequence is incremented as each method goes through Teardown.

* '''TestFixture::Exercise2'''
** Add new test method declaration called ''Exercise2'' to TestFixture.h
** Add new test method implementation of ''Exercise2'' to TestFixture.cpp
** Add a ''NOTE'' that will display the name of the test
** Validate that '''Sequence3''' is persisted when starting the Thread

* Execute only ''TestFixture''

> stride --options_file myoptions.txt --run TestFixture

Loading database...
Connecting to device...
Executing...
test unit "TestFixture"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
--------------------------------------------------------------
Summary: 3 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space (--space TestFixture). If you have not added test space options to your options file (<tt>myoptions.txt</tt>) please see [[Training_Getting_Started#Test_Space_Access| testspace access]].

> stride --options_file myoptions.txt --run TestFixture --space TestFixture --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===

* [[Test_Unit_Pragmas#Fixturing_Pragmas | Fixturing Pragmas]] -- important information for setting up and tearing down ''things'' between each executed test method within a Test Unit
** Review the types of Test Units these pragmas are applicable for

=== Samples ===

* [[Test_Class_Sample | Test Class Sample]] - Specifically the [[Test_Class_Sample#Basic::Fixtures | Basic Fixtures]] Test Unit
* [[Test_CClass_Sample | Test C Class Sample]] - Specifically the [[Test_CClass_Sample#basic_fixtures | Basic Fixtures]] Test Unit
* [[Test_Function_List_Sample | Test Function List Sample]]- Specifically the [[Test_Function_List_Sample#basic_fixtures |Basic Fixtures]] Test Unit

[[Category: Training]]

Training Parameters

2013-01-31T21:24:47Z

Jeffs: /* Run and Publish Results */

== Objectives ==

This Training Module is focused on '''Parameter passing''' in the context of the Test Unit. The module covers the following:
* [[Parameterized_Test_Units | Passing parameters]] via the Runner

There are two test files used -- '''TestParam.cpp & TestParam.h'''. These implement one Test Unit:
* '''TestParam'''

The Test Unit has test cases already implemented (used for reference) and has one test method that you are required to implement (called '''Exercise'''). Currently this method is empty and returns a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Param'' Test Unit only
: Note that parameters are passed to the test unit from the host command line.

> stride --options_file myoptions.txt --run "TestParam(9,4,\"STRING1\")"

Loading database...
Connecting to device...
Executing...
test unit "TestParam"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
--------------------------------------------------------------
Summary: 1 passed, 0 failed, 0 in progress, 1 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

;Exercise()
* Add a ''NOTE'' that will display the ''integer parameters'' passed via command line
* Add an Assert macro guarding against no parameters being passed (i.e. default value of 0 received)
* Call ''sut_add()'' with parameters and validate that it returns the expected value
* Call ''sut_mult()'' with parameters and validate that it returns the expected value
** ''Hint:'' You will probably want to read about [[Parameterized Test Units]]

* Execute only ''TestParam'' '''without''' parameters

> stride --options_file myoptions.txt --run TestParam

Loading database...
Connecting to device...
Executing...
test unit "TestParam"
> 0 passed, 2 failed, 0 in progress, 0 not in use.
--------------------------------------------------------------
Summary: 0 passed, 2 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

* Execute only ''TestParam'' '''with''' parameters
> stride -O myoptions.txt --run "TestParam(9,4,\"STRING1\")"

Loading database...
Connecting to device...
Executing...
test unit "TestParam"
> 2 passed, 0 failed, 0 in progress, 0 not in use.
--------------------------------------------------------------
Summary: 2 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space (--space TestParam). If you have not added test space options to your options file (<tt>myoptions.txt</tt>) please see [[Training_Getting_Started#Test_Space_Access| testspace access]].

> stride --options_file myoptions.txt --run "TestParam(9,4,\"STRING1\")" --space TestParam --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===

* [[Parameterized_Test_Units |Parameterized Test Units]] - Describes how to supply parameters to your Test Units
* [[STRIDE_Runner | STRIDE Runner]] - Information on command line options
** Refer to "Test Unit Specification Examples" - specifically examples using parameters
** Review <tt>run[--r] arg</tt> option

=== Samples ===
* [[Test_Class_Sample | Test Class Sample]] - Specifically the [[Test_Class_Sample#Basic::Parameterized | Parameterized]] Test Unit
* [[Test_CClass_Sample | Test C Class Sample]] - Specifically the [[Test_CClass_Sample#basic_parameterized | Parameterized]] Test Unit

[[Category: Training]]

Training Test Macros

2013-01-31T21:19:56Z

Jeffs:

== Objectives ==

This Training Module focuses on the basics of writing and executing tests. It covers the following topics:
* The C++ class [[Test_Units | Test Unit packaging]] option
* How to leverage [[Test_Macros | Test Macros]]
* Using [[Test_Macros#Notes | Notes]] and [[Test_Log | Test Logs]], what is the difference?
* Creating [[Test_Documentation_in_C/C%2B%2B | Test Documentation]] via the build process
* Executing Tests using the [[STRIDE_Runner | Runner]]

The test unit is implemented in two source files: '''TestBasic.cpp''' and '''TestBasic.h'''. The comments and descriptions are contained in '''TestBasic.h'''.

The Test Unit has test cases already implemented (used for reference) and has a test method that you are required to implement (called '''Exercise'''). Currently this method is empty and returns a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using the SDK makefile
* Invoke TestApp found in the ''/out/bin'' directory
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute the ''Test Basic'' Test Units

> stride --options_file myoptions.txt --run TestBasic
Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
--------------------------------------------------------------
Summary: 3 passed, 1 failed, 0 in progress, 1 not in use.

Disconnecting from device...
Saving result file...

You can also review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the sample_src directory (based on the output option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercises ===

Now edit the training source code to complete the following exercises:

;Exercise()
* '''Assignment 1:''' Add an ''srNOTE'' to <tt>Exercise()</tt> that will add a simple message to the test report (e.g. "Exercise ...")
** ''Hint:'' You will probably want to read about [[Test Macros]]
** ''Hint:'' srTestCaseSetStatus(srTEST_CASE_DEFAULT, srTEST_NOTINUSE, 0) is a placeholder to be removed when you add your test code.
* '''Assignment 2:''' Within <tt>Exercise()</tt> validate that <tt>sut_mult(1,1)</tt> does NOT equal <tt>sut_add(1,1)</tt>
** ''Hint:'' You will use a [[Test Macros | Test Macro]].

=== Check Results ===

* Before you rebuild <tt>TestApp.exe</tt>, you will need to shut it down by entering 'q' in its console window or closing the window directly.

* After rebuilding, invoke <tt>TestApp.exe</tt> once again.

* Execute only ''TestBasic'' using the stride runner:

> stride --options_file myoptions.txt --run TestBasic

Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 4 passed, 1 failed, 0 in progress, 0 not in use.
--------------------------------------------------------------
Summary: 4 passed, 1 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

* If you have TestApp.xml already open in your browser, you can simply refresh (F5), to view the latest test results.

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space(--space TestBasic). If you have not added test space options to your options file (<tt>myoptions.txt</tt>) please see [[Training_Getting_Started#Test_Space_Access| testspace access]]

> stride --options_file myoptions.txt --run TestBasic --space TestBasic --upload

Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 4 passed, 1 failed, 0 in progress, 0 not in use.
------------------------------------------------------------
Summary: 4 passed, 1 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...
Uploading to test space...

=== Validate Uploaded Results ===

Navigate to Test Space using your browser, then validate your results against the pre-configured baseline. For details, please see [[Training_Getting_Started#Confirming_Training_Exercise_Results | Confirming Training Exercise Results]]

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

You can view the results on test space by pointing your browser at '''<tt><nowiki>https://yourcompany.stridetestspace.com</nowiki></tt>'''. Log in using the credentials you supplied earlier in the options file.

== Reference ==
The following reference information is related to Test Unit basics.

=== Wiki ===

* [[Test_Units_Overview | Test Units Overview]] - Provides a general overview Test Units (i.e. writing tests in C and C++)
* [[Test_Units |Test Unit Packaging]] - Discusses the three types of packaging that can be used for Test Units (we prefer Test Classes even for c programmers)
* [[Test_Macros|Test Macros]] - Optional macros that provide shortcuts for testing assertions and automatic report annotation (you will want to use these).
* [[Test_Macros#Note_Macros|Notes]] - Used to add logging information to your '''test logic''' (automatically added to test reports)
* [[Test_Log|Test Logs]] - Used to add logging information to your '''source code''' (added to test reports if enabled)
* [[Tracing|Tracing]] - The Runner allows tracing on logs and test points.

=== Samples ===

* [[Test_Class_Sample|Test Class Sample]] - Specifically the [[Test_Class_Sample#Basic|Basic Simple]] Test Unit
* [[Test_Macros_Sample|Test Macro Sample]] - This sample covers simple uses of each of the Test Macros.

Note - Other Test Unit Samples related to packaging that are useful are the following:

* [[Test_CClass_Sample|Test C Class Sample]]
* [[Test_Function_List_Sample|Test Function List Sample]]

[[Category: Training]]

Training Test Macros

2013-01-31T21:04:56Z

Jeffs: /* Implement Exercises */

== Objectives ==

This Training Module focuses on the basics of writing and executing tests. It covers the following topics:
* The C++ class [[Test_Units | Test Unit packaging]] option
* How to leverage [[Test_Macros | Test Macros]]
* Using [[Test_Macros#Notes | Notes]] and [[Test_Log | Test Logs]], what is the difference?
* Creating [[Test_Documentation_in_C/C%2B%2B | Test Documentation]] via the build process
* Executing Tests using the [[STRIDE_Runner | Runner]]

The test unit is implemented in two source files: '''TestBasic.cpp''' and '''TestBasic.h'''. The comments and descriptions are contained in '''TestBasic.h'''.

The Test Unit has test cases already implemented (used for reference) and has a test method that you are required to implement (called '''Exercise'''). Currently this method is empty and returns a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using the SDK makefile
* Invoke TestApp found in the ''/out/bin'' directory
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute the ''Test Basic'' Test Units

> stride --options_file myoptions.txt --run TestBasic
Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
--------------------------------------------------------------
Summary: 3 passed, 1 failed, 0 in progress, 1 not in use.

Disconnecting from device...
Saving result file...

You can also review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the sample_src directory (based on the output option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercises ===

Now edit the training source code to complete the following exercises:

;Exercise()
* '''Assignment 1:''' Add an ''srNOTE'' to <tt>Exercise()</tt> that will add a simple message to the test report (e.g. "Exercise ...")
** ''Hint:'' You will probably want to read about [[Test Macros]]
** ''Hint:'' srTestCaseSetStatus(srTEST_CASE_DEFAULT, srTEST_NOTINUSE, 0) is a placeholder to be removed when you add your test code.
* '''Assignment 2:''' Within <tt>Exercise()</tt> validate that <tt>sut_mult(1,1)</tt> does NOT equal <tt>sut_add(1,1)</tt>
** ''Hint:'' You will use a [[Test Macros | Test Macro]].

=== Check Results ===

* Before you rebuild <tt>TestApp.exe</tt>, you will need to shut it down by entering 'q' in its console window or closing the window directly.

* After rebuilding, invoke <tt>TestApp.exe</tt> once again.

* Execute only ''TestBasic'' using the stride runner:

> stride --options_file myoptions.txt --run TestBasic

Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 4 passed, 1 failed, 0 in progress, 0 not in use.
--------------------------------------------------------------
Summary: 4 passed, 1 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

* If you have TestApp.xml already open in your browser, you can simply refresh (F5), to view the latest test results.

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (<tt>myoptions.txt</tt>) with the following:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

> stride --options_file myoptions.txt --run TestBasic --space TestBasic --upload

Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 4 passed, 1 failed, 0 in progress, 0 not in use.
------------------------------------------------------------
Summary: 4 passed, 1 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...
Uploading to test space...

=== Validate Uploaded Results ===

Navigate to Test Space using your browser, then validate your results against the pre-configured baseline. For details, please see [[Training_Getting_Started#Confirming_Training_Exercise_Results | Confirming Training Exercise Results]]

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

You can view the results on test space by pointing your browser at '''<tt><nowiki>https://yourcompany.stridetestspace.com</nowiki></tt>'''. Log in using the credentials you supplied earlier in the options file.

== Reference ==
The following reference information is related to Test Unit basics.

=== Wiki ===

* [[Test_Units_Overview | Test Units Overview]] - Provides a general overview Test Units (i.e. writing tests in C and C++)
* [[Test_Units |Test Unit Packaging]] - Discusses the three types of packaging that can be used for Test Units (we prefer Test Classes even for c programmers)
* [[Test_Macros|Test Macros]] - Optional macros that provide shortcuts for testing assertions and automatic report annotation (you will want to use these).
* [[Test_Macros#Note_Macros|Notes]] - Used to add logging information to your '''test logic''' (automatically added to test reports)
* [[Test_Log|Test Logs]] - Used to add logging information to your '''source code''' (added to test reports if enabled)
* [[Tracing|Tracing]] - The Runner allows tracing on logs and test points.

=== Samples ===

* [[Test_Class_Sample|Test Class Sample]] - Specifically the [[Test_Class_Sample#Basic|Basic Simple]] Test Unit
* [[Test_Macros_Sample|Test Macro Sample]] - This sample covers simple uses of each of the Test Macros.

Note - Other Test Unit Samples related to packaging that are useful are the following:

* [[Test_CClass_Sample|Test C Class Sample]]
* [[Test_Function_List_Sample|Test Function List Sample]]

[[Category: Training]]

Training Getting Started

2013-01-31T19:40:46Z

Jeffs: /* Training Setup */

== Overview ==

Our training approach is based on a self-guided tour of the [[STRIDE_Overview#STRIDE_Testing_Features | STRIDE Testing Features]] using reference examples and assigned exercises. The set of examples and the implemented exercises will be built and executed using a standard desktop computer.

The ''software under test'', ''reference examples'', and ''exercises'' have been designed to be as '''''simple as possible while sufficiently demonstrating the topic at hand'''''. In particular, the ''software under test'' is '''very light''' on core application logic -- the focus instead is on the test code that leverages STRIDE to define and execute tests.

The user is expected to work through each of the training modules covering a specific testing feature. Once the exercises are completed (actual test cases implemented), the results are published to [[STRIDE_Overview#STRIDE_Test_Space | Test Space]] (and validated against a pre-created baseline).

The training collateral consists of the following:
# The [[STRIDE_Overview#STRIDE_Framework | STRIDE Framework]] used to implement and execute tests
#* The Framework is configured in an [[STRIDE_Off-Target_Environment | Off-Target Environment]]
# A set of specific [[#Training | Training Modules]] that will guide you through the exercises
# [[Main_Page | ''Wiki articles'']] that will provide background material and other technical information
# Reference [[C/C%2B%2B_Samples | Samples]]
# and [[STRIDE_Overview#STRIDE_Test_Space | STRIDE Test Space]]

For more details concerning STRIDE refer to the following:
* ''[[STRIDE_Overview_Video | Overview screencast]]''
* ''[[What is Unique About STRIDE | What is Unique About STRIDE]]''
* ''[[Types of Testing Supported by STRIDE | Types of Testing Supported]]''
* ''[[Frequently Asked Questions About STRIDE | Frequently Asked Questions]]''

''For questions and support [mailto:training@s2technologies.com email us]''

== Before Starting ==
Before you can start setting up your environment you need the following 3 items:
* '''STRIDE Desktop Installation Package''' (one of the following)
:<tt>STRIDE_framework-windows_4.3.yy.zip</tt> (Windows desktop)
''or''
:<tt>STRIDE_framework-linux_4.3.yy.tgz</tt> (Linux desktop)

* '''Training Source files'''
:<tt>STRIDE_training_source.zip</tt>

* '''Test Space User Account URL and Logon Credentials'''
** Test Space URL (<tt>https://YourCompany.stridetestspace.com</tt>)
** User-name
** User-password

=== Desktop Setup ===
The training requires that you install the [[Desktop_Installation | Desktop Framework]] and that the [[STRIDE_Off-Target_Environment | Off-Target Environment]] is setup correctly and verified.

For an overview of the installation steps please refer to [[Installation_Overview | this]] article.

The following steps are required:
# Install your [[Desktop_Installation | desktop Framework package]]
# Read about the [[STRIDE_Off-Target_Environment | Off-Target Environment]]
# Install [[STRIDE_Off-Target_Environment#Host_Compiler | host C++ compiler]] for your desktop (if needed)
# Use the Off-Target SDK to [[Building_an_Off-Target_Test_App | build a Test App]]
# [[Building_an_Off-Target_Test_App#Diagnostics | Run STRIDE diagnostics]] with the Test App built

=== Training Setup ===
The following source code can be found in the '''STRIDE_training_source.zip''' file:

software_under_test.c | h
TestBasic.cpp | h
TestParam.cpp | h
TestFixture.cpp | h
TestExpect.cpp | h
TestRuntime.cpp | h
TestFile.cpp | h
TestDouble.cpp | h

The ''software under test'' is contained in the file '''software_under_test.c | h'''. All of the public functions use '''sut_''' as a prefix. All training modules test against this file. Although the test examples are contained in C++ files, most of the test logic is written in standard C.

* Extract STRIDE_training_source.zip into the directory <tt>%STRIDE_DIR%\SDK\Windows\sample_src</tt> (or <tt>$STRIDE_DIR/SDK/Posix/sample_src</tt> for Linux)
* Rebuild the TestApp following the instructions for [[Building_an_Off-Target_Test_App#Build_Steps | Building a TestApp]].
* List all [[Test_Units | Test Units]] within the generated database file using the following command:

'''Windows'''
> stride --database="%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb" --list

'''Linux'''
> stride --database="$STRIDE_DIR/SDK/Posix/out/TestApp.sidb" --list

The following remoted Functions and Test Units should be displayed:

Functions
sut_strcpy(char const * input, char * output) : void
strlen(char const * _Str) : size_t
Test Units
TestBasic()
TestDouble_Reference()
TestDouble_Definition()
TestExpect_Seq()
TestExpect_Data()
TestExpect_Misc()
TestFile()
TestFixture()
TestParam(int data1, int data2, char * szString)
TestRuntime_Static()
TestRuntime_Dynamic(int NumberOfTestCases)

=== Run Training Tests ===
Here we will run all tests in the <tt>TestApp.sidb</tt> database.<ref>Note that the S2 diagnostic tests are treated separately, and are not run unless the <tt>--diagnostics</tt> option is specified to <tt>stride</tt>.</ref>

* If you haven't done so already, [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using the SDK makefile
* Invoke TestApp found in the ''/out/bin'' directory
* Create an [[Stride_Runner#Options | option file]] (<tt>myoptions.txt</tt>) that includes the following content in <tt>%STRIDE_DIR%\SDK\Windows\myoptions.txt</tt> (or <tt>$STRIDE_DIR/SDK/Posix/myoptions.txt</tt> for Linux)
'''Windows'''
##### Command Line Options ######
--device "TCP:localhost:8000"
--database "%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb"
--output "%STRIDE_DIR%\SDK\Windows\sample_src\TestApp.xml"
--log_level all

'''Linux'''
##### Command Line Options #####
--device "TCP:localhost:8000"
--database "$STRIDE_DIR/SDK/Posix/out/TestApp.sidb"
--output "$STRIDE_DIR/SDK/Posix/sample_src/TestApp.xml"
--log_level all

A couple of things to note:
* If you set up an [[STRIDE_Runner#Environment_Variables | environment variable]] for the '''device''' option then it is not required in the option file. Note: Command line options override environment variables.
* ''stride --help'' provides [[STRIDE_Runner#Options | options information]]

If you haven't done so already, start <tt>TestApp</tt> running in a separate console window.

Now run stride as shown below and verify summary results (starting from the <tt>SDK\Windows</tt> or <tt>SDK/Posix</tt> directory):

> stride --options_file myoptions.txt --run "*"

The output should look like this:
<pre>
Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestParam"
> 0 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------------------
Summary: 21 passed, 3 failed, 0 in progress, 11 not in use.

Disconnecting from device...
Saving result file...

</pre>

=== Interpreting Results ===
Open '''TestApp.xml''' in your browser; this file is created in the directory from which you ran '''stride''' (or at the path specified via the <tt>--output</tt> command line option). If you were connected to the Internet when you ran the tests, the TestApp.xsl file is also generated in the directory. By opening TestApp.xml in a web browser, the xsl is automatically applied to create html in the browser. If you use Google Chrome, please see [[Browser Compatibility]].

If you're interested in the details of the tests, please see the test documentation contained in the test report.

=== Test Space Access ===

As part of the training, users implement exercises (test cases). As each exercise is completed, the results are expected to be uploaded to Test Space. Accessing Test Space (uploading, viewing, etc.) requires a user-name and password. Before working on a training module please confirm that your user account has been set up by [[Test_Space_Setup | logging in]].

Test Space will have expected results using [[Creating_And_Using_Baselines | baselines]]. This is used to automatically verify if the exercises have been implemented correctly (at least to some degree). For capturing test results a '''Training''' project has been created with the following '''Spaces''':

Sandbox - Results for training setup
TestBasic - Basics training results: TestBasic.cpp | h
TestParam - Parameters training results: TestParam.cpp | h
TestFixture - Fixturing training results: TestFixture.cpp | h
TestExpect - Expectations training results: TestExpect.cpp | h
TestRuntime - Runtime API training results: TestRuntime.cpp | h
TestFile - File IO training results: TestFile.cpp | h
TestDouble - Doubling training results: TestDouble.cpp | h

To publish your results using the [[STRIDE_Runner | STRIDE Runner]] the following command-line options should be used:

--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--space ''MODULENAME''
--name ''YOURNAME''

Notes:
* Concerning the ''MODULENAME'' option: use the name that corresponds to the training module currently underway (e.g. TestBasic, TestParam, etc.)
* ''YOURNAME'' should be set to your name (i.e. JohnD); omit spaces from this string
* If you access the Internet via an HTTP proxy please read [[STRIDE_Runner#Using_a_Proxy | '''this article''']]

To make it easier for now we recommend that you update your existing option file (<tt>myoptions.txt</tt>) with the following:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

=== Publish Test Space Results ===

To complete the setup publish your results to Test Space. Please make sure to use '''Sandbox''' as the space to upload your results to (see below).

> stride --options_file myoptions.txt --run "*" --space Sandbox --upload

Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestParam"
> 0 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------------------
Summary: 21 passed, 3 failed, 0 in progress, 11 not in use.
Disconnecting from device...
Saving result file...
Uploading to test space...

=== Confirming Setup ===

After you have run and uploaded the results of the training setup, you should confirm the correctness of your work. The training setup results will end up with a mix of passes and failures, so we use a [[Creating_And_Using_Baselines | Test Space baseline]] to get an aggregate comparison between expected and actual results.

To validate your setup:
# Navigate to Sandbox set in Test Space and look under the column labeled '''''SEQUENTIAL COMPARISON'''''
# Confirm that '''''none''''' is indicated for each of the sub-column. If so, you have completed the setup and are ready to move onto the training modules.

[[image:STRIDE_results_012813.JPG|none|600px| Test Space baseline example]]

== Training ==
At this point you should be ready to start the actual training. There are '''7 separate training modules''' and we recommend the following order:

===Introductory===
* [[Training_Basics | '''Basics''']] - Covers basics of implementing and executing test cases
* [[Training_Parameters |'''Parameters''']] - How to pass parameters to a test
* [[Training_Fixturing | '''Fixturing''']] - Leveraging setup and teardown featuring

===Advanced===
* [[Training_Expectations | '''Expectations''']] - Validating code sequencing along with state data
* [[Training_Runtime_API | '''Runtime API''']] - Using the runtime services to dynamically create test suits / cases
* [[Training_File_IO | '''File IO''']] - Reading and writing files existing on the host
* [[Training_Doubling | '''Doubling''']] - Replacing a dependency with a stub, fake, or mock

== Training Confirmation ==

As you run and upload each test unit containing your worked training exercises, you should confirm the correctness of your work.

The correctly worked training test units will end up with a mix of passes and failures, so we use a [[Creating_And_Using_Baselines | Test Space baseline]] to get an aggregate comparison between expected and actual results

To validate your work:
# Navigate to your result set in Test Space and look under the column labeled '''''SEQUENTIAL COMPARISON'''''
# Confirm that '''''none''''' is indicated for each of the sub-column.

==Notes==
<references/>

[[Category: Training]]

Training Getting Started

2013-01-30T22:22:54Z

Jeffs: /* Run Training Tests */

== Overview ==

Our training approach is based on a self-guided tour of the [[STRIDE_Overview#STRIDE_Testing_Features | STRIDE Testing Features]] using reference examples and assigned exercises. The set of examples and the implemented exercises will be built and executed using a standard desktop computer.

The ''software under test'', ''reference examples'', and ''exercises'' have been designed to be as '''''simple as possible while sufficiently demonstrating the topic at hand'''''. In particular, the ''software under test'' is '''very light''' on core application logic -- the focus instead is on the test code that leverages STRIDE to define and execute tests.

The user is expected to work through each of the training modules covering a specific testing feature. Once the exercises are completed (actual test cases implemented), the results are published to [[STRIDE_Overview#STRIDE_Test_Space | Test Space]] (and validated against a pre-created baseline).

The training collateral consists of the following:
# The [[STRIDE_Overview#STRIDE_Framework | STRIDE Framework]] used to implement and execute tests
#* The Framework is configured in an [[STRIDE_Off-Target_Environment | Off-Target Environment]]
# A set of specific [[#Training | Training Modules]] that will guide you through the exercises
# [[Main_Page | ''Wiki articles'']] that will provide background material and other technical information
# Reference [[C/C%2B%2B_Samples | Samples]]
# and [[STRIDE_Overview#STRIDE_Test_Space | STRIDE Test Space]]

For more details concerning STRIDE refer to the following:
* ''[[STRIDE_Overview_Video | Overview screencast]]''
* ''[[What is Unique About STRIDE | What is Unique About STRIDE]]''
* ''[[Types of Testing Supported by STRIDE | Types of Testing Supported]]''
* ''[[Frequently Asked Questions About STRIDE | Frequently Asked Questions]]''

''For questions and support [mailto:training@s2technologies.com email us]''

== Before Starting ==
Before you can start setting up your environment you need the following 3 items:
* '''STRIDE Desktop Installation Package''' (one of the following)
:<tt>STRIDE_framework-windows_4.3.yy.zip</tt> (Windows desktop)
''or''
:<tt>STRIDE_framework-linux_4.3.yy.tgz</tt> (Linux desktop)

* '''Training Source files'''
:<tt>STRIDE_training_source.zip</tt>

* '''Test Space User Account URL and Logon Credentials'''
** Test Space URL (<tt>https://YourCompany.stridetestspace.com</tt>)
** User-name
** User-password

=== Desktop Setup ===
The training requires that you install the [[Desktop_Installation | Desktop Framework]] and that the [[STRIDE_Off-Target_Environment | Off-Target Environment]] is setup correctly and verified.

For an overview of the installation steps please refer to [[Installation_Overview | this]] article.

The following steps are required:
# Install your [[Desktop_Installation | desktop Framework package]]
# Read about the [[STRIDE_Off-Target_Environment | Off-Target Environment]]
# Install [[STRIDE_Off-Target_Environment#Host_Compiler | host C++ compiler]] for your desktop (if needed)
# Use the Off-Target SDK to [[Building_an_Off-Target_Test_App | build a Test App]]
# [[Building_an_Off-Target_Test_App#Diagnostics | Run STRIDE diagnostics]] with the Test App built

=== Training Setup ===
The following source code can be found in the '''STRIDE_training_source.zip''' file:

software_under_test.c | h
TestBasic.cpp | h
TestParam.cpp | h
TestFixture.cpp | h
TestExpect.cpp | h
TestRuntime.cpp | h
TestFile.cpp | h
TestDouble.cpp | h

The ''software under test'' is contained in the file '''software_under_test.c | h'''. All of the public functions use '''sut_''' as a prefix. All training modules test against this file. Although the test examples are contained in C++ files, most of the test logic is written in standard C.

* Extract STRIDE_training_source.zip into the directory <tt>%STRIDE_DIR%\SDK\Windows\sample_src</tt> (or <tt>$STRIDE_DIR/SDK/Posix/sample_src</tt> for Linux)
* Rebuild the TestApp following the instructions for [[Building_an_Off-Target_Test_App | Building a TestApp]].
* List all [[Test_Units | Test Units]] within the generated database file using the following command:

'''Windows'''
> stride --database="%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb" --list

'''Linux'''
> stride --database="$STRIDE_DIR/SDK/Posix/out/TestApp.sidb" --list

The following remoted Functions and Test Units should be displayed:

Functions
sut_strcpy(char const * input, char * output) : void
strlen(char const * _Str) : size_t
Test Units
TestBasic()
TestDouble_Reference()
TestDouble_Definition()
TestExpect_Seq()
TestExpect_Data()
TestExpect_Misc()
TestFile()
TestFixture()
TestParam(int data1, int data2, char * szString)
TestRuntime_Static()
TestRuntime_Dynamic(int NumberOfTestCases)

=== Run Training Tests ===
Here we will run all tests in the <tt>TestApp.sidb</tt> database.<ref>Note that the S2 diagnostic tests are treated separately, and are not run unless the <tt>--diagnostics</tt> option is specified to <tt>stride</tt>.</ref>

* If you haven't done so already, [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using the SDK makefile
* Invoke TestApp found in the ''/out/bin'' directory
* Create an [[Stride_Runner#Options | option file]] (<tt>myoptions.txt</tt>) that includes the following content in <tt>%STRIDE_DIR%\SDK\Windows\myoptions.txt</tt> (or <tt>$STRIDE_DIR/SDK/Posix/myoptions.txt</tt> for Linux)
'''Windows'''
##### Command Line Options ######
--device "TCP:localhost:8000"
--database "%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb"
--output "%STRIDE_DIR%\SDK\Windows\sample_src\TestApp.xml"
--log_level all

'''Linux'''
##### Command Line Options #####
--device "TCP:localhost:8000"
--database "$STRIDE_DIR/SDK/Posix/out/TestApp.sidb"
--output "$STRIDE_DIR/SDK/Posix/sample_src/TestApp.xml"
--log_level all

A couple of things to note:
* If you set up an [[STRIDE_Runner#Environment_Variables | environment variable]] for the '''device''' option then it is not required in the option file. Note: Command line options override environment variables.
* ''stride --help'' provides [[STRIDE_Runner#Options | options information]]

If you haven't done so already, start <tt>TestApp</tt> running in a separate console window.

Now run stride as shown below and verify summary results (starting from the <tt>SDK\Windows</tt> or <tt>SDK/Posix</tt> directory):

> stride --options_file myoptions.txt --run "*"

The output should look like this:
<pre>
Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestParam"
> 0 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------------------
Summary: 21 passed, 3 failed, 0 in progress, 11 not in use.

Disconnecting from device...
Saving result file...

</pre>

=== Interpreting Results ===
Open '''TestApp.xml''' in your browser; this file is created in the directory from which you ran '''stride''' (or at the path specified via the <tt>--output</tt> command line option). If you were connected to the Internet when you ran the tests, the TestApp.xsl file is also generated in the directory. By opening TestApp.xml in a web browser, the xsl is automatically applied to create html in the browser. If you use Google Chrome, please see [[Browser Compatibility]].

If you're interested in the details of the tests, please see the test documentation contained in the test report.

=== Test Space Access ===

As part of the training, users implement exercises (test cases). As each exercise is completed, the results are expected to be uploaded to Test Space. Accessing Test Space (uploading, viewing, etc.) requires a user-name and password. Before working on a training module please confirm that your user account has been set up by [[Test_Space_Setup | logging in]].

Test Space will have expected results using [[Creating_And_Using_Baselines | baselines]]. This is used to automatically verify if the exercises have been implemented correctly (at least to some degree). For capturing test results a '''Training''' project has been created with the following '''Spaces''':

Sandbox - Results for training setup
TestBasic - Basics training results: TestBasic.cpp | h
TestParam - Parameters training results: TestParam.cpp | h
TestFixture - Fixturing training results: TestFixture.cpp | h
TestExpect - Expectations training results: TestExpect.cpp | h
TestRuntime - Runtime API training results: TestRuntime.cpp | h
TestFile - File IO training results: TestFile.cpp | h
TestDouble - Doubling training results: TestDouble.cpp | h

To publish your results using the [[STRIDE_Runner | STRIDE Runner]] the following command-line options should be used:

--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--space ''MODULENAME''
--name ''YOURNAME''

Notes:
* Concerning the ''MODULENAME'' option: use the name that corresponds to the training module currently underway (e.g. TestBasic, TestParam, etc.)
* ''YOURNAME'' should be set to your name (i.e. JohnD); omit spaces from this string
* If you access the Internet via an HTTP proxy please read [[STRIDE_Runner#Using_a_Proxy | '''this article''']]

To make it easier for now we recommend that you update your existing option file (<tt>myoptions.txt</tt>) with the following:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

=== Publish Test Space Results ===

To complete the setup publish your results to Test Space. Please make sure to use '''Sandbox''' as the space to upload your results to (see below).

> stride --options_file myoptions.txt --run "*" --space Sandbox --upload

Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestParam"
> 0 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------------------
Summary: 21 passed, 3 failed, 0 in progress, 11 not in use.
Disconnecting from device...
Saving result file...
Uploading to test space...

=== Confirming Setup ===

After you have run and uploaded the results of the training setup, you should confirm the correctness of your work. The training setup results will end up with a mix of passes and failures, so we use a [[Creating_And_Using_Baselines | Test Space baseline]] to get an aggregate comparison between expected and actual results.

To validate your setup:
# Navigate to Sandbox set in Test Space and look under the column labeled '''''SEQUENTIAL COMPARISON'''''
# Confirm that '''''none''''' is indicated for each of the sub-column. If so, you have completed the setup and are ready to move onto the training modules.

[[image:STRIDE_results_012813.JPG|none|600px| Test Space baseline example]]

== Training ==
At this point you should be ready to start the actual training. There are '''7 separate training modules''' and we recommend the following order:

===Introductory===
* [[Training_Basics | '''Basics''']] - Covers basics of implementing and executing test cases
* [[Training_Parameters |'''Parameters''']] - How to pass parameters to a test
* [[Training_Fixturing | '''Fixturing''']] - Leveraging setup and teardown featuring

===Advanced===
* [[Training_Expectations | '''Expectations''']] - Validating code sequencing along with state data
* [[Training_Runtime_API | '''Runtime API''']] - Using the runtime services to dynamically create test suits / cases
* [[Training_File_IO | '''File IO''']] - Reading and writing files existing on the host
* [[Training_Doubling | '''Doubling''']] - Replacing a dependency with a stub, fake, or mock

== Training Confirmation ==

As you run and upload each test unit containing your worked training exercises, you should confirm the correctness of your work.

The correctly worked training test units will end up with a mix of passes and failures, so we use a [[Creating_And_Using_Baselines | Test Space baseline]] to get an aggregate comparison between expected and actual results

To validate your work:
# Navigate to your result set in Test Space and look under the column labeled '''''SEQUENTIAL COMPARISON'''''
# Confirm that '''''none''''' is indicated for each of the sub-column.

==Notes==
<references/>

[[Category: Training]]

Training Getting Started

2013-01-30T22:18:52Z

Jeffs: /* Training Setup */

== Overview ==

Our training approach is based on a self-guided tour of the [[STRIDE_Overview#STRIDE_Testing_Features | STRIDE Testing Features]] using reference examples and assigned exercises. The set of examples and the implemented exercises will be built and executed using a standard desktop computer.

The ''software under test'', ''reference examples'', and ''exercises'' have been designed to be as '''''simple as possible while sufficiently demonstrating the topic at hand'''''. In particular, the ''software under test'' is '''very light''' on core application logic -- the focus instead is on the test code that leverages STRIDE to define and execute tests.

The user is expected to work through each of the training modules covering a specific testing feature. Once the exercises are completed (actual test cases implemented), the results are published to [[STRIDE_Overview#STRIDE_Test_Space | Test Space]] (and validated against a pre-created baseline).

The training collateral consists of the following:
# The [[STRIDE_Overview#STRIDE_Framework | STRIDE Framework]] used to implement and execute tests
#* The Framework is configured in an [[STRIDE_Off-Target_Environment | Off-Target Environment]]
# A set of specific [[#Training | Training Modules]] that will guide you through the exercises
# [[Main_Page | ''Wiki articles'']] that will provide background material and other technical information
# Reference [[C/C%2B%2B_Samples | Samples]]
# and [[STRIDE_Overview#STRIDE_Test_Space | STRIDE Test Space]]

For more details concerning STRIDE refer to the following:
* ''[[STRIDE_Overview_Video | Overview screencast]]''
* ''[[What is Unique About STRIDE | What is Unique About STRIDE]]''
* ''[[Types of Testing Supported by STRIDE | Types of Testing Supported]]''
* ''[[Frequently Asked Questions About STRIDE | Frequently Asked Questions]]''

''For questions and support [mailto:training@s2technologies.com email us]''

== Before Starting ==
Before you can start setting up your environment you need the following 3 items:
* '''STRIDE Desktop Installation Package''' (one of the following)
:<tt>STRIDE_framework-windows_4.3.yy.zip</tt> (Windows desktop)
''or''
:<tt>STRIDE_framework-linux_4.3.yy.tgz</tt> (Linux desktop)

* '''Training Source files'''
:<tt>STRIDE_training_source.zip</tt>

* '''Test Space User Account URL and Logon Credentials'''
** Test Space URL (<tt>https://YourCompany.stridetestspace.com</tt>)
** User-name
** User-password

=== Desktop Setup ===
The training requires that you install the [[Desktop_Installation | Desktop Framework]] and that the [[STRIDE_Off-Target_Environment | Off-Target Environment]] is setup correctly and verified.

For an overview of the installation steps please refer to [[Installation_Overview | this]] article.

The following steps are required:
# Install your [[Desktop_Installation | desktop Framework package]]
# Read about the [[STRIDE_Off-Target_Environment | Off-Target Environment]]
# Install [[STRIDE_Off-Target_Environment#Host_Compiler | host C++ compiler]] for your desktop (if needed)
# Use the Off-Target SDK to [[Building_an_Off-Target_Test_App | build a Test App]]
# [[Building_an_Off-Target_Test_App#Diagnostics | Run STRIDE diagnostics]] with the Test App built

=== Training Setup ===
The following source code can be found in the '''STRIDE_training_source.zip''' file:

software_under_test.c | h
TestBasic.cpp | h
TestParam.cpp | h
TestFixture.cpp | h
TestExpect.cpp | h
TestRuntime.cpp | h
TestFile.cpp | h
TestDouble.cpp | h

The ''software under test'' is contained in the file '''software_under_test.c | h'''. All of the public functions use '''sut_''' as a prefix. All training modules test against this file. Although the test examples are contained in C++ files, most of the test logic is written in standard C.

* Extract STRIDE_training_source.zip into the directory <tt>%STRIDE_DIR%\SDK\Windows\sample_src</tt> (or <tt>$STRIDE_DIR/SDK/Posix/sample_src</tt> for Linux)
* Rebuild the TestApp following the instructions for [[Building_an_Off-Target_Test_App | Building a TestApp]].
* List all [[Test_Units | Test Units]] within the generated database file using the following command:

'''Windows'''
> stride --database="%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb" --list

'''Linux'''
> stride --database="$STRIDE_DIR/SDK/Posix/out/TestApp.sidb" --list

The following remoted Functions and Test Units should be displayed:

Functions
sut_strcpy(char const * input, char * output) : void
strlen(char const * _Str) : size_t
Test Units
TestBasic()
TestDouble_Reference()
TestDouble_Definition()
TestExpect_Seq()
TestExpect_Data()
TestExpect_Misc()
TestFile()
TestFixture()
TestParam(int data1, int data2, char * szString)
TestRuntime_Static()
TestRuntime_Dynamic(int NumberOfTestCases)

=== Run Training Tests ===
Here we will run all tests in the <tt>TestApp.sidb</tt> database.<ref>Note that the S2 diagnostic tests are treated separately, and are not run unless the <tt>--diagnostics</tt> option is specified to <tt>stride</tt>.</ref>

* If you haven't done so already, [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using the SDK makefile
* Invoke TestApp found in the ''/out/bin'' directory
* Create an [[Stride_Runner#Options | option file]] (<tt>myoptions.txt</tt>) that includes the following content (SDK\Windows or SDK/Posix).

'''Windows'''
##### Command Line Options ######
--device "TCP:localhost:8000"
--database "%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb"
--output "%STRIDE_DIR%\SDK\Windows\sample_src\TestApp.xml"
--log_level all

'''Linux'''
##### Command Line Options #####
--device "TCP:localhost:8000"
--database "$STRIDE_DIR/SDK/Posix/out/TestApp.sidb"
--output "$STRIDE_DIR/SDK/Posix/sample_src/TestApp.xml"
--log_level all

A couple of things to note:
* If you set up an [[STRIDE_Runner#Environment_Variables | environment variable]] for the '''device''' option then it is not required in the option file. Note: Command line options override environment variables.
* ''stride --help'' provides [[STRIDE_Runner#Options | options information]]

If you haven't done so already, start <tt>TestApp</tt> running in a separate console window.

Now run stride as shown below and verify summary results (starting from the <tt>SDK\Windows</tt> or <tt>SDK/Posix</tt> directory):

> stride --options_file myoptions.txt --run "*"

The output should look like this:
<pre>
Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestParam"
> 0 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------------------
Summary: 21 passed, 3 failed, 0 in progress, 11 not in use.

Disconnecting from device...
Saving result file...

</pre>

=== Interpreting Results ===
Open '''TestApp.xml''' in your browser; this file is created in the directory from which you ran '''stride''' (or at the path specified via the <tt>--output</tt> command line option). If you were connected to the Internet when you ran the tests, the TestApp.xsl file is also generated in the directory. By opening TestApp.xml in a web browser, the xsl is automatically applied to create html in the browser. If you use Google Chrome, please see [[Browser Compatibility]].

If you're interested in the details of the tests, please see the test documentation contained in the test report.

=== Test Space Access ===

As part of the training, users implement exercises (test cases). As each exercise is completed, the results are expected to be uploaded to Test Space. Accessing Test Space (uploading, viewing, etc.) requires a user-name and password. Before working on a training module please confirm that your user account has been set up by [[Test_Space_Setup | logging in]].

Test Space will have expected results using [[Creating_And_Using_Baselines | baselines]]. This is used to automatically verify if the exercises have been implemented correctly (at least to some degree). For capturing test results a '''Training''' project has been created with the following '''Spaces''':

Sandbox - Results for training setup
TestBasic - Basics training results: TestBasic.cpp | h
TestParam - Parameters training results: TestParam.cpp | h
TestFixture - Fixturing training results: TestFixture.cpp | h
TestExpect - Expectations training results: TestExpect.cpp | h
TestRuntime - Runtime API training results: TestRuntime.cpp | h
TestFile - File IO training results: TestFile.cpp | h
TestDouble - Doubling training results: TestDouble.cpp | h

To publish your results using the [[STRIDE_Runner | STRIDE Runner]] the following command-line options should be used:

--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--space ''MODULENAME''
--name ''YOURNAME''

Notes:
* Concerning the ''MODULENAME'' option: use the name that corresponds to the training module currently underway (e.g. TestBasic, TestParam, etc.)
* ''YOURNAME'' should be set to your name (i.e. JohnD); omit spaces from this string
* If you access the Internet via an HTTP proxy please read [[STRIDE_Runner#Using_a_Proxy | '''this article''']]

To make it easier for now we recommend that you update your existing option file (<tt>myoptions.txt</tt>) with the following:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

=== Publish Test Space Results ===

To complete the setup publish your results to Test Space. Please make sure to use '''Sandbox''' as the space to upload your results to (see below).

> stride --options_file myoptions.txt --run "*" --space Sandbox --upload

Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestParam"
> 0 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------------------
Summary: 21 passed, 3 failed, 0 in progress, 11 not in use.
Disconnecting from device...
Saving result file...
Uploading to test space...

=== Confirming Setup ===

After you have run and uploaded the results of the training setup, you should confirm the correctness of your work. The training setup results will end up with a mix of passes and failures, so we use a [[Creating_And_Using_Baselines | Test Space baseline]] to get an aggregate comparison between expected and actual results.

To validate your setup:
# Navigate to Sandbox set in Test Space and look under the column labeled '''''SEQUENTIAL COMPARISON'''''
# Confirm that '''''none''''' is indicated for each of the sub-column. If so, you have completed the setup and are ready to move onto the training modules.

[[image:STRIDE_results_012813.JPG|none|600px| Test Space baseline example]]

== Training ==
At this point you should be ready to start the actual training. There are '''7 separate training modules''' and we recommend the following order:

===Introductory===
* [[Training_Basics | '''Basics''']] - Covers basics of implementing and executing test cases
* [[Training_Parameters |'''Parameters''']] - How to pass parameters to a test
* [[Training_Fixturing | '''Fixturing''']] - Leveraging setup and teardown featuring

===Advanced===
* [[Training_Expectations | '''Expectations''']] - Validating code sequencing along with state data
* [[Training_Runtime_API | '''Runtime API''']] - Using the runtime services to dynamically create test suits / cases
* [[Training_File_IO | '''File IO''']] - Reading and writing files existing on the host
* [[Training_Doubling | '''Doubling''']] - Replacing a dependency with a stub, fake, or mock

== Training Confirmation ==

As you run and upload each test unit containing your worked training exercises, you should confirm the correctness of your work.

The correctly worked training test units will end up with a mix of passes and failures, so we use a [[Creating_And_Using_Baselines | Test Space baseline]] to get an aggregate comparison between expected and actual results

To validate your work:
# Navigate to your result set in Test Space and look under the column labeled '''''SEQUENTIAL COMPARISON'''''
# Confirm that '''''none''''' is indicated for each of the sub-column.

==Notes==
<references/>

[[Category: Training]]

Training Expectations

2013-01-29T17:44:52Z

Jeffs: /* Implement Exercise */

== Objectives ==

This Training Module is focused on ''Test Points'' and how to validate them using ''Expectations''. The module covers the following topics:
* Presentation of a [[Expectations | validation]] technique based on ''code sequencing'' and ''state data''
* Overview of [[Source_Instrumentation_Overview#Instrumentation | source instrumentation]]
* Review of [[Test_Point_Testing_in_C/C%2B%2B | expectation tables and predicates]]
* Example use cases such as concurrent validation, using trigger conditions, etc.

There are two test files used -- '''TestExpect.cpp & TestExpect.h''' -- that implement three Test Units:
* '''TestExpect_Seq'''
* '''TestExpect_Data'''
* '''TestExpect_Misc'''

The first two Test Units have two test methods already implemented and have one method each that you are required to implement called '''Exercise'''. The third Test Unit has four test methods already implemented and also has one method to implement as an exercise. Currently the ''exercise methods'' return a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Expectations'' Test Units only

> stride --options_file myoptions.txt --run TestExpect_Seq --run TestExpect_Data --run TestExpect_Misc

Loading database...
Connecting to device...
Executing...
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
-----------------------------------------------------------
Summary: 7 passed, 1 failed, 0 in progress, 3 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestExpect_Seq::Exercise'''
** Validate '''ALL''' upper case Test Points (A - I)
** Use ''Unordered'' and ''Nonstrict'' sequencing
** Use ''sut_DoSequencing(SEQ_1)'' to generate part of the sequence
** Use ''sut_start_thread(SEQ_3)'' to generate the rest of the sequence
** Add a ''NOTE_INFO(..)'' to log that the test is executing

* '''TestExpect_Data::Exercise'''
** Validate the following Test Points '''{D, G, F, H}'''
** Check that '''F''' occurs 2 times
** Use ''Unordered'' sequencing
** Write a new custom predicate that validates data for both '''D''' and '''H''' Test Points
*** Add ''NOTE_INFO(..)'' to capture content of the Test Points
*** Add extra check for '''D''' that the status is '''GOOD'''
*** Confirm that data fields '''d1''' and '''d2''' are as expected
*** Pass the expected data fields (for both Test Points) as part of the '''user''' data within the setup
*** Use ''sut_start_thread(SEQ_3)'' to generate the sequence
** Write another custom predicate for validating data for '''G'''
*** Add ''NOTE_INFO(..)'' to capture content of the Test Point
*** Validate the expected string using '''user''' data
** Add a ''NOTE_INFO(..)'' to log that the test is executing

* '''TestExpect_Misc::Exercise'''
** Validate 2 sequences using a Trigger in between
*** Sequence 1 = '''D''' '''E''' and '''A'''
*** Trigger = '''C'''
*** Sequence 2 = '''F''' and '''F''' (2 occurrences)
** Use ''ANY AT ALL'' special member with trigger
** Use ''Ordered'' and ''Strict'' sequencing
** Use ''sut_start_thread(SEQ_2)'' to generate the expected sequences

* Execute ''Test Expectations'' Test Units only

> stride --options_file myoptions.txt --run TestExpect_Seq --run TestExpect_Data --run TestExpect_Misc

Loading database...
Connecting to device...
Executing...
test unit "TestExpect_Seq"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestExpect_Data"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestExpect_Misc"
> 4 passed, 1 failed, 0 in progress, 0 not in use.
-----------------------------------------------------------
Summary: 10 passed, 1 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following if not already done:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

> stride --options_file myoptions.txt --run TestExpect_Seq --run TestExpect_Data --run TestExpect_Misc --space TestExpect --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===

* [[Source_Instrumentation_Overview | Instrumentation Overview]] providing high-level concepts of this validation technique
* [[Test_Point |Test Point]] Macro definition
* [[Expectations |Expectations]] definition and how to set your ''Expectations''

=== Samples ===

* [[Test_Point_Sample | Test Point Sample]] - Demonstrates simple technique to monitor and test activity occurring in another thread

[[Category: Training]]

Training Fixturing

2013-01-29T17:33:32Z

Jeffs: /* Implement Exercise */

== Objectives ==

This Training Module focuses on '''Fixturing''' in the context of the STRIDE Test System. For a high-level overview refer to the following wiki section on [[What_is_Unique_About_STRIDE#Fixturing | fixturing]]. This module covers the following topics:
* Startup logic at the beginning of a Test Unit
* [[Test_Unit_Pragmas#Fixturing_Pragmas | Setup]] logic for each test method
* [[Test_Unit_Pragmas#Fixturing_Pragmas | Teardown]] logic for each test method

There are two test files used -- '''TestFixture.cpp & TestFixture.h'''. These implement one Test Unit:
* '''TestFixture'''

The Test Unit has one test case already implemented (used for reference) and has '''two test methods''' that you are required to implement (called '''Exercise1''' and '''Exercise2'''). Currently the ''Exercise1'' method is empty and returns a ''NOT IN USE'' status. The ''Exercise2'' method does not yet exist.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Fixture'' Test Unit only

> stride -options_file myoptions.txt --run TestFixture

Loading database...
Connecting to device...
Executing...
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
--------------------------------------------------------------
Summary: 1 passed, 0 failed, 0 in progress, 1 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestFixture::Exercise1'''
** Add a ''NOTE'' that will display the ''name'' of the test.
** Validate that '''Sequence2''' is stored when starting the Thread by checking that "Sequence2" equals MyString[0].
*** ''HINT'': The method to focus on in SUT is DoSequencing()
*** ''HINT'': m_sequence is incremented as each method goes through Teardown.

* '''TestFixture::Exercise2'''
** Add new test method declaration called ''Exercise2'' to TestFixture.h
** Add new test method implementation of ''Exercise2'' to TestFixture.cpp
** Add a ''NOTE'' that will display the name of the test
** Validate that '''Sequence3''' is persisted when starting the Thread

* Execute only ''TestFixture''

> stride --options_file myoptions.txt --run TestFixture

Loading database...
Connecting to device...
Executing...
test unit "TestFixture"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
--------------------------------------------------------------
Summary: 3 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following if not already done:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

> stride --options_file myoptions.txt --run TestFixture --space TestFixture --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===

* [[Test_Unit_Pragmas#Fixturing_Pragmas | Fixturing Pragmas]] -- important information for setting up and tearing down ''things'' between each executed test method within a Test Unit
** Review the types of Test Units these pragmas are applicable for

=== Samples ===

* [[Test_Class_Sample | Test Class Sample]] - Specifically the [[Test_Class_Sample#Basic::Fixtures | Basic Fixtures]] Test Unit
* [[Test_CClass_Sample | Test C Class Sample]] - Specifically the [[Test_CClass_Sample#basic_fixtures | Basic Fixtures]] Test Unit
* [[Test_Function_List_Sample | Test Function List Sample]]- Specifically the [[Test_Function_List_Sample#basic_fixtures |Basic Fixtures]] Test Unit

[[Category: Training]]

Training Parameters

2013-01-29T17:20:44Z

Jeffs: /* Implement Exercise */

== Objectives ==

This Training Module is focused on '''Parameter passing''' in the context of the Test Unit. The module covers the following:
* [[Parameterized_Test_Units | Passing parameters]] via the Runner

There are two test files used -- '''TestParam.cpp & TestParam.h'''. These implement one Test Unit:
* '''TestParam'''

The Test Unit has test cases already implemented (used for reference) and has one test method that you are required to implement (called '''Exercise'''). Currently this method is empty and returns a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Param'' Test Unit only
: Note that parameters are passed to the test unit from the host command line.

> stride --options_file myoptions.txt --run "TestParam(9,4,\"STRING1\")"

Loading database...
Connecting to device...
Executing...
test unit "TestParam"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
--------------------------------------------------------------
Summary: 1 passed, 0 failed, 0 in progress, 1 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

;Exercise()
* Add a ''NOTE'' that will display the ''integer parameters'' passed via command line
* Add an Assert macro guarding against no parameters being passed (i.e. default value of 0 received)
* Call ''sut_add()'' with parameters and validate that it returns the expected value
* Call ''sut_mult()'' with parameters and validate that it returns the expected value
** ''Hint:'' You will probably want to read about [[Parameterized Test Units]]

* Execute only ''TestParam'' '''without''' parameters

> stride --options_file myoptions.txt --run TestParam

Loading database...
Connecting to device...
Executing...
test unit "TestParam"
> 0 passed, 2 failed, 0 in progress, 0 not in use.
--------------------------------------------------------------
Summary: 0 passed, 2 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

* Execute only ''TestParam'' '''with''' parameters
> stride -O myoptions.txt --run "TestParam(9,4,\"STRING1\")"

Loading database...
Connecting to device...
Executing...
test unit "TestParam"
> 2 passed, 0 failed, 0 in progress, 0 not in use.
--------------------------------------------------------------
Summary: 2 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following if not already done:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

> stride --options_file myoptions.txt --run "TestParam(9,4,\"STRING1\")" --space TestParam --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===

* [[Parameterized_Test_Units |Parameterized Test Units]] - Describes how to supply parameters to your Test Units
* [[STRIDE_Runner | STRIDE Runner]] - Information on command line options
** Refer to "Test Unit Specification Examples" - specifically examples using parameters
** Review <tt>run[--r] arg</tt> option

=== Samples ===
* [[Test_Class_Sample | Test Class Sample]] - Specifically the [[Test_Class_Sample#Basic::Parameterized | Parameterized]] Test Unit
* [[Test_CClass_Sample | Test C Class Sample]] - Specifically the [[Test_CClass_Sample#basic_parameterized | Parameterized]] Test Unit

[[Category: Training]]

Training Doubling

2013-01-28T18:53:08Z

Jeffs: /* Build and Run TestApp */

== Objectives ==

This Training Module is focused on explaining how to leverage '''Test Doubles''' in the context of executing a test. For an overview of '''intercepting''' existing functions please refer to [[Using_Test_Doubles | Test Doubles]]. The module covers following topics:
* How to apply pragmas for [[Function_Capturing | function intercepting]]
* How to decide what kind of ''mangling'' is required
** ''Definition'' verses ''Reference''
** ''Explicit'' verses ''Implicit''
* Setting and Resetting the Double implementation

There are two new files used -- '''TestDouble.cpp & TestDouble.h''' --- that implement two test Units:
* '''TestDouble_Reference'''
* '''TestDouble_Definition'''

All of the Test Units have test cases already implemented (used for reference) and have one test method that you are required to implement called '''Exercise'''. Currently the exercise methods return a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Double'' Test Units only

> stride --options_file myoptions.txt --run TestDouble_Reference --run TestDouble_Definition

Loading database...
Connecting to device...
Executing...
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------
Summary: 4 passed, 0 failed, 0 in progress, 2 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestDouble_Reference::Exercise'''
** Implement a new ''test double'' for the ''strlen()'' routine:
*** Add a ''NOTE'' capturing its name when it is called
*** Uses the real ''strlen()'' to return the length of the passed in string
** Use ''srEXPECT_EQ()'' to validate that ''sut_strcheck()'' returns correct length
** Make sure to restore the original routine

* '''TestDouble_Definition::Exercise'''
** Implement a new ''test double'' for the ''sut_strcpy()'' routine:
*** Log its name when it is called
*** Validate string passed to ''sut_strsave()'' is received correctly by the ''test double''
*** Remember that ''sut_strsave()'' calls ''sut_strcpy()'' with the string passed to it
*** Can use a test macro to validate the string is correctly passed to the double
*** Call the original function (''sut_strcpy()'') with a '''DIFFERENT STRING''' (i.e "Exercise Test String 2")
**** Call the original within the test double
**** Make sure to reset the original function within the mock before calling it (i.e. srDOUBLE_RESET)
** Call ''sut_strsave()'' with a string (i.e. "Exercise Test String 1")
** Use ''sut_strget()'' to retrieve a string
** Compare retrieved string with the '''DIFFERENT STRING''' inserted by the ''test double''
** NOTE - won't work if you wait until the end of the test to restore the original routine (i.e. srDOUBLE_SET)

* Execute ''Test Double'' Test Units only

> stride --options_file myoptions.txt --run TestDouble_Reference TestDouble_Definition

Loading database...
Connecting to device...
Executing...
test unit "TestDouble_Reference"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestDouble_Definition"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------
Summary: 6 passed, 0 failed, 0 in progress, 0 not in use.

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following if not already done:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

> stride --options_file myoptions.txt --run TestDouble_Reference TestDouble_Definition --space TestDouble --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===
* [[Using_Test_Doubles | Using Test Doubles]] - Outlines the basic steps to enable a ''double''
** Capture the f(x) that is going to be doubled
** Code the test logic to additionally set the Double and restore original f(x)

* [[Scl_function | Intercepting a function]] - Pragma specifics outline
** For this exercise, there are '''NO''' optional parameters when capturing a f(x) to be '''intercepted'''

* Details of the pragma parameters can be understood in the [[Intercept_Module#Interceptor | Intercept Module]] article. Please review in detail the following:
** ''context'' - This refers to incepting a callee or the called routine.
** ''name-mangling'' - How the function name is switched during the compiling process
** ''group-id'' - Used to associate a group of intercepted functions

* Some '''external links''' to review:
** [http://www.martinfowler.com/bliki/TestDouble.html Martin Fowler definition]
** [http://en.wikipedia.org/wiki/Test_Double Wikipedia definition]
** [http://msdn.microsoft.com/en-us/magazine/cc163358.aspx MSDN Magazine description]

=== Samples ===

* [[Test_Double_Sample | Test Double Sample]] that can be a useful reference. This reference example can be built and executed like all of samples using the Off-Target environment.

[[Category: Training]]

Training File IO

2013-01-28T18:52:42Z

Jeffs: /* Build and Run TestApp */

== Objectives ==

This Training Module is focused on leveraging [[File_Transfer_Services | File IO]] from the context of a Test Unit executing on a target. The module covers the following:
* Accessing host based files
* Read and Writing content to / from host files

There are two test files used -- '''TestFile.cpp & TestFile.h''' -- that implement one Test Unit:
* '''TestFile'''

The Test Unit has test cases already implemented (used for reference) and has one test method that you are required to implement (called '''Exercise'''). Currently this method is empty and returns a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test File'' Test Unit only

> stride --options_file myoptions.txt --run TestFile

Loading database...
Connecting to device...
Executing...
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
--------------------------------------------------------------
Summary: 0 passed, 0 failed, 0 in progress, 1 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* Read data content ''TestFileInput.dat'' file created from the previous test method (SDK\Windows or SDK/Posix).
* Make sure to Exit the test if an error occurs when opening file(s) (i.e. use '''srEXIT_XX''')
* Take a summation of the data content using ''sut_add()''
* Validate summation with content in ''TestFileSum.dat'' file
* Add a ''NOTE'' capturing important information such as ''bytes read'', ''summation'', etc. _

* Execute only ''TestFile''

> stride --options_file myoptions.txt --run TestFile

Loading database...
Connecting to device...
Executing...
test unit "TestFile"
> 1 passed, 0 failed, 0 in progress, 0 not in use.
--------------------------------------------------------------
Summary: 1 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following if not already done:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

> stride --options_file myoptions.txt --run TestFile --space TestFile --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===
* [[File_Transfer_Services | File Transfer Services]] - important information for using file operations on host file from test code executing on the target
** Review ''srFileOpen()'' and ''srFileClose()''
** Review ''srFileRead()'' and ''srFileWrite()''
** Other references as needed

=== Samples ===
* [[File_Services_Sample | File Services Sample]] - presents a few basic examples of how to use the File Transfer Services API to interact with the host file system.

[[Category: Training]]

Training Runtime API

2013-01-28T18:52:07Z

Jeffs: /* Build and Run TestApp */

== Objectives ==

This Training Module is focused leveraging the [[Runtime_Test_Services | Runtime APIs]] in the context of writing a test. The module covers the following topics:
* How to set [[Runtime_Test_Services#srTestCaseSetStatus | test status]]
* Setting [[Runtime_Test_Services#srTestCaseSetName | test case name]] and [[Runtime_Test_Services#srTestCaseSetDescription | description]] directly via the API
* Setting [[Runtime_Test_Services#srTestSuiteSetName | test suite name]] and [[Runtime_Test_Services#srTestSuiteSetDescription | description]] directly via the API
* Dynamically creating [[Runtime_Test_Services#srTestSuiteAddSuite | test suites]] and [[Runtime_Test_Services#srTestSuiteAddCase | test cases]]

There are two new files used -- '''TestRuntime.cpp & TestRuntime.h''' --- that implement two test Units:
* '''TestRuntime_Static'''
* '''TestRuntime_Dynamic'''

All of the Test Units have test cases already implemented (used for reference) and have one test method that you are required to implement: one is called '''Exercise''' and the other is called '''dynamic_Exercise'''. Currently the exercise methods return a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Runtime APIs'' Test Units only

> stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic

Loading database...
Connecting to device...
Executing...
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
------------------------------------------------------------
Summary: 6 passed, 0 failed, 0 in progress, 2 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestRuntime_Static::Exercise'''
** Validate the ''sut_mult()'' routine using some simple data input
** Set the ''test method'' name to '''Mult'''
** Use direct Runtime APIs to:
*** Set the test case description
*** Capture logging information via comments
*** Set the status of the test case

* '''TestRuntime_Dynamic->Exercise'''
** Pass in '''5''' via command line for the number of test cases to generate
** Check using '''srEXIT_XX''' that the number of test cases has been passed in correctly
** Add a new Test Suite using '''Exercise''' for the name
** Write a loop generating a dynamic test case using ''NumberOfTestCases''
*** Test Case name shall be '''Test_n''' where "n" is the loop count as each case must have a unique name.
*** Add a description for each test case
*** Validate ''sut_foo()'' using the loop count

* note- The ''dynamic'' Test Unit is using ''CClass'' packaging

* Execute ''Test Runtime API'' Test Units only (NOTE - requires passing parameter)

> stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic(5)

Loading database...
Connecting to device...
Executing...
test unit "TestRuntime_Static"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestRuntime_Dynamic"
> 10 passed, 0 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 13 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following if not already done:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

> stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic --space TestRuntime --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===

* [[Runtime_Test_Services#C_Test_Functions | Runtime Test Services]] with special attention to the following:
** Review srTestCaseSetDescription()
** Review srTestCaseSetStatus()
** Review srTestCaseSetName()
** Review srTestCaseAddComment()
** Review srTestSuiteAddCase()
** Review srTestSuiteAddSuite()
** Review srTestSuiteSetDescription()
** Review srTestSuiteSetName()

=== Samples ===
* [[Test_Class_Sample | Test Class Sample]] - Specifically the [[Test_Class_Sample#Runtime_Services |Runtime Services]] Test Units
* [[Test_CClass_Sample | Test C Class Sample]] - Specifically the [[Test_CClass_Sample#Runtime_Services | Runtime Services]] Test Units
* [[Test_Function_List_Sample | Test Function List Sample]] - Specifically the [[Test_Function_List_Sample#Runtime_Services | Runtime Services]] Test Units

[[Category: Training]]

Training Expectations

2013-01-28T18:51:46Z

Jeffs: /* Build and Run TestApp */

== Objectives ==

This Training Module is focused on ''Test Points'' and how to validate them using ''Expectations''. The module covers the following topics:
* Presentation of a [[Expectations | validation]] technique based on ''code sequencing'' and ''state data''
* Overview of [[Source_Instrumentation_Overview#Instrumentation | source instrumentation]]
* Review of [[Test_Point_Testing_in_C/C%2B%2B | expectation tables and predicates]]
* Example use cases such as concurrent validation, using trigger conditions, etc.

There are two test files used -- '''TestExpect.cpp & TestExpect.h''' -- that implement three Test Units:
* '''TestExpect_Seq'''
* '''TestExpect_Data'''
* '''TestExpect_Misc'''

The first two Test Units have two test methods already implemented and have one method each that you are required to implement called '''Exercise'''. The third Test Unit has four test methods already implemented and also has one method to implement as an exercise. Currently the ''exercise methods'' return a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Expectations'' Test Units only

> stride --options_file myoptions.txt --run TestExpect_Seq --run TestExpect_Data --run TestExpect_Misc

Loading database...
Connecting to device...
Executing...
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
-----------------------------------------------------------
Summary: 7 passed, 1 failed, 0 in progress, 3 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestExpect_Seq::Exercise'''
** Validate '''ALL''' upper case Test Points
** Use ''Unordered'' and ''Nonstrict'' sequencing
** Use ''sut_DoSequencing(SEQ_1)'' to generate part of the sequence
** Use ''sut_start_thread(SEQ_3)'' to generate the rest of the sequence
** Add a ''NOTE_INFO(..)'' to log that the test is executing

* '''TestExpect_Data::Exercise'''
** Validate the following Test Points '''{D, G, F, H}'''
** Check that '''F''' occurs 2 times
** Use ''Unordered'' sequencing
** Write a new custom predicate that validates data for both '''D''' and '''H''' Test Points
*** Add ''NOTE_INFO(..)'' to capture content of the Test Points
*** Add extra check for '''D''' that the status is '''GOOD'''
*** Confirm that data fields '''d1''' and '''d2''' are as expected
*** Pass the expected data fields (for both Test Points) as part of the '''user''' data within the setup
*** Use ''sut_start_thread(SEQ_3)'' to generate the sequence
** Write another custom predicate for validating data for '''G'''
*** Add ''NOTE_INFO(..)'' to capture content of the Test Point
*** Validate the expected string using '''user''' data
** Add a ''NOTE_INFO(..)'' to log that the test is executing

* '''TestExpect_Misc::Exercise'''
** Validate 2 sequences using a Trigger in between
*** Sequence 1 = '''D''' '''E''' and '''A'''
*** Trigger = '''C'''
*** Sequence 2 = '''F''' and '''F''' (2 occurrences)
** Use ''ANY AT ALL'' special member with trigger
** Use ''Ordered'' and ''Strict'' sequencing
** Use ''sut_start_thread(SEQ_2)'' to generate the expected sequences

* Execute ''Test Expectations'' Test Units only

> stride --options_file myoptions.txt --run TestExpect_Seq --run TestExpect_Data --run TestExpect_Misc

Loading database...
Connecting to device...
Executing...
test unit "TestExpect_Seq"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestExpect_Data"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
test unit "TestExpect_Misc"
> 4 passed, 1 failed, 0 in progress, 0 not in use.
-----------------------------------------------------------
Summary: 10 passed, 1 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following if not already done:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

> stride --options_file myoptions.txt --run TestExpect_Seq --run TestExpect_Data --run TestExpect_Misc --space TestExpect --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===

* [[Source_Instrumentation_Overview | Instrumentation Overview]] providing high-level concepts of this validation technique
* [[Test_Point |Test Point]] Macro definition
* [[Expectations |Expectations]] definition and how to set your ''Expectations''

=== Samples ===

* [[Test_Point_Sample | Test Point Sample]] - Demonstrates simple technique to monitor and test activity occurring in another thread

[[Category: Training]]

Training Fixturing

2013-01-28T18:51:14Z

Jeffs: /* Build and Run TestApp */

== Objectives ==

This Training Module focuses on '''Fixturing''' in the context of the STRIDE Test System. For a high-level overview refer to the following wiki section on [[What_is_Unique_About_STRIDE#Fixturing | fixturing]]. This module covers the following topics:
* Startup logic at the beginning of a Test Unit
* [[Test_Unit_Pragmas#Fixturing_Pragmas | Setup]] logic for each test method
* [[Test_Unit_Pragmas#Fixturing_Pragmas | Teardown]] logic for each test method

There are two test files used -- '''TestFixture.cpp & TestFixture.h'''. These implement one Test Unit:
* '''TestFixture'''

The Test Unit has one test case already implemented (used for reference) and has '''two test methods''' that you are required to implement (called '''Exercise1''' and '''Exercise2'''). Currently the ''Exercise1'' method is empty and returns a ''NOT IN USE'' status. The ''Exercise2'' method does not yet exist.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Fixture'' Test Unit only

> stride -options_file myoptions.txt --run TestFixture

Loading database...
Connecting to device...
Executing...
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
--------------------------------------------------------------
Summary: 1 passed, 0 failed, 0 in progress, 1 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

* '''TestFixture::Exercise1'''
** Add a ''NOTE'' that will display the ''name'' of the test.
** Validate that '''Sequence2''' is stored when starting the Thread by checking that "Sequence2" equals MyString[0].
** NOTE: The method to focus on in SUT is DoSequencing()
** NOTE: m_sequence is incremented as each method goes through Teardown.

* '''TestFixture::Exercise2'''
** Add new test method declaration called ''Exercise2'' to TestFixture.h
** Add new test method implementation of ''Exercise2'' to TestFixture.cpp
** Add a ''NOTE'' that will display the name of the test
** Validate that '''Sequence3''' is persisted when starting the Thread

* Execute only ''TestFixture''

> stride --options_file myoptions.txt --run TestFixture

Loading database...
Connecting to device...
Executing...
test unit "TestFixture"
> 3 passed, 0 failed, 0 in progress, 0 not in use.
--------------------------------------------------------------
Summary: 3 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following if not already done:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

> stride --options_file myoptions.txt --run TestFixture --space TestFixture --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===

* [[Test_Unit_Pragmas#Fixturing_Pragmas | Fixturing Pragmas]] -- important information for setting up and tearing down ''things'' between each executed test method within a Test Unit
** Review the types of Test Units these pragmas are applicable for

=== Samples ===

* [[Test_Class_Sample | Test Class Sample]] - Specifically the [[Test_Class_Sample#Basic::Fixtures | Basic Fixtures]] Test Unit
* [[Test_CClass_Sample | Test C Class Sample]] - Specifically the [[Test_CClass_Sample#basic_fixtures | Basic Fixtures]] Test Unit
* [[Test_Function_List_Sample | Test Function List Sample]]- Specifically the [[Test_Function_List_Sample#basic_fixtures |Basic Fixtures]] Test Unit

[[Category: Training]]

Training Parameters

2013-01-28T18:50:43Z

Jeffs: /* Build and Run TestApp */

== Objectives ==

This Training Module is focused on '''Parameter passing''' in the context of the Test Unit. The module covers the following:
* [[Parameterized_Test_Units | Passing parameters]] via the Runner

There are two test files used -- '''TestParam.cpp & TestParam.h'''. These implement one Test Unit:
* '''TestParam'''

The Test Unit has test cases already implemented (used for reference) and has one test method that you are required to implement (called '''Exercise'''). Currently this method is empty and returns a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute ''Test Param'' Test Unit only
: Note that parameters are passed to the test unit from the host command line.

> stride --options_file myoptions.txt --run "TestParam(9,4,\"STRING1\")"

Loading database...
Connecting to device...
Executing...
test unit "TestParam"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
--------------------------------------------------------------
Summary: 1 passed, 0 failed, 0 in progress, 1 not in use.

Disconnecting from device...
Saving result file...

* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercise ===

;Exercise()
* Add a ''NOTE'' that will display the ''integer parameters'' passed via command line
* Add an Assert macro guarding against no parameters being passed (i.e. default value of 0 received)
* Call ''sut_add()'' with parameters and validate that it returns the expected value
* Call ''sut_mult()'' with parameters and validate that it returns the expected value

* Execute only ''TestParam'' '''without''' parameters

> stride --options_file myoptions.txt --run TestParam

Loading database...
Connecting to device...
Executing...
test unit "TestParam"
> 0 passed, 2 failed, 0 in progress, 0 not in use.
--------------------------------------------------------------
Summary: 0 passed, 2 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

* Execute only ''TestParam'' '''with''' parameters
> stride -O myoptions.txt --run "TestParam(9,4,\"STRING1\")"

Loading database...
Connecting to device...
Executing...
test unit "TestParam"
> 2 passed, 0 failed, 0 in progress, 0 not in use.
--------------------------------------------------------------
Summary: 2 passed, 0 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following if not already done:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

> stride --options_file myoptions.txt --run "TestParam(9,4,\"STRING1\")" --space TestParam --upload

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

== Reference ==
The following reference information is related to passing parameters to Test Units.

=== Wiki ===

* [[Parameterized_Test_Units |Parameterized Test Units]] - Describes how to supply parameters to your Test Units
* [[STRIDE_Runner | STRIDE Runner]] - Information on command line options
** Refer to "Test Unit Specification Examples" - specifically examples using parameters
** Review <tt>run[--r] arg</tt> option

=== Samples ===
* [[Test_Class_Sample | Test Class Sample]] - Specifically the [[Test_Class_Sample#Basic::Parameterized | Parameterized]] Test Unit
* [[Test_CClass_Sample | Test C Class Sample]] - Specifically the [[Test_CClass_Sample#basic_parameterized | Parameterized]] Test Unit

[[Category: Training]]

Training Test Macros

2013-01-28T18:47:47Z

Jeffs: /* Build and Run TestApp */

== Objectives ==

This Training Module focuses on the basics of writing and executing tests. It covers the following topics:
* The C++ class [[Test_Units | Test Unit packaging]] option
* How to leverage [[Test_Macros | Test Macros]]
* Using [[Test_Macros#Notes | Notes]] and [[Test_Log | Test Logs]], what is the difference?
* Creating [[Test_Documentation_in_C/C%2B%2B | Test Documentation]] via the build process
* Executing Tests using the [[STRIDE_Runner | Runner]]

The test unit is implemented in two source files: '''TestBasic.cpp''' and '''TestBasic.h'''. The comments and descriptions are contained in '''TestBasic.h'''.

The Test Unit has test cases already implemented (used for reference) and has a test method that you are required to implement (called '''Exercise'''). Currently this method is empty and returns a ''NOT IN USE'' status.

== Instructions ==

=== Build and Run TestApp ===

* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using the SDK makefile
* Invoke TestApp found in the ''/out/bin'' directory
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]

* Execute the ''Test Basic'' Test Units

> stride --options_file myoptions.txt --run TestBasic
Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
--------------------------------------------------------------
Summary: 3 passed, 1 failed, 0 in progress, 1 not in use.

Disconnecting from device...
Saving result file...

You can also review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the sample_src directory (based on the output option). By opening the xml file in a web browser the xsl is automatically applied to create html.

=== Implement Exercises ===

Now edit the training source code to complete the following exercises:

;Exercise()
* '''Assignment 1:''' Add an ''srNOTE'' to <tt>Exercise()</tt> that will add a simple message to the test report (e.g. "Exercise ...")
** ''Hint:'' You will probably want to read about [[Test Macros]]

* '''Assignment 2:''' Within <tt>Exercise()</tt> validate that <tt>sut_mult(1,1)</tt> does NOT equal <tt>sut_add(1,1)</tt>
** ''Hint:'' You will use a [[Test Macros | Test Macro]].

=== Check Results ===

* Before you rebuild <tt>TestApp.exe</tt>, you will need to shut it down by entering 'q' in its console window or closing the window directly.

* After rebuilding, invoke <tt>TestApp.exe</tt> once again.

* Execute only ''TestBasic'' using the stride runner:

> stride --options_file myoptions.txt --run TestBasic

Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 4 passed, 1 failed, 0 in progress, 0 not in use.
--------------------------------------------------------------
Summary: 4 passed, 1 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...

* If you have TestApp.xml already open in your browser, you can simply refresh (F5), to view the latest test results.

=== Run and Publish Results ===

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (<tt>myoptions.txt</tt>) with the following:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

> stride --options_file myoptions.txt --run TestBasic --space TestBasic --upload

Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 4 passed, 1 failed, 0 in progress, 0 not in use.
------------------------------------------------------------
Summary: 4 passed, 1 failed, 0 in progress, 0 not in use.

Disconnecting from device...
Saving result file...
Uploading to test space...

=== Validate Uploaded Results ===

Navigate to Test Space using your browser, then validate your results against the pre-configured baseline. For details, please see [[Training_Getting_Started#Confirming_Training_Exercise_Results | Confirming Training Exercise Results]]

Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.

You can view the results on test space by pointing your browser at '''<tt><nowiki>https://yourcompany.stridetestspace.com</nowiki></tt>'''. Log in using the credentials you supplied earlier in the options file.

== Reference ==
The following reference information is related to Test Unit basics.

=== Wiki ===

* [[Test_Units_Overview | Test Units Overview]] - Provides a general overview Test Units (i.e. writing tests in C and C++)
* [[Test_Units |Test Unit Packaging]] - Discusses the three types of packaging that can be used for Test Units (we prefer Test Classes even for c programmers)
* [[Test_Macros|Test Macros]] - Optional macros that provide shortcuts for testing assertions and automatic report annotation (you will want to use these).
* [[Test_Macros#Note_Macros|Notes]] - Used to add logging information to your '''test logic''' (automatically added to test reports)
* [[Test_Log|Test Logs]] - Used to add logging information to your '''source code''' (added to test reports if enabled)
* [[Tracing|Tracing]] - The Runner allows tracing on logs and test points.

=== Samples ===

* [[Test_Class_Sample|Test Class Sample]] - Specifically the [[Test_Class_Sample#Basic|Basic Simple]] Test Unit
* [[Test_Macros_Sample|Test Macro Sample]] - This sample covers simple uses of each of the Test Macros.

Note - Other Test Unit Samples related to packaging that are useful are the following:

* [[Test_CClass_Sample|Test C Class Sample]]
* [[Test_Function_List_Sample|Test Function List Sample]]

[[Category: Training]]

Training Getting Started

2013-01-28T18:30:38Z

Jeffs: /* Training Confirmation */

== Overview ==

Our training approach is based on a self-guided tour of the [[STRIDE_Overview#STRIDE_Testing_Features | STRIDE Testing Features]] using reference examples and assigned exercises. The set of examples and the implemented exercises will be built and executed using a standard desktop computer.

The ''software under test'', ''reference examples'', and ''exercises'' have been designed to be as '''''simple as possible while sufficiently demonstrating the topic at hand'''''. In particular, the ''software under test'' is '''very light''' on core application logic -- the focus instead is on the test code that leverages STRIDE to define and execute tests.

The user is expected to work through each of the training modules covering a specific testing feature. Once the exercises are completed (actual test cases implemented), the results are published to [[STRIDE_Overview#STRIDE_Test_Space | Test Space]] (and validated against a pre-created baseline).

The training collateral consists of the following:
# The [[STRIDE_Overview#STRIDE_Framework | STRIDE Framework]] used to implement and execute tests
# A set of specific [[#Recommend_Order | Training Modules]] that will guide you through the exercises
# [[Main_Page | ''Wiki articles'']] that will provide background material and other technical information
# Reference [[C/C%2B%2B_Samples | Samples]]
# The [[STRIDE_Off-Target_Environment | Off-Target Environment]]
# and [[STRIDE_Overview#STRIDE_Test_Space | STRIDE Test Space]]

For more details concerning STRIDE refer to the following:
* ''[[STRIDE_Overview_Video | Overview screencast]]''
* ''[[What is Unique About STRIDE | What is Unique About STRIDE]]''
* ''[[Types of Testing Supported by STRIDE | Types of Testing Supported]]''
* ''[[Frequently Asked Questions About STRIDE | Frequently Asked Questions]]''

''For questions and support [mailto:training@s2technologies.com email us]''

== Before Starting ==
Before you can start setting up your environment you need the following 3 items:
* '''STRIDE Desktop Installation Package''' (one of the following)
:<tt>STRIDE_framework-windows_4.3.yy.zip</tt> (Windows desktop)
''or''
:<tt>STRIDE_framework-linux_4.3.yy.tgz</tt> (Linux desktop)

* '''Training Source files'''
:<tt>STRIDE_training_source.zip</tt>

* '''Test Space User Account URL and Logon Credentials'''
** Test Space URL (<tt>https://YourCompany.stridetestspace.com</tt>)
** User-name
** User-password

=== Desktop Setup ===
The training requires that you install the [[Desktop_Installation | Desktop Framework]] and that the [[STRIDE_Off-Target_Environment | Off-Target Environment]] is setup correctly and verified.

For an overview of the installation steps please refer to [[Installation_Overview | this]] article.

The following steps are required:
# Install your [[Desktop_Installation | desktop Framework package]]
# Read about the [[STRIDE_Off-Target_Environment | Off-Target Environment]]
# Install [[STRIDE_Off-Target_Environment#Host_Compiler | host compiler]] for your desktop
# Use the Off-Target SDK to [[Building_an_Off-Target_Test_App | build a Test App]]
# [[Building_an_Off-Target_Test_App#Diagnostics | Run STRIDE diagnostics]] with the Test App built

=== Training Setup ===
The following source code can be found in the '''STRIDE_training_source.zip''' file:

software_under_test.c | h
TestBasic.cpp | h
TestParam.cpp | h
TestFixture.cpp | h
TestExpect.cpp | h
TestRuntime.cpp | h
TestFile.cpp | h
TestDouble.cpp | h

The ''software under test'' is contained in the file '''software_under_test.c | h'''. All of the public functions use '''sut_''' as a prefix. All training modules test against this file. Although the test examples are contained in C++ files, most of the test logic is written in standard C.

Once the files are obtained the next step is to rebuild the TestApp using the training source code. Move '''all''' of the files into the directory <tt>%STRIDE_DIR%\SDK\Windows\sample_src</tt> (or <tt>$STRIDE_DIR/SDK/Posix/sample_src</tt> for Linux) and follow the instructions for [[Building_an_Off-Target_Test_App | Building a TestApp]].

Once built, list all [[Test_Units | Test Units]] within the generated database file using the following command:

'''Windows'''
> stride --database="%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb" --list

'''Linux'''
> stride --database="$STRIDE_DIR/SDK/Posix/out/TestApp.sidb" --list

The following remoted Functions and Test Units should be displayed:

Functions
sut_strcpy(char const * input, char * output) : void
strlen(char const * _Str) : size_t
Test Units
TestBasic()
TestDouble_Reference()
TestDouble_Definition()
TestExpect_Seq()
TestExpect_Data()
TestExpect_Misc()
TestFile()
TestFixture()
TestParam(int data1, int data2, char * szString)
TestRuntime_Static()
TestRuntime_Dynamic(int NumberOfTestCases)

=== Run Training Tests ===
Here we will run all tests in the <tt>TestApp.sidb</tt> database.<ref>Note that the S2 diagnostic tests are treated separately, and are not run unless the <tt>--diagnostics</tt> option is specified to <tt>stride</tt>.</ref>

* If you haven't done so already, [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using the SDK makefile
* Invoke TestApp found in the ''/out/bin'' directory
* Create an [[Stride_Runner#Options | option file]] (<tt>myoptions.txt</tt>) that includes the following content (SDK\Windows or SDK/Posix).

'''Windows'''
##### Command Line Options ######
--device "TCP:localhost:8000"
--database "%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb"
--output "%STRIDE_DIR%\SDK\Windows\sample_src\TestApp.xml"
--log_level all

'''Linux'''
##### Command Line Options #####
--device "TCP:localhost:8000"
--database "$STRIDE_DIR/SDK/Posix/out/TestApp.sidb"
--output "$STRIDE_DIR/SDK/Posix/sample_src/TestApp.xml"
--log_level all

A couple of things to note:
* If you setup an [[STRIDE_Runner#Environment_Variables | environment variable]] for the '''device''' option then it is not required in the option file. Note: Command line options override environment variables.
* ''stride --help'' provides [[STRIDE_Runner#Options | options information]]

If you haven't done so already, start <tt>TestApp</tt> running in a separate console window.

Now run stride as shown below and verify summary results (starting from the <tt>SDK\Windows</tt> or <tt>SDK/Posix</tt> directory):

> stride --options_file myoptions.txt --run "*"

The output should look like this:
<pre>
Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestParam"
> 0 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------------------
Summary: 21 passed, 3 failed, 0 in progress, 11 not in use.

Disconnecting from device...
Saving result file...

</pre>

=== Interpreting Results ===
Open '''TestApp.xml''' in your browser; this file is created in the directory from which you ran '''stride''' (or the directory via the <tt>--output</tt> command line option). If you were connected to the Internet when you ran the tests, the TestApp.xsl file is also generated in the directory. By opening TestApp.xml in a web browser, the xsl is automatically applied to create html in the browser. If you use Google Chrome, please see [[Browser Compatibility]].

If you're interested in the details of the tests, please see the test documentation contained in the test report.

=== Test Space Access ===

As part of the training users implement exercises (test cases). As each exercise is completed, the results are expected to be uploaded to Test Space. Accessing Test Space (uploading, viewing, etc.) requires an user-name and password. Before working on a training module please confirm that your user account has been set up by [[Test_Space_Setup | logging in]].

Test Space will have expected results using [[Creating_And_Using_Baselines | baselines]]. This is used to automatically verify if the exercises have been implemented correctly (at least to some degree). For capturing test results a '''Training''' project has been created with the following '''Spaces''':

Sandbox - Results for training setup
TestBasic - Basics training results: TestBasic.cpp | h
TestParam - Parameters training results: TestParam.cpp | h
TestFixture - Fixturing training results: TestFixture.cpp | h
TestExpect - Expectations training results: TestExpect.cpp | h
TestRuntime - Runtime API training results: TestRuntime.cpp | h
TestFile - File IO training results: TestFile.cpp | h
TestDouble - Doubling training results: TestDouble.cpp | h

To publish your results using the [[STRIDE_Runner | STRIDE Runner]] the following command-line options should be used:

--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--space ''MODULENAME''
--name ''YOURNAME''

Notes:
* Concerning the ''MODULENAME'' option: use the name that corresponds to the training module currently underway (e.g. TestBasic, TestParam, etc.)
* ''YOURNAME'' should be set to your name (i.e. JohnD); omit spaces from this string
* If you access the Internet via an HTTP proxy please read [[STRIDE_Runner#Using_a_Proxy | '''this article''']]

To make it easier for now we recommend that you update your existing option file (<tt>myoptions.txt</tt>) with the following:

#### Test Space options (partial) #####
#### Note - make sure to change username, etc. ####
--testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
--project Training
--name YOURNAME

=== Publish Test Space Results ===

To complete the setup publish your results to Test Space.

> stride --options_file myoptions.txt --run "*" --space Sandbox --upload

Loading database...
Connecting to device...
Executing...
test unit "TestBasic"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Definition"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestDouble_Reference"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Data"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Misc"
> 3 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestExpect_Seq"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFile"
> 0 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestFixture"
> 1 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestParam"
> 0 passed, 1 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Dynamic"
> 4 passed, 0 failed, 0 in progress, 1 not in use.
test unit "TestRuntime_Static"
> 2 passed, 0 failed, 0 in progress, 1 not in use.
---------------------------------------------------------------------
Summary: 21 passed, 3 failed, 0 in progress, 11 not in use.
Disconnecting from device...
Saving result file...
Uploading to test space...

=== Confirming Setup ===

After you have run and uploaded the results of the training setup, you should confirm the correctness of your work. The training setup results will end up with a mix of passes and failures, so we use a [[Creating_And_Using_Baselines | Test Space baseline]] to get an aggregate comparison between expected and actual results.

To validate your setup:
# Navigate to Sandbox set in Test Space and look under the column labeled '''''SEQUENTIAL COMPARISON'''''
# Confirm that '''''none''''' is indicated for each of the sub-column. If so, you have completed the setup and are ready to move onto the training modules.

[[image:STRIDE_results_012813.JPG|none|600px| Test Space baseline example]]

== Training ==
At this point you should be ready to start the actual training. There are '''7 separate training modules''' and we recommend the following order:

===Introductory===
* [[Training_Basics | '''Basics''']] - Covers basics of implementing and executing test cases
* [[Training_Parameters |'''Parameters''']] - How to pass parameters to a test
* [[Training_Fixturing | '''Fixturing''']] - Leveraging setup and teardown featuring

===Advanced===
* [[Training_Expectations | '''Expectations''']] - Validating code sequencing along with state data
* [[Training_Runtime_API | '''Runtime API''']] - Using the runtime services to dynamically create test suits / cases
* [[Training_File_IO | '''File IO''']] - Reading and writing files existing on the host
* [[Training_Doubling | '''Doubling''']] - Replacing a dependency with a stub, fake, or mock

== Training Confirmation ==

As you run and upload each test unit containing your worked training exercises, you should confirm the correctness of your work.

The correctly worked training test units will end up with a mix of passes and failures, so we use a [[Creating_And_Using_Baselines | Test Space baseline]] to get an aggregate comparison between expected and actual results

To validate your work:
# Navigate to your result set in Test Space and look under the column labeled '''''SEQUENTIAL COMPARISON'''''
# Confirm that '''''none''''' is indicated for each of the sub-column.

[[Category: Training]]