Running and Publishing the Expectations Sample: Difference between revisions
Jump to navigation
Jump to search
(Created page with '== Building the TargetApp to Include Instrumented Source Under Test == In this step, will will add pre-instrumented examples source that provide an overview of STRIDE instrumenta…') |
|||
| (30 intermediate revisions by 4 users not shown) | |||
| Line 1: | Line 1: | ||
== Building | == Building Instrumented Source Under Test == | ||
In this step, | In this step, we will add pre-instrumented example source that provide an overview of STRIDE instrumentation techniques. The [[Expectations Sample]] sources are described in the linked article. | ||
To begin, be sure that TestApp is not running, then copy the <tt>.c</tt> and <tt>.h</tt> files found in <tt>Samples/test_in_scrips/Expectations</tt> to <tt>SDK/Windows/sample_src</tt> (or <tt>SDK/Posix/sample_src</tt> for Linux). | |||
Once the files have been copied to <tt>sample_src</tt>, simply build TestApp as described in [[Building_an_Off-Target_Test_App | here]]. | |||
== Running Expectations Sample == | |||
Here we will run the test module that is provided in this example . | |||
# Run the build above TestApp in a console window. | |||
# Invoke <tt>stride</tt> in a separate console window (different from the running TestApp) -- as shown below and verify Summary results. | |||
Here are the command line parameters that we will submit to <tt>stride</tt> | |||
<pre> | |||
--run=%STRIDE_DIR%\Samples\test_in_script\Expectations\s2_expectations_testmodule.pm | |||
--database ./out/TestApp.sidb | |||
--device TCP:localhost:8000 | |||
</pre> | |||
The command line arguments are very long, so we'll want to create a text file named ''RunSample.txt'' (for example) in the <tt>SDK\Windows</tt> (or <tt>SDK/Posix</tt> for Linux) directory as an option file to submit to <tt>stride</tt>. | |||
If you haven't done so already, start <tt>TestApp</tt> running in a separate console window. | |||
Now run stride as follows (starting from the <tt>SDK\Windows</tt> or <tt>SDK/Posix</tt> directory): | |||
<source lang="bash"> | <source lang="bash"> | ||
stride -O RunSample.txt | |||
</source> | </source> | ||
The output should look like this: | |||
<pre> | |||
Loading database... | |||
Connecting to device... | |||
Executing... | |||
test module "C:\stride\Samples\test_in_script\Expectations\s2_expectations_testmodule.pm"... | |||
> 12 passed, 2 failed, 0 in progress, 0 not in use. | |||
--------------------------------------------------------------------- | |||
Summary: 12 passed, 2 failed, 0 in progress, 0 not in use. | |||
Disconnecting from device... | |||
Saving result file... | |||
</pre> | |||
== Publishing Results to Test Space == | |||
To automatically publish test results add the following options to RunSample.txt: | |||
<pre> | <pre> | ||
--upload | --upload | ||
--testspace | --testspace USER:PASS@mycompany.stridetestspace.com | ||
--project Sandbox | --project Sandbox | ||
--space | --space Samples | ||
</pre> | </pre> | ||
| Line 79: | Line 61: | ||
<source lang="bash"> | <source lang="bash"> | ||
stride -O | stride -O RunSample.txt | ||
</source> | </source> | ||
| Line 87: | Line 69: | ||
Loading database... | Loading database... | ||
Connecting to device... | Connecting to device... | ||
Executing... | |||
Executing | test module "C:\stride\Samples\test_in_script\Expectations\s2_expectations_testmodule.pm"... | ||
> 12 passed, 2 failed, 0 in progress, 0 not in use. | |||
> | |||
--------------------------------------------------------------------- | --------------------------------------------------------------------- | ||
Summary: | Summary: 12 passed, 2 failed, 0 in progress, 0 not in use. | ||
Disconnecting from device... | Disconnecting from device... | ||
Saving result file... | Saving result file... | ||
Uploading to test space... | |||
</pre> | </pre> | ||
A few things to note: | |||
* the run parameter specifies the test module to execute | |||
* You will have to replace ''USER:PASS'' with your S2-assigned TestSpace user name and password | |||
* You will have to replace ''mycompany'' with your S2-assigned subdomain name | |||
* The project "Sandbox" and TestSpace "Expectations" should already be created within your company STRIDE TestSpace | |||
=== Viewing Results in Test Space === | === Viewing Results in Test Space === | ||
| Line 103: | Line 93: | ||
At the top of the next page, click on the All Projects link to view the status of existing projects. Here you should see the Sandbox project listed, with its contained TestSpace ''Expectations'' shown. | At the top of the next page, click on the All Projects link to view the status of existing projects. Here you should see the Sandbox project listed, with its contained TestSpace ''Expectations'' shown. | ||
Clicking the ''Expectations'' link will present you with the ''Expectations'' TestSpace page. From the top-line results at the bottom of the page you can drill down into the Sequence_1 results to see the test details. | Clicking the ''Expectations'' link will present you with the ''Expectations'' TestSpace page. From the top-line results at the bottom of the page you can drill down into the Sequence_1 (assuming this is the first time publishing) results to see the test details. | ||
=== Analyzing the Results === | === Analyzing the Results === | ||
At this point, we recommend that you take some time to review the techniques used in the | At this point, we recommend that you take some time to review the techniques used in the Expectations sample tests and correlate the results shown in Test Space with the various STRIDE constructs in the sample source. The article [[Expectations Sample]] describes the tests in detail. | ||
<hr/> | <hr/> | ||
<references/> | <references/> | ||
[[Category: | |||
[[Category:Installation]] | |||
Latest revision as of 20:53, 20 February 2013
Building Instrumented Source Under Test
In this step, we will add pre-instrumented example source that provide an overview of STRIDE instrumentation techniques. The Expectations Sample sources are described in the linked article.
To begin, be sure that TestApp is not running, then copy the .c and .h files found in Samples/test_in_scrips/Expectations to SDK/Windows/sample_src (or SDK/Posix/sample_src for Linux).
Once the files have been copied to sample_src, simply build TestApp as described in here.
Running Expectations Sample
Here we will run the test module that is provided in this example .
- Run the build above TestApp in a console window.
- Invoke stride in a separate console window (different from the running TestApp) -- as shown below and verify Summary results.
Here are the command line parameters that we will submit to stride
--run=%STRIDE_DIR%\Samples\test_in_script\Expectations\s2_expectations_testmodule.pm --database ./out/TestApp.sidb --device TCP:localhost:8000
The command line arguments are very long, so we'll want to create a text file named RunSample.txt (for example) in the SDK\Windows (or SDK/Posix for Linux) directory as an option file to submit to stride.
If you haven't done so already, start TestApp running in a separate console window.
Now run stride as follows (starting from the SDK\Windows or SDK/Posix directory):
stride -O RunSample.txt
The output should look like this:
Loading database...
Connecting to device...
Executing...
test module "C:\stride\Samples\test_in_script\Expectations\s2_expectations_testmodule.pm"...
> 12 passed, 2 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 12 passed, 2 failed, 0 in progress, 0 not in use.
Disconnecting from device...
Saving result file...
Publishing Results to Test Space
To automatically publish test results add the following options to RunSample.txt:
--upload --testspace USER:PASS@mycompany.stridetestspace.com --project Sandbox --space Samples
Now run stride as follows:
stride -O RunSample.txt
The output should look like this:
Loading database...
Connecting to device...
Executing...
test module "C:\stride\Samples\test_in_script\Expectations\s2_expectations_testmodule.pm"...
> 12 passed, 2 failed, 0 in progress, 0 not in use.
---------------------------------------------------------------------
Summary: 12 passed, 2 failed, 0 in progress, 0 not in use.
Disconnecting from device...
Saving result file...
Uploading to test space...
A few things to note:
- the run parameter specifies the test module to execute
- You will have to replace USER:PASS with your S2-assigned TestSpace user name and password
- You will have to replace mycompany with your S2-assigned subdomain name
- The project "Sandbox" and TestSpace "Expectations" should already be created within your company STRIDE TestSpace
Viewing Results in Test Space
First navigate to the S2-provided TestSpace with your browser. The URL has the form: https://companyname.stridetestspace.com. On the page that is presented, enter your login credentials.
At the top of the next page, click on the All Projects link to view the status of existing projects. Here you should see the Sandbox project listed, with its contained TestSpace Expectations shown.
Clicking the Expectations link will present you with the Expectations TestSpace page. From the top-line results at the bottom of the page you can drill down into the Sequence_1 (assuming this is the first time publishing) results to see the test details.
Analyzing the Results
At this point, we recommend that you take some time to review the techniques used in the Expectations sample tests and correlate the results shown in Test Space with the various STRIDE constructs in the sample source. The article Expectations Sample describes the tests in detail.