Studio:Scripting Overview: Difference between revisions

From STRIDE Wiki
Jump to navigation Jump to search
No edit summary
m (Text replace - 'Category:Scripting' to 'Category:Studio:Scripting')
 
(17 intermediate revisions by 2 users not shown)
Line 1: Line 1:
=The Role of Scripting in STRIDE=
=The Role of Scripting in STRIDE Studio=


Using scripts to drive test scenarios or to simulate missing components has traditionally been difficult, if not impossible, when the unit under test is located on the target platform. Moreover, scripting languages and the C language do not necessarily mix well. There are data type mismatches, etc. *** Because of these difficulties, developers rarely use scripts for testing and integration, despite the potential benefits.
Scripting (running programs implemented in a supported scripting language) is fundamental to performing tests within [[STRIDE Studio]].


* You don't have to recompile your target when test changes are made in the script
STRIDE makes it possible to use scripts written in either JScript or Perl<ref>S2 Technologies provides full support for Microsoft JScript and Active State Perl, however, other languages that support COM Automation have been used successfully. Contact S2 Technologies for more information.</ref>, to perform test-related tasks.
* Scripting is very flexible
* Wide spectrum of personnel are able to write scripts


{{ref|ScriptCompat}}
Typically, scripts are used for the following tasks:


STRIDE makes it possible to use scripts written in either JScript or Perl {{note|ScriptCompat}} to test your software units regardless of whether they are located on the host, target, or both. There are two common use cases for using scripts to integrate and test:
* Manipulating/controlling STRIDE Studio
* Controlling Intercept Module code generation
* Driving tests
* Simulating missing software units
* Controlling the content of reports and their generation


*  To drive the testing of a unit or units by setting parameter values, calling functions, and receiving return values (or sending/receiving messages)
STRIDE extends the intrinsic capabilities of your scripting language by providing a set of [http://en.wikipedia.org/wiki/OLE_Automation COM Automation] compatible objects and injecting an instance of each into your running script.


*  To mock (simulate) a missing function or other component so an interdependent unit can be tested.
{| class="wikitable" border="1"
|+ ''Object instances that are injected into STRIDE Studio scripts''
! Injected Object !! Perl Variable Name !! JScript Variable Name
|-
| '''[[STRIDE_Studio|studio]]''' || <code>$main::studio</code> || <code>studio</code>
|-
| '''[[AutoScript|ascript]]''' || <code>$main::ascript</code> || <code>ascript</code>
|-
| '''[[Reporter#reporter.Suites.Item|testSuite]]''' || <code>$main::testSuite</code> || <code>testSuite</code>
|-
| '''[[Reporter|reporter]]'''<br><div style="font-size:80%">(rarely used directly in STRIDE Studio)</div>||<code>$main::reporter</code> || <code>reporter</code>
|}


To illustrate these two use cases, consider a hypothetical embedded
Each object presents an object model through which you can perform needed operations and get/set desired properties. For details of each object model, click on the links above.


application containing three functions: A, B, and C. A calls B and B calls
=Controling STRIDE Studio from a Script=
The '''studio''' instance injected into each script has been initialized to reflect the current settings and properties applied to the currently loaded workspace.


C as shown on the left in the diagram below. Suppose that you developed
Note: You can also instantiate a '''studio''' object from an external script running outside of STRIDE Studio. In this case, the initial state does not reflect a workspace until one is explicity loaded or created.


function B and you are ready to test it, but function C is not yet completed.
The types of things controlled in Studio


One solution, which would allow you to continue testing regardless of
* <u>Workspace-related</u>
** Control the composition of a workspace
*** Add/remove files and folders
** Control workspace settings
*** Set folder/file properties
*** Set compiler properties


the availability of function C, is to use scripts as shown on the right
* <u>Compiler-related</u>
** Compile the workspace
** Control intercept module settings and generation


in the diagram below. One script simulates the missing function C (Use
* <u>Test-run related</u>
** Run scripts
** Control optional trace views


Case #2) while the other drives the testing of function B (Use Case #1).


=Calling Target C Functions from a Script=
The set of captured interfaces is accessed via the injected '''ascript''' object. `This object connects scripts to the workspace's database and to the STRIDE runtime. The injected ascript instance has been initialized to reflect the current state of the currently loaded workspace's database.


Note: You can also instantiate an '''ascript''' object from an external script running outside of STRIDE Studio. In this case you must provide information by calling the new instance's Initialize() method with the required parameters.


[[Image:Overview_Scripting.png]]
the '''ascript''' object provides the means by which:
* on-target C functions can be called and executed
* on-target functions can be mocked (simulated) by script code running on the host


As implied in the diagram, the fact that scripts replace functions A
In either case, function parameter values can be set by the function's caller (known as the ''User'') and read by the function implementation (known as the ''Owner''). Similarly, function return values can be set by the function implementation (''Owner'') and read by the function's caller (''User'') upon return from the called function.


and C does not change the interfaces with function B; via STRIDE, interfaces
==A Typical Scenario==
In a typical testing scenario, the target system we are testing is up and running and the communication link between the host PC system and the target is operational. By default, the registered Owner of each function is the target implementation, therefore any calls we make to the function will be routed to the target by the host runtime.


defined in your embedded application are available at the source level
The steps we would take in a script to call the remote function are:
# Get the desired Function User object instance from the ascript Functions collection
# Set the function's input parameter values
# Call the function (can be done synchronously or asynchronously)
# Upon the function's return, compare the return value against an expected value


in your favorite scripting language. In addition, function B could be
In most scripts, the first step would be done once, and the remaining steps would be performed in a loop, presenting a different set of input parameter values each iteration.


on the target application, whereas the scripts replacing functions A and
Script syntax is available for navigating complex payloads comprising structures, unions, arrays and pointers.


C are running on the host.
For additional details and example scripts, see '''[[Using Scripts to Automate Software Testing]]'''.


<references/>
=Reporting from a Script=
Reporting is performed using the injected '''testSuite''' object instance (the injected '''reporter''' object is also used occasionally).


[[Category:Scripting]]
STRIDE Studio automatically creates a hierarchy of test suites within the '''reporter''' instance that mirrors the hierarchy of folders under the workspace ''Script Files'' folder. The injected '''testSuite''' object is the instantiated '''Suite''' object in the '''reporter''' hierarchy that corresponds to the parent folder of the running script.
 
To create report output from a script, you do the following:
# create a new '''Test''' object instance as a child of the '''testSuite'''
# add comments and/or annotations to the '''Test''' object as needed
# set the Test status to "PASS" or "FAIL"
 
<u>Note:</u> You are not limited to a single Test instance in a script, although a common pattern is to create one script per test case.
 
An overall report is created automatically by STRIDE Studio after all scripts in the workspace have run. The report's native format is XML, which is commonly transformed to locally-viewable HTML using an XSL script when test are run interactively, or uploaded to a server and aggregated when test are run unattended.
 
=Mocking Target C Functions Using a Script=
Mocking (simulating) a target C function comprises two steps:
 
# Implement the required functionality in a script
# Register the script as the function's ''Owner''
 
Registering as the function ''Owner'' causes the STRIDE runtime to route all calls to the function to the script implementation.
 
It is also possible to switch between your target function implementation and your script implementation at runtime.
 
For additional details and example scripts, see '''[[Using Scripts to Simulate Missing Software Units]]'''.
 
=Notes=
<div class="references-small"><references/></div>
 
[[Category:Studio:Scripting]]

Latest revision as of 23:45, 20 August 2009

The Role of Scripting in STRIDE Studio

Scripting (running programs implemented in a supported scripting language) is fundamental to performing tests within STRIDE Studio.

STRIDE makes it possible to use scripts written in either JScript or Perl[1], to perform test-related tasks.

Typically, scripts are used for the following tasks:

  • Manipulating/controlling STRIDE Studio
  • Controlling Intercept Module code generation
  • Driving tests
  • Simulating missing software units
  • Controlling the content of reports and their generation

STRIDE extends the intrinsic capabilities of your scripting language by providing a set of COM Automation compatible objects and injecting an instance of each into your running script.

Object instances that are injected into STRIDE Studio scripts
Injected Object Perl Variable Name JScript Variable Name
studio $main::studio studio
ascript $main::ascript ascript
testSuite $main::testSuite testSuite
reporter
(rarely used directly in STRIDE Studio)
$main::reporter reporter

Each object presents an object model through which you can perform needed operations and get/set desired properties. For details of each object model, click on the links above.

Controling STRIDE Studio from a Script

The studio instance injected into each script has been initialized to reflect the current settings and properties applied to the currently loaded workspace.

Note: You can also instantiate a studio object from an external script running outside of STRIDE Studio. In this case, the initial state does not reflect a workspace until one is explicity loaded or created.

The types of things controlled in Studio

  • Workspace-related
    • Control the composition of a workspace
      • Add/remove files and folders
    • Control workspace settings
      • Set folder/file properties
      • Set compiler properties
  • Compiler-related
    • Compile the workspace
    • Control intercept module settings and generation
  • Test-run related
    • Run scripts
    • Control optional trace views


Calling Target C Functions from a Script

The set of captured interfaces is accessed via the injected ascript object. `This object connects scripts to the workspace's database and to the STRIDE runtime. The injected ascript instance has been initialized to reflect the current state of the currently loaded workspace's database.

Note: You can also instantiate an ascript object from an external script running outside of STRIDE Studio. In this case you must provide information by calling the new instance's Initialize() method with the required parameters.

the ascript object provides the means by which:

  • on-target C functions can be called and executed
  • on-target functions can be mocked (simulated) by script code running on the host

In either case, function parameter values can be set by the function's caller (known as the User) and read by the function implementation (known as the Owner). Similarly, function return values can be set by the function implementation (Owner) and read by the function's caller (User) upon return from the called function.

A Typical Scenario

In a typical testing scenario, the target system we are testing is up and running and the communication link between the host PC system and the target is operational. By default, the registered Owner of each function is the target implementation, therefore any calls we make to the function will be routed to the target by the host runtime.

The steps we would take in a script to call the remote function are:

  1. Get the desired Function User object instance from the ascript Functions collection
  2. Set the function's input parameter values
  3. Call the function (can be done synchronously or asynchronously)
  4. Upon the function's return, compare the return value against an expected value

In most scripts, the first step would be done once, and the remaining steps would be performed in a loop, presenting a different set of input parameter values each iteration.

Script syntax is available for navigating complex payloads comprising structures, unions, arrays and pointers.

For additional details and example scripts, see Using Scripts to Automate Software Testing.

Reporting from a Script

Reporting is performed using the injected testSuite object instance (the injected reporter object is also used occasionally).

STRIDE Studio automatically creates a hierarchy of test suites within the reporter instance that mirrors the hierarchy of folders under the workspace Script Files folder. The injected testSuite object is the instantiated Suite object in the reporter hierarchy that corresponds to the parent folder of the running script.

To create report output from a script, you do the following:

  1. create a new Test object instance as a child of the testSuite
  2. add comments and/or annotations to the Test object as needed
  3. set the Test status to "PASS" or "FAIL"

Note: You are not limited to a single Test instance in a script, although a common pattern is to create one script per test case.

An overall report is created automatically by STRIDE Studio after all scripts in the workspace have run. The report's native format is XML, which is commonly transformed to locally-viewable HTML using an XSL script when test are run interactively, or uploaded to a server and aggregated when test are run unattended.

Mocking Target C Functions Using a Script

Mocking (simulating) a target C function comprises two steps:

  1. Implement the required functionality in a script
  2. Register the script as the function's Owner

Registering as the function Owner causes the STRIDE runtime to route all calls to the function to the script implementation.

It is also possible to switch between your target function implementation and your script implementation at runtime.

For additional details and example scripts, see Using Scripts to Simulate Missing Software Units.

Notes

  1. S2 Technologies provides full support for Microsoft JScript and Active State Perl, however, other languages that support COM Automation have been used successfully. Contact S2 Technologies for more information.