Studio:Scripting Overview: Difference between revisions

From STRIDE Wiki
Jump to navigation Jump to search
(New page: STRIDE affords you a number of different options for automating your testing, including datasets and scripts. Each option provides a unique set of testing capabilities. When conditional...)
 
m (Text replace - 'Category:Scripting' to 'Category:Studio:Scripting')
 
(18 intermediate revisions by 2 users not shown)
Line 1: Line 1:
=The Role of Scripting in STRIDE Studio=


STRIDE affords you a number of different options for automating your
Scripting (running programs implemented in a supported scripting language) is fundamental to performing tests within [[STRIDE Studio]].


testing, including datasets and scripts. Each option provides a unique
STRIDE makes it possible to use scripts written in either JScript or Perl<ref>S2 Technologies provides full support for Microsoft JScript and Active State Perl, however, other languages that support COM Automation have been used successfully. Contact S2 Technologies for more information.</ref>, to perform test-related tasks.


set of testing capabilities. When conditional logic, event processing,
Typically, scripts are used for the following tasks:


user intervention, or other complex operations are required, scripts may
* Manipulating/controlling STRIDE Studio
* Controlling Intercept Module code generation
* Driving tests
* Simulating missing software units
* Controlling the content of reports and their generation


be the most appropriate option.
STRIDE extends the intrinsic capabilities of your scripting language by providing a set of [http://en.wikipedia.org/wiki/OLE_Automation COM Automation] compatible objects and injecting an instance of each into your running script.


Using scripts to drive test scenarios or to simulate missing components
{| class="wikitable" border="1"
|+ ''Object instances that are injected into STRIDE Studio scripts''
! Injected Object !! Perl Variable Name !! JScript Variable Name
|-
| '''[[STRIDE_Studio|studio]]''' || <code>$main::studio</code> || <code>studio</code>
|-
| '''[[AutoScript|ascript]]''' || <code>$main::ascript</code> || <code>ascript</code>
|-
| '''[[Reporter#reporter.Suites.Item|testSuite]]''' || <code>$main::testSuite</code> || <code>testSuite</code>
|-
| '''[[Reporter|reporter]]'''<br><div style="font-size:80%">(rarely used directly in STRIDE Studio)</div>||<code>$main::reporter</code> || <code>reporter</code>
|}


has traditionally been difficult, if not impossible, when the unit under
Each object presents an object model through which you can perform needed operations and get/set desired properties. For details of each object model, click on the links above.


test is located on the target platform. Moreover, scripting languages
=Controling STRIDE Studio from a Script=
The '''studio''' instance injected into each script has been initialized to reflect the current settings and properties applied to the currently loaded workspace.


and the C language do not necessarily mix well. Because of these difficulties,
Note: You can also instantiate a '''studio''' object from an external script running outside of STRIDE Studio. In this case, the initial state does not reflect a workspace until one is explicity loaded or created.


developers rarely use scripts for testing and integration, despite the
The types of things controlled in Studio


potential benefits.
* <u>Workspace-related</u>
** Control the composition of a workspace
*** Add/remove files and folders
** Control workspace settings
*** Set folder/file properties
*** Set compiler properties


STRIDE makes it possible to use scripts written in any scripting language
* <u>Compiler-related</u>
** Compile the workspace
** Control intercept module settings and generation


to test your software units regardless of whether they are located on
* <u>Test-run related</u>
** Run scripts
** Control optional trace views


the host, target, or both. There are two common use cases for using scripts


to integrate and test:
=Calling Target C Functions from a Script=
The set of captured interfaces is accessed via the injected '''ascript''' object. `This object connects scripts to the workspace's database and to the STRIDE runtime. The injected ascript instance has been initialized to reflect the current state of the currently loaded workspace's database.


*  To drive the testing of a unit or units by calling functions
Note: You can also instantiate an '''ascript''' object from an external script running outside of STRIDE Studio. In this case you must provide information by calling the new instance's Initialize() method with the required parameters.


and/or sending commands and receiving messages.
the '''ascript''' object provides the means by which:
* To simulate a missing function or other component so an interdependent
* on-target C functions can be called and executed
unit can be tested.
* on-target functions can be mocked (simulated) by script code running on the host


To illustrate these two use cases, consider a hypothetical embedded
In either case, function parameter values can be set by the function's caller (known as the ''User'') and read by the function implementation (known as the ''Owner''). Similarly, function return values can be set by the function implementation (''Owner'') and read by the function's caller (''User'') upon return from the called function.


application containing three functions: A, B, and C. A calls B and B calls
==A Typical Scenario==
In a typical testing scenario, the target system we are testing is up and running and the communication link between the host PC system and the target is operational. By default, the registered Owner of each function is the target implementation, therefore any calls we make to the function will be routed to the target by the host runtime.


C as shown on the left in the diagram below. Suppose that you developed
The steps we would take in a script to call the remote function are:
# Get the desired Function User object instance from the ascript Functions collection
# Set the function's input parameter values
# Call the function (can be done synchronously or asynchronously)
# Upon the function's return, compare the return value against an expected value


function B and you are ready to test it, but function C is not yet completed.
In most scripts, the first step would be done once, and the remaining steps would be performed in a loop, presenting a different set of input parameter values each iteration.


One solution, which would allow you to continue testing regardless of
Script syntax is available for navigating complex payloads comprising structures, unions, arrays and pointers.


the availability of function C, is to use scripts as shown on the right
For additional details and example scripts, see '''[[Using Scripts to Automate Software Testing]]'''.


in the diagram below. One script simulates the missing function C (Use
=Reporting from a Script=
Reporting is performed using the injected '''testSuite''' object instance (the injected '''reporter''' object is also used occasionally).


Case #2) while the other drives the testing of function B (Use Case #1).
STRIDE Studio automatically creates a hierarchy of test suites within the '''reporter''' instance that mirrors the hierarchy of folders under the workspace ''Script Files'' folder. The injected '''testSuite''' object is the instantiated '''Suite''' object in the '''reporter''' hierarchy that corresponds to the parent folder of the running script.


To create report output from a script, you do the following:
# create a new '''Test''' object instance as a child of the '''testSuite'''
# add comments and/or annotations to the '''Test''' object as needed
# set the Test status to "PASS" or "FAIL"


<u>Note:</u> You are not limited to a single Test instance in a script, although a common pattern is to create one script per test case.


[[Image:Overview_Scripting.png]]
An overall report is created automatically by STRIDE Studio after all scripts in the workspace have run. The report's native format is XML, which is commonly transformed to locally-viewable HTML using an XSL script when test are run interactively, or uploaded to a server and aggregated when test are run unattended.


As implied in the diagram, the fact that scripts replace functions A
=Mocking Target C Functions Using a Script=
Mocking (simulating) a target C function comprises two steps:


and C does not change the interfaces with function B; via STRIDE, interfaces
# Implement the required functionality in a script
# Register the script as the function's ''Owner''


defined in your embedded application are available at the source level
Registering as the function ''Owner'' causes the STRIDE runtime to route all calls to the function to the script implementation.


in your favorite scripting language. In addition, function B could be
It is also possible to switch between your target function implementation and your script implementation at runtime.


on the target application, whereas the scripts replacing functions A and
For additional details and example scripts, see '''[[Using Scripts to Simulate Missing Software Units]]'''.


C are running on the host.
=Notes=
<div class="references-small"><references/></div>
 
[[Category:Studio:Scripting]]

Latest revision as of 23:45, 20 August 2009

The Role of Scripting in STRIDE Studio

Scripting (running programs implemented in a supported scripting language) is fundamental to performing tests within STRIDE Studio.

STRIDE makes it possible to use scripts written in either JScript or Perl[1], to perform test-related tasks.

Typically, scripts are used for the following tasks:

  • Manipulating/controlling STRIDE Studio
  • Controlling Intercept Module code generation
  • Driving tests
  • Simulating missing software units
  • Controlling the content of reports and their generation

STRIDE extends the intrinsic capabilities of your scripting language by providing a set of COM Automation compatible objects and injecting an instance of each into your running script.

Object instances that are injected into STRIDE Studio scripts
Injected Object Perl Variable Name JScript Variable Name
studio $main::studio studio
ascript $main::ascript ascript
testSuite $main::testSuite testSuite
reporter
(rarely used directly in STRIDE Studio)
$main::reporter reporter

Each object presents an object model through which you can perform needed operations and get/set desired properties. For details of each object model, click on the links above.

Controling STRIDE Studio from a Script

The studio instance injected into each script has been initialized to reflect the current settings and properties applied to the currently loaded workspace.

Note: You can also instantiate a studio object from an external script running outside of STRIDE Studio. In this case, the initial state does not reflect a workspace until one is explicity loaded or created.

The types of things controlled in Studio

  • Workspace-related
    • Control the composition of a workspace
      • Add/remove files and folders
    • Control workspace settings
      • Set folder/file properties
      • Set compiler properties
  • Compiler-related
    • Compile the workspace
    • Control intercept module settings and generation
  • Test-run related
    • Run scripts
    • Control optional trace views


Calling Target C Functions from a Script

The set of captured interfaces is accessed via the injected ascript object. `This object connects scripts to the workspace's database and to the STRIDE runtime. The injected ascript instance has been initialized to reflect the current state of the currently loaded workspace's database.

Note: You can also instantiate an ascript object from an external script running outside of STRIDE Studio. In this case you must provide information by calling the new instance's Initialize() method with the required parameters.

the ascript object provides the means by which:

  • on-target C functions can be called and executed
  • on-target functions can be mocked (simulated) by script code running on the host

In either case, function parameter values can be set by the function's caller (known as the User) and read by the function implementation (known as the Owner). Similarly, function return values can be set by the function implementation (Owner) and read by the function's caller (User) upon return from the called function.

A Typical Scenario

In a typical testing scenario, the target system we are testing is up and running and the communication link between the host PC system and the target is operational. By default, the registered Owner of each function is the target implementation, therefore any calls we make to the function will be routed to the target by the host runtime.

The steps we would take in a script to call the remote function are:

  1. Get the desired Function User object instance from the ascript Functions collection
  2. Set the function's input parameter values
  3. Call the function (can be done synchronously or asynchronously)
  4. Upon the function's return, compare the return value against an expected value

In most scripts, the first step would be done once, and the remaining steps would be performed in a loop, presenting a different set of input parameter values each iteration.

Script syntax is available for navigating complex payloads comprising structures, unions, arrays and pointers.

For additional details and example scripts, see Using Scripts to Automate Software Testing.

Reporting from a Script

Reporting is performed using the injected testSuite object instance (the injected reporter object is also used occasionally).

STRIDE Studio automatically creates a hierarchy of test suites within the reporter instance that mirrors the hierarchy of folders under the workspace Script Files folder. The injected testSuite object is the instantiated Suite object in the reporter hierarchy that corresponds to the parent folder of the running script.

To create report output from a script, you do the following:

  1. create a new Test object instance as a child of the testSuite
  2. add comments and/or annotations to the Test object as needed
  3. set the Test status to "PASS" or "FAIL"

Note: You are not limited to a single Test instance in a script, although a common pattern is to create one script per test case.

An overall report is created automatically by STRIDE Studio after all scripts in the workspace have run. The report's native format is XML, which is commonly transformed to locally-viewable HTML using an XSL script when test are run interactively, or uploaded to a server and aggregated when test are run unattended.

Mocking Target C Functions Using a Script

Mocking (simulating) a target C function comprises two steps:

  1. Implement the required functionality in a script
  2. Register the script as the function's Owner

Registering as the function Owner causes the STRIDE runtime to route all calls to the function to the script implementation.

It is also possible to switch between your target function implementation and your script implementation at runtime.

For additional details and example scripts, see Using Scripts to Simulate Missing Software Units.

Notes

  1. S2 Technologies provides full support for Microsoft JScript and Active State Perl, however, other languages that support COM Automation have been used successfully. Contact S2 Technologies for more information.