Studio:Using Scripts to Automate Software Testing

This entry shows how to use a script running in STRIDE Studio to test a function. The basic technique is also applicable to less-frequently-used messaging interfaces.

In STRIDE terminology, the script is the User of the interface. (In contrast, Using Scripts to Simulate Missing Software Units involves a script that is the Owner of an interface.)

The script's injected ascript object instance provides the bridge between the interfaces in your embedded application and your favorite scripting language.

Example

In this example, we will be testing a target-based function that takes an integer as an argument, and returns an enumeration indicating whether the passed-in number is a leap year.

In an actual testing scenario, the function would be implemented on your target system. Also, the example header file would be added to your workspace and the script file would be created and added to your workspace.

Header File

Below is the sample .h file that defines the interface we will be testing. Typically, this is a header file from the target codebase.

Notice that it is annotated with SCL #pragmas to provide needed metadata to the STRIDE compiler. When SCL #pragmas are inline with the source, we generally surround them with #ifdefs to prevent them from being seen by your code compiler.

#ifndef _LEAPYEAR_H_
#define _LEAPYEAR_H_

// first year of the gregorian calendar
#define FIRST_GREGORIAN_YEAR 1582

typedef enum 
{
    NO,
    YES,
} eIsLeapYear;


/**
 * Tests whether the passed-in year is a leap year
 * 
 * @param nYear  The year to be tested. Valid values are from
 *               FIRST_GREGORIAN_YEAR to 0xffffffff inclusive
 * 
 * @return YES - the passed-in year is a leap year
 *         NO - the passed-in year is not a leap year, or is pre-gregorian
 */
eIsLeapYear IsLeapYear(unsigned int uYear);


#ifdef _SCL
// Tell the STRIDE compiler that the IsLeapYear should be captured (i.e. instrumented).
// Capturing the function allows it to be called from a script
#pragma scl_function(IsLeapYear)
// Tell the STRIDE compiler that the return value of IsLeapYear is constrained to
// the enumeration eIsLeapYear
#pragma scl_values(IsLeapYear(), eIsLeapYear)
#endif

#endif // _LEAPYEAR_H_

In a testing scenario, this header file is processed by the STRIDE compiler to produce a metadata database, and an Intercept Module (IM) is generated. The IM comprises a set of source files that provide proxy/stub functionality for the captured functions. The IM files become part of your target code build.

Note that once built, STRIDE compilation and IM generation only needs to be done if the function signature changes.

You can now use a script to interact with the captured function.

Test Scripts

The following example test scripts demonstrate how to set up and call a remote function. They also show rudimentary use of the testSuite object to report on test results.

The scripts provide identical functionality: a common pattern of exercising the target function by calling it in a loop iterating through a set of inputs. The function's return value is validated and the test result set accordingly.

Note that we use a single test case to test the entire set of input values, and we bail out of the test at the first failure. Another approach is to create a test case for each input value.

Perl Test

# standard STRIDE preamble for Perl
use strict;
use Win32::OLE;
Win32::OLE->Option(Warn => 3);

# create a new test in the parent suite
my $test = $main::testSuite->Tests->Add("Leap Year Positive Test");

# initialize the test input values
my @input = (1600, 1604, 1996, 2000);

# get the function user instance
my $f = $main::ascript->Functions->Item("IsLeapYear")->User;

# iterate through the members of arInput
for my $i (0..$#input)
{
    # set the parameter value
    $f->ParameterList->{uYear} = $input[$i];

    # call the function synchronously; the STRIDE runtime will route the call to the
    # currently registered function owner
    $f->Call();

    # test the return value; note that the value is returned as the name of the enumerator
    if ($f->ReturnValue eq "YES") {
        $test->{Status} = "PASS";
    }
    else {
        $test->{Status} = "FAIL";
        last;
    } 
}

JScript Test

// create a new test in the parent suite
var test = testSuite.Tests.Add("Leap Year Test");

// initialize the test input values
var arInput = new Array(1600, 1604, 1996, 2000);

// get the function user instance
var f = ascript.Functions.Item("IsLeapYear").User;

var nIndex;

// iterate through the members of arInput
for (nIndex in arInput) {
    // set the parameter value
    f.ParameterList.uYear = arInput[nIndex];
    
    // call the function synchronously; the STRIDE runtime will route the call to the
    // currently registered function owner
    f.Call();
    
    // test the return value; note that the value is returned as the name of the enumerator
    if (f.ReturnValue == "YES") {
        test.Status = "PASS";
    }
    else {
        test.Status = "FAIL";
        break;
    }
}