Perl Script APIs

From STRIDE Wiki
Revision as of 18:23, 28 January 2010 by Mikee (talk | contribs) (→‎User Defined Predicates)
Jump to navigation Jump to search

The STRIDE Framework perl script model requires you to create perl modules (*.pm files) to group your tests. The following documents the API for the STRIDE::Test base class that you use when creating these modules.

STRIDE::Test

This is the base class for test modules. It provides the following methods.

Declaring Tests

Once you have created a package that inherits form STRIDE::Test, you can declare any subroutine to be a test method by declaring it with the : Test attribute. In addition to test methods, the following attributes declare other kinds of subroutines:

subroutine attributes
attribute description
Test declares a test method - will be executed automatically when the module is run.
Test(startup) startup method, called once before any of the test methods have been executed.
Test(shutdown) shutdown method, called once after all test methods have been executed.
Test(setup) setup fixture, called before each test method.
Test(teardown) teardown fixture, called after each test method.

You are free to declare as many methods as you like with these attributes. When more than one method has been declared with the same attribute, the methods will be called at the appropriate time in the order declared.

Methods

These methods are all available in the context of a test module that inherits from STRIDE::Test.

Methods
name description 
TestCase
returns the current test case object. 
TestSuite
returns the current test suite object. 
AddAnnotation( 
  test_case => case,
  label => "", 
  level => LEVEL,
  message => "")
Adds an annotation to the current test case (or to test_case if that named parameter is provided). A label can be provided but will default to "Host Annotation". A level can be provided as one of the following, but will default to INFO level:
  • ANNOTATION_TRACE
  • ANNOTATION_DEBUG
  • ANNOTATION_INFO
  • ANNOTATION_WARNING
  • ANNOTATION_ERROR
  • ANNOTATION_FATAL

The message parameter is optional and specifies the text to use in the annotation description field. 

AddTestCase(suite)
Creates a new test case in the specified suite (or the current TestSuite(), if none is provided). This also updates the current test case value that is returned by TestCase(). 
AddTestSuite(suite)
Creates a new sub-suite in the specified suite (or the current TestSuite(), if none is provided). This also updates the current test suite value that is returned by TestSuite(). 
Functions
Returns a STRIDE::Function object that was initialized with the active database. This object is used for calling captured functions in the database. 
Constants
Returns a STRIDE::Function->{constants} hash that was initialized with the active database. This object is a tied hash that can be used to retrieve database constant values (macros and enums) at the time of compilation. 
TestPointSetup( 
  order => ORDER, 
  expected => [], 
  unexpected => [], 
  test_case => case)
Creates a new instance of STRIDE::TestPoint, automatically passing the default TestCase() as the test case if none is provided. The order value can be one of these defined constants:
  • TEST_POINT_EXPECT_ORDERED
  • TEST_POINT_EXPECT_UNORDERED

It will default to TEST_POINT_EXPECT_UNORDERED if not provided.

The expected parameter is an array reference containing elements that are either just strings representing the test point labels OR an anonymous sub-array that contains these four items: [label, count, predicate, expected_data] where

label
is the test point label (same as the non array argument type).
count
the number of expected occurrences - can be any positive integer value, or the special value TEST_POINT_ANY_COUNT.
predicate
must be a perl coderef for a function to call as a predicate. You are free to define your own predicate function OR use on the three standard ones provided by STRIDE::Test.
expected_data
if provided, will be passed as an argument to the predicate - intended to be used to specify expected data.

If using the array form of expectation, only the label entry is required - the remaining elements are optional. The unexpected parameter is an array reference containing labels that are to be treated as failure if they are encountered. For either expected and unexpected, the special value TEST_POINT_EVERYTHING_ELSE can be used alone to indicate that any test points not explicitly listed in the set are considered part of this set.

The returned object is of type STRIDE::TestPoint and has access to all it's member functions.

Assertions

Each of the following assertion methods are provided for standard comparisons. For each, there there are three different types, depending on the desired behavior upon failure: EXPECT, ASSERT, and EXIT. EXPECT checks will fail the current test case but continue executing the test method. ASSERT checks will fail the current test case and exit the test method immediately. EXIT checks will fail the current test case, immediately exit the current test method AND cease further execution of the test module.


Boolean
macro Pass if
prefix_TRUE(cond); cond is true
prefix_FALSE(cond); cond is false


Comparison
macro Pass if
prefix_EQ(val1, val2); val1 == val2
prefix_NE(val1, val2); val1 != val2
prefix_LT(val1, val2); val1 < val2
prefix_LE(val1, val2); val1 <= val2
prefix_GT(val1, val2); val1 > val2
prefix_GE(val1, val2); val1 >= val2

For all of the value comparison methods (_EQ, _NE, etc.), the comparison is numeric if both arguments are numeric -- otherwise the comparison is a case sensitive string comparison. If case insensitive comparison is needed, simply wrap both arguments with perl's builtin lc() (lowercase) or uc() (uppercase) functions.

Predicates
macro Pass if
prefix_PRED(coderef, data) &coderef(data) returns true. The predicate function is specified by coderef with optional data data. The predicate can also return the special value TEST_POINT_IGNORE to indicate that the event should be ignored.

Each of these expectation methods also supports the following optional named arguments:

test_case => case
allows you to apply the check to a test case other than the current default
message => "message"
allows you to specify an additional message to include if the check fails.

Because these arguments are optional, they are passed using named argument (hash-style) syntax after the required parameters that are shown above.

Logging

Logging Methods
name description 
TEST_LOG_INFO(message)
creates an info log message. 
TEST_LOG_WARN(message)
creates an warning log message. 
TEST_LOG_ERROR(message)
creates an error log message.

Each of these log methods also supports the following optional named arguments:

test_case => case
allows you to add the log to a test case other than the current default
file => file
allows you to attach a file along with the annotation message that is generated for the log message.

Because these arguments are optional, they are passed using named argument (hash-style) syntax after the required parameters that are shown above.

Documentation

We have preliminary support for documentation extraction in the test modules using the standard perl POD formatting tokens.

The POD that you include in your test module currently must follow these conventions:

  • it must begin with a head1 NAME section and the text of this section must contain the name of the package, preferably near the beginning.
  • a head1 DESCRIPTION can follow the NAME section. If provided, it will be used as the description of the test suite created for the test unit.
  • This NAME/DESCRIPTION block must finish with an empty head1 METHODS section.
  • each of the test methods can be documented by preceding them with a head2 section with the same name as the test method (subroutine name). The text in this section will be used as the testcase description.

Predicates

STRIDE expectation testing allows you to specify predicate functions for sophisticated data validation. We provide several standard predicates in the STRIDE::Test package, or you are free to define your own predicate functions.

Builtin Predicates

The STRIDE::Test library provides a few standard predicates which you are free to use in your expectations:

Built-In Predicates
predicate description
TestPointStrCmp does a case sensitive comparison of the test point data and the expected_data (specified as part of the expectation)
TestPointStrCaseCmp does a case insensitive comparison of the test point data and the expected_data
TestPointMemCmp does a bytewise comparison of the test point data and the expected_data

User Defined Predicates

User defined predicates are subroutines of the following form: 

sub myPredicate 
{
    my ($test_point, $expected_data) = @_;
    my $status = 0;
    # access the test point data as $test_point->{data}, 
    # and the label as $test_point->{label}

    # set $status according to whether or not your predicate passes
    return $status;
}

The predicate function is passed two arguments: the current test point and the expected data that was specified as part of the expectation. The test point data is a reference to a hash with the following fields:

label
the test point label
data
the data payload for the test point (if any)
size
the size of the data payload
bin
flag indicating whether or not the data payload is binary
file
the source file for the test point
line
the line number for the test point

The expected data is passed as a single scalar, but you can use references to compound data structures (hashes, arrays) if you need more complex expected data.

The predicate function should return a true value if it passes, false if not, or TEST_POINT_IGNORE if the test point should be ignored completely.

STRIDE::Function

The STRIDE::Function class uses perl AUTOLOADing to provide a convenient syntax for making simple function calls and retrieving database constants in perl. Given any properly initialized Function object, any captured function or constant (macro) is available directly as method or properties of the Function object. Constants can also be accessed via the tied hash constants member (which is exported as Constants in the STRIDE::Test package).

For example, given a database with two functions: int foo(const char * path) and void bar(double value), and a macro #define MY_PI_VALUE 3.1415927, these methods/constants are invokable using the exported STRIDE::Test Functions and Constants objects: 

my $retval = Functions->foo("my string");
Functions->bar(Constants->{MY_PI_VALUE});

Asynchronous invocation

Functions can also be called asynchronously by setting the non_blocking property to 1 before making a function call and then calling WaitReturn later if/when you want to retrieve the return value - for example: 

Functions->{non_blocking} = 1;
Functions->foo("my string");
my $retval = Functions->WaitReturn('foo', 1000);

The WaitReturn function takes two arguments: the first is the name of the function for which you want to wait and the second is a timeout duration (in milliseconds) that indicates the maximum time to wait for the function to return. If the timeout value is not provided, WaitReturn will wait indefinitely. If a timeout is specified and expires before the function returns, the method with die with a timeout error message.

The non_blocking property is currently reset immediately after a function invocation, so callers are required to set this property immediately before calling any function asynchronously.

STRIDE::TestPoint

STRIDE::TestPoint objects are used to create test point expectation tests. These objects are created using the exported TestPointSetup factory function of the STRIDE::Test class. Once a STRIDE::TestPoint object has been created with the desired expectations, two functions can be called:

Wait(timeout)
This method processes test points that have occurred on the target and assesses failure based on the parameters you provided when creating the TestPoint object. The timeout parameter indicates how long (in milliseconds) to Wait for the specified events. If no timeout value is provided, Wait will proceed indefinitely or until a clear pass/failure determination can be made.
Check()
this is equivalent to Wait with a very small timeout. As such, it essentially verifies that your specified test points have already been hit.
Retrieved from "https://www.stridewiki.com/index.php?title=Perl_Script_APIs&oldid=11839"