Perl Script APIs - Revision history

Ivailop: /* Methods */

2016-07-27T22:22:46Z

Methods

← Older revision		Revision as of 22:22, 27 July 2016
Line 59:		Line 59:
	\|-valign="top"		\|-valign="top"
	\| <source lang="perl">GetParam(name, defval)</source>		\| <source lang="perl">GetParam(name, defval)</source>
	\| returns an input parameter's value associated with the ~~default~~ test suite.		\| returns an input parameter's value associated with the test suite.

	\|-valign="top"		\|-valign="top"
	\| <source lang="perl">TestCase AddCase(name, description)</source>		\| <source lang="perl">TestCase AddCase(name, description)</source>
	\| creates a new test case and adds it to the ~~default~~ test suite.		\| creates a new test case and adds it to the test suite.

	\|-valign="top"		\|-valign="top"
	\| <source lang="perl">TestAnnotation AddAnnotation(level, name, description)</source>		\| <source lang="perl">TestAnnotation AddAnnotation(level, name, description)</source>
	\| creates a new test annotation and adds it to the ~~default~~ test suite. The value of ''level'' could be one of the predefined constants:		\| creates a new test annotation and adds it to the test suite. The value of ''level'' could be one of the predefined constants:
	* '''ANNOTATION_TRACE'''<ref name="const">symbols like these are exported as perl constants - don't quote these values when you use them - rather, use the bare symbols and perl will use the constant value we provide</ref>		* '''ANNOTATION_TRACE'''<ref name="const">symbols like these are exported as perl constants - don't quote these values when you use them - rather, use the bare symbols and perl will use the constant value we provide</ref>
	* '''ANNOTATION_DEBUG'''		* '''ANNOTATION_DEBUG'''
Line 78:		Line 78:
	\|-valign="top"		\|-valign="top"
	\| <source lang="perl">SetData(name, value)</source>		\| <source lang="perl">SetData(name, value)</source>
	\| associates a custom name-value pair with the ~~default~~ test suite.		\| associates a custom name-value pair with the test suite.

	\|-valign="top"		\|-valign="top"

Ivailop: /* Assertions */

2015-08-28T22:43:49Z

Assertions

← Older revision		Revision as of 22:43, 28 August 2015
Line 114:		Line 114:

	Each of the following assertion methods are provided for standard comparisons. For each, there there are three different types, depending on the desired behavior upon failure: '''EXPECT''', '''ASSERT''', and '''EXIT'''. '''EXPECT''' checks will fail the current test case but continue executing the test method. '''ASSERT''' checks will fail the current test case and exit the test method immediately. '''EXIT''' checks will fail the current test case, immediately exit the current test method AND cease further execution of the test module.		Each of the following assertion methods are provided for standard comparisons. For each, there there are three different types, depending on the desired behavior upon failure: '''EXPECT''', '''ASSERT''', and '''EXIT'''. '''EXPECT''' checks will fail the current test case but continue executing the test method. '''ASSERT''' checks will fail the current test case and exit the test method immediately. '''EXIT''' checks will fail the current test case, immediately exit the current test method AND cease further execution of the test module.

			For simplicity, we refer to all the macros using a '''''prefix''''' tag - when using the macros in test code, the '''''prefix''''' should be replaced by one of the following: '''EXPECT''', '''ASSERT''', or '''EXIT''', depending on how the test writer wants failures to be handled.

Marku at 20:49, 6 July 2015

2015-07-06T20:49:17Z

← Older revision		Revision as of 20:49, 6 July 2015
Line 1:		Line 1:
	__NOTOC__		__NOTOC__
			The Stride perl script model requires you to create perl modules (*.pm files) to group your tests. The following documents the API for the STRIDE::Test base class that you use when creating these modules.
	The ~~STRIDE Framework~~ perl script model requires you to create perl modules (*.pm files) to group your tests. The following documents the API for the STRIDE::Test base class that you use when creating these modules.

	= STRIDE::Test =		= STRIDE::Test =
Line 103:		Line 102:
	If using the array form of expectation, only the label entry is required - the remaining elements are optional.		If using the array form of expectation, only the label entry is required - the remaining elements are optional.
	; unexpected : is an array reference containing labels that are to be treated as failure if they are encountered. For either ''expected'' and ''unexpected'', the special value '''TEST_POINT_EVERYTHING_ELSE'''<ref name="const"/> can be used alone to indicate that any test points not explicitly listed in the set are considered part of this set.		; unexpected : is an array reference containing labels that are to be treated as failure if they are encountered. For either ''expected'' and ''unexpected'', the special value '''TEST_POINT_EVERYTHING_ELSE'''<ref name="const"/> can be used alone to indicate that any test points not explicitly listed in the set are considered part of this set.
	; expect_file : if you have previously captured YAML trace data in a file (using the [[~~STRIDE~~ Runner]]), you can specify the file as the source of expectations using this parameter. If you specify a YAML trace file, you should NOT also specify the '''expected''' items as they will be overridden by the YAML trace file.		; expect_file : if you have previously captured YAML trace data in a file (using the [[Stride Runner]]), you can specify the file as the source of expectations using this parameter. If you specify a YAML trace file, you should NOT also specify the '''expected''' items as they will be overridden by the YAML trace file.
	; predicate : is a perl coderef to a default predicate function to use for all expectation items. If any specific entry in the expectation list has a predicate function, the expectation's predicate will override this global value. By default, no global predicate is assumed and no predicate is called unless specified for each expectation item.		; predicate : is a perl coderef to a default predicate function to use for all expectation items. If any specific entry in the expectation list has a predicate function, the expectation's predicate will override this global value. By default, no global predicate is assumed and no predicate is called unless specified for each expectation item.
	; replay_file : allows you to specify a YAML trace file and '''''input''''' to the current test. If specified, the expectation will be validated against the events specified in the file rather than live events generated by a device under test.		; replay_file : allows you to specify a YAML trace file and '''''input''''' to the current test. If specified, the expectation will be validated against the events specified in the file rather than live events generated by a device under test.

Marku at 18:36, 2 July 2015

2015-07-02T18:36:40Z

← Older revision		Revision as of 18:36, 2 July 2015
Line 1:		Line 1:
			__NOTOC__

	The STRIDE Framework perl script model requires you to create perl modules (*.pm files) to group your tests. The following documents the API for the STRIDE::Test base class that you use when creating these modules.		The STRIDE Framework perl script model requires you to create perl modules (*.pm files) to group your tests. The following documents the API for the STRIDE::Test base class that you use when creating these modules.

	== STRIDE::Test ==		= STRIDE::Test =

	This is the base class for test modules. It provides the following methods.		This is the base class for test modules. It provides the following methods.

	=== Declaring Tests ===		== Declaring Tests ==

	Once you have created a package that inherits from STRIDE::Test, you can declare any subroutine to be a test method by declaring it with the '': Test'' attribute. In addition to test methods, the following attributes declare other kinds of subroutines:		Once you have created a package that inherits from STRIDE::Test, you can declare any subroutine to be a test method by declaring it with the '': Test'' attribute. In addition to test methods, the following attributes declare other kinds of subroutines:
Line 39:		Line 41:
	You are free to declare as many methods as you like with these attributes. When more than one method has been declared with the same attribute, the methods will be called at the appropriate time in the order declared.		You are free to declare as many methods as you like with these attributes. When more than one method has been declared with the same attribute, the methods will be called at the appropriate time in the order declared.

	=== Methods ===		== Methods ==

	These methods are all available in the context of a test module that inherits from STRIDE::Test.		These methods are all available in the context of a test module that inherits from STRIDE::Test.
Line 110:		Line 112:
	\|}		\|}

	=== Assertions ===		== Assertions ==

	Each of the following assertion methods are provided for standard comparisons. For each, there there are three different types, depending on the desired behavior upon failure: '''EXPECT''', '''ASSERT''', and '''EXIT'''. '''EXPECT''' checks will fail the current test case but continue executing the test method. '''ASSERT''' checks will fail the current test case and exit the test method immediately. '''EXIT''' checks will fail the current test case, immediately exit the current test method AND cease further execution of the test module.		Each of the following assertion methods are provided for standard comparisons. For each, there there are three different types, depending on the desired behavior upon failure: '''EXPECT''', '''ASSERT''', and '''EXIT'''. '''EXPECT''' checks will fail the current test case but continue executing the test method. '''ASSERT''' checks will fail the current test case and exit the test method immediately. '''EXIT''' checks will fail the current test case, immediately exit the current test method AND cease further execution of the test module.
Line 181:		Line 183:
	Because these arguments are optional, they are passed using named argument (hash-style) syntax after the required parameters that are shown above.		Because these arguments are optional, they are passed using named argument (hash-style) syntax after the required parameters that are shown above.

	=== Annotations ===		== Annotations ==

	The following methods can be used to annotate a test case. Typically these methods are used to add additional information about the state of the test to the report.		The following methods can be used to annotate a test case. Typically these methods are used to add additional information about the state of the test to the report.
Line 212:		Line 214:
	Because these arguments are optional, they are passed using named argument (hash-style) syntax after the required parameters that are shown above.		Because these arguments are optional, they are passed using named argument (hash-style) syntax after the required parameters that are shown above.

	=== Documentation ===		== Documentation ==

	We have preliminary support for documentation extraction in the test modules using the standard perl POD formatting tokens.		We have preliminary support for documentation extraction in the test modules using the standard perl POD formatting tokens.
Line 223:		Line 225:
	* each of the test methods can be documented by preceding them with a ''head2'' section with the same name as the test method (subroutine name). The text in this section will be used as the testcase description.		* each of the test methods can be documented by preceding them with a ''head2'' section with the same name as the test method (subroutine name). The text in this section will be used as the testcase description.

	=== Predicates ===		== Predicates ==

	STRIDE expectation testing allows you to specify predicate functions for sophisticated data validation. We provide several standard predicates in the STRIDE::Test package, or you are free to define your own predicate functions.		STRIDE expectation testing allows you to specify predicate functions for sophisticated data validation. We provide several standard predicates in the STRIDE::Test package, or you are free to define your own predicate functions.

	==== Builtin Predicates ====		=== Builtin Predicates ===
	The STRIDE::Test library provides a few standard predicates which you are free to use in your expectations:		The STRIDE::Test library provides a few standard predicates which you are free to use in your expectations:

Line 254:		Line 256:
	\|}		\|}

	==== User Defined Predicates ====		=== User Defined Predicates ===
	User defined predicates are subroutines of the following form:		User defined predicates are subroutines of the following form:

Line 283:		Line 285:
	The predicate function should return a true value if it passes, false if not, or '''TEST_POINT_IGNORE'''<ref name="const"/> if the test point should be ignored completely.		The predicate function should return a true value if it passes, false if not, or '''TEST_POINT_IGNORE'''<ref name="const"/> if the test point should be ignored completely.

	== STRIDE::Remote ==		= STRIDE::Remote =

	The ''STRIDE::Remote'' class uses perl '''AUTOLOAD'''-ing to provide a convenient syntax for making simple function calls and retrieving database constants in perl. Given any properly initialized ''STRIDE::Test'' object, any captured function or constant (macro) is available directly as method or properties of the exported ''Remote'' object. Constants can also be accessed via the tied hash ''constants'' member.		The ''STRIDE::Remote'' class uses perl '''AUTOLOAD'''-ing to provide a convenient syntax for making simple function calls and retrieving database constants in perl. Given any properly initialized ''STRIDE::Test'' object, any captured function or constant (macro) is available directly as method or properties of the exported ''Remote'' object. Constants can also be accessed via the tied hash ''constants'' member.
Line 303:		Line 305:
	</source>		</source>

	=== Asynchronous invocation ===		== Asynchronous invocation ==

	Functions can also be called asynchronously by using the ''async'' delegator within the ''Remote'' object. When invoked this way, the function call will return a handle object that can be used to wait for the function return value - for example:		Functions can also be called asynchronously by using the ''async'' delegator within the ''Remote'' object. When invoked this way, the function call will return a handle object that can be used to wait for the function return value - for example:
Line 314:		Line 316:
	The '''Wait''' function takes one optional argument -- the timeout duration (in milliseconds) that indicates the maximum time to wait for the function to return. If the timeout value is not provided, ''Wait'' will wait indefinitely for the function to return. If a timeout is specified and expires before the function returns, the method with ''die'' with a timeout error message - so you might want to wrap your '''Wait''' call in an <tt>eval{};</tt> statement if you want to gracefully handle the timeout condition.		The '''Wait''' function takes one optional argument -- the timeout duration (in milliseconds) that indicates the maximum time to wait for the function to return. If the timeout value is not provided, ''Wait'' will wait indefinitely for the function to return. If a timeout is specified and expires before the function returns, the method with ''die'' with a timeout error message - so you might want to wrap your '''Wait''' call in an <tt>eval{};</tt> statement if you want to gracefully handle the timeout condition.

	== STRIDE::TestPoint ==		= STRIDE::TestPoint =

	STRIDE::TestPoint objects are used to create test point expectation tests. These objects are created using the exported TestPointSetup factory function of the STRIDE::Test class. Once a STRIDE::TestPoint object has been created with the desired expectations, two functions can be called:		STRIDE::TestPoint objects are used to create test point expectation tests. These objects are created using the exported TestPointSetup factory function of the STRIDE::Test class. Once a STRIDE::TestPoint object has been created with the desired expectations, two functions can be called:
Line 321:		Line 323:
	; Check() : this is equivalent to Wait with a very small timeout. As such, it essentially verifies that your specified test points have already been hit.		; Check() : this is equivalent to Wait with a very small timeout. As such, it essentially verifies that your specified test points have already been hit.

	== Reporting Model ==		= Reporting Model =

	The STRIDE perl framework includes an implementation of our [[Reporting Model]] that is common across all STRIDE components. The Test module gives explicit access to key elements of the report model, Cases and Annotaions. Here are a description of the methods available for each of these Objects.		The STRIDE perl framework includes an implementation of our [[Reporting Model]] that is common across all STRIDE components. The Test module gives explicit access to key elements of the report model, Cases and Annotaions. Here are a description of the methods available for each of these Objects.

	=== TestCase ===		== TestCase ==

	{\| class="prettytable"		{\| class="prettytable"
Line 358:		Line 360:
	\|}		\|}

	=== TestAnnotation ===		== TestAnnotation ==

	{\| class="prettytable"		{\| class="prettytable"
Line 374:		Line 376:
	== Notes ==		== Notes ==
	<references/>		<references/>

	~~[[Category:Tests in Script]]~~

Ivailop: /* Annotations */

2015-03-12T19:21:00Z

Annotations

← Older revision		Revision as of 19:21, 12 March 2015
Line 206:		Line 206:

	Each of these note methods also supports the following optional named arguments:		Each of these note methods also supports the following optional named arguments:
	; test_case => case : allows you to add the ~~log~~ to a test case other than the current default		; test_case => case : allows you to add the note to a test case other than the current default
	; file => file : allows you to attach a file along with the annotation ~~message that is generated for the log~~ message.		; file => file : allows you to attach a file along with the annotation message.
	; test_point => test_point_hashref : If you are annotating your report in the context of a predicate with a specific test point, you might want to specify the test point using this parameter. This will cause your annotation to be grouped in the final report with the annotation message that corresponds to the test point hit message. By default, a host timestamp value is used to generate the NOTE annotation, which generally causes the NOTE annotations to group toward the end of the test case report.		; test_point => test_point_hashref : If you are annotating your report in the context of a predicate with a specific test point, you might want to specify the test point using this parameter. This will cause your annotation to be grouped in the final report with the annotation message that corresponds to the test point hit message. By default, a host timestamp value is used to generate the NOTE annotation, which generally causes the NOTE annotations to group toward the end of the test case report.

Ivailop: /* Annotations */

2015-03-12T19:19:01Z

Annotations

← Older revision		Revision as of 19:19, 12 March 2015
Line 197:		Line 197:
	\|-valign="top"		\|-valign="top"
	\| <source lang="perl">NOTE_WARN(message)</source>		\| <source lang="perl">NOTE_WARN(message)</source>
	\| creates an warning note in your test results report.		\| creates a warning note in your test results report.

	\|-valign="top"		\|-valign="top"

Ivailop: /* TestCase */

2014-08-05T19:17:34Z

TestCase

← Older revision		Revision as of 19:17, 5 August 2014
Line 354:		Line 354:
	\|-valign="top"		\|-valign="top"
	\| <source lang="perl">SetData(name, value)</source>		\| <source lang="perl">SetData(name, value)</source>
	\| associates a custom name\|value pair with the test case.		\| associates a custom name-value pair with the test case.

	\|}		\|}

Ivailop: /* Methods */

2014-08-05T19:15:16Z

Methods

← Older revision		Revision as of 19:15, 5 August 2014
Line 73:		Line 73:
	* '''ANNOTATION_ERROR'''		* '''ANNOTATION_ERROR'''
	* '''ANNOTATION_FATAL'''		* '''ANNOTATION_FATAL'''
	\|}		\|

	\|-valign="top"		\|-valign="top"

Ivailop: /* Methods */

2014-08-05T19:14:15Z

Methods

← Older revision		Revision as of 19:14, 5 August 2014
Line 53:		Line 53:

	\|-valign="top"		\|-valign="top"
	\| <source lang="perl">~~TestSuite~~</source>		\| <source lang="perl">Remote</source>
	\| returns the default test suite ~~object~~. ~~See [[#TestSuite~~\|~~here]] for more info~~.		\| Returns a [[#STRIDE::Remote\|STRIDE::Remote]] object that was initialized with the active database. This object is used for accessing captured at the time of compilation remote functions and constant values (macros and enums).

			\|-valign="top"
			\| <source lang="perl">GetParam(name, defval)</source>
			\| returns an input parameter's value associated with the default test suite.

			\|-valign="top"
			\| <source lang="perl">TestCase AddCase(name, description)</source>
			\| creates a new test case and adds it to the default test suite.

	\|-valign="top"		\|-valign="top"
	\| <source lang="perl">~~Remote~~</source>		\| <source lang="perl">TestAnnotation AddAnnotation(level, name, description)</source>
	\| ~~Returns~~ a ~~[[#STRIDE::Remote\|STRIDE::Remote]] object that was initialized with~~ the ~~active database~~. ~~This object is used for accessing captured at~~ the ~~time of compilation remote functions~~ and constant ~~values~~ (~~macros and enums~~).		\| creates a new test annotation and adds it to the default test suite. The value of ''level'' could be one of the predefined constants:
			* '''ANNOTATION_TRACE'''<ref name="const">symbols like these are exported as perl constants - don't quote these values when you use them - rather, use the bare symbols and perl will use the constant value we provide</ref>
			* '''ANNOTATION_DEBUG'''
			* '''ANNOTATION_INFO'''
			* '''ANNOTATION_WARNING'''
			* '''ANNOTATION_ERROR'''
			* '''ANNOTATION_FATAL'''
			\|}

			\|-valign="top"
			\| <source lang="perl">SetData(name, value)</source>
			\| associates a custom name-value pair with the default test suite.

	\|-valign="top"		\|-valign="top"

Ivailop: /* Reporting Model */

2014-08-05T19:07:04Z

Reporting Model

@@ Line 304: / Line 304: @@
 == Reporting Model ==
-The STRIDE perl framework includes an implementation of our [[Reporting Model]] that is common across all STRIDE components. The Test module gives explicit access to key elements of the report model, Suites, Cases and Annotaions. Here are a description of the methods available for each of these Objects.
+The STRIDE perl framework includes an implementation of our [[Reporting Model]] that is common across all STRIDE components. The Test module gives explicit access to key elements of the report model, Cases and Annotaions. Here are a description of the methods available for each of these Objects.
 === TestCase ===
@@ Line 350: / Line 315: @@
 |-valign="top"
-| <source lang="perl">SetName(name)</source>
+| <source lang="perl">SetStatus(status, duration) </source>
 | sets the test case status. The value of ''status'' could be one of the predefined constants:
 * '''TEST_FAIL'''<ref name="const"/>
@@ Line 365: / Line 322: @@
 * '''TEST_INPROGRESS'''
 * '''TEST_DONE''' - applicable to dynamic cases - sets the status to '''pass''' unless already set to '''fail''' or '''not-in-use'''
 |-valign="top"
@@ Line 379: / Line 332: @@
 * '''ANNOTATION_ERROR'''
 * '''ANNOTATION_FATAL'''
 |}