Reporting Model - Revision history

Marku at 16:04, 4 July 2015

2015-07-04T16:04:41Z

← Older revision		Revision as of 16:04, 4 July 2015
Line 1:		Line 1:
			__NOTOC__
	The following describes the entities that we represent internally for the purpose of capturing and storing test results.		The following describes the entities that we represent internally for the purpose of capturing and storing test results.

	~~==Reporting Entities==~~		The primary reporting entities are:

	The STRIDE Framework has in internal reporting model that is manifested in the [[Runtime_Test_Services\|Runtime Test Services API]] and in the navigable [[STRIDE_Test_Space#Results_View\|results view]] of STRIDE Test Space. The primary reporting entities are:
	* '''Suites'''		* '''Suites'''
	* '''Test Cases'''		* '''Test Cases'''
Line 18:		Line 17:
	* Comments are a simple leaf object and consist only of a label and a value (string).		* Comments are a simple leaf object and consist only of a label and a value (string).

	~~To see how these entities are represented in Test Space, refer to [[Reporting_Entities\|this article]]~~		=Suites=

	===Suites===
	A Suite acts as a container holding Test Cases. Suites serve to group tests together in a report, but also--importantly--provide subtotals of Pass/Fail counts, Duration, and Error count values, thus allowing you to better pinpoint where problems are occurring.		A Suite acts as a container holding Test Cases. Suites serve to group tests together in a report, but also--importantly--provide subtotals of Pass/Fail counts, Duration, and Error count values, thus allowing you to better pinpoint where problems are occurring.

Line 50:		Line 47:
	\|}		\|}

	===Test Cases===		=Test Cases=
	Test Cases are the individual units of pass/fail in the STRIDE Framework. Each Test Case has a single status value and an optional description. When relying on the default Framework behavior of test units, (i.e. '''not''' using dyanimic test case creation), a test case is automatically created by the Framework for each Test Method when it is executed.		Test Cases are the individual units of pass/fail in the STRIDE Framework. Each Test Case has a single status value and an optional description. When relying on the default Framework behavior of test units, (i.e. '''not''' using dyanimic test case creation), a test case is automatically created by the Framework for each Test Method when it is executed.

Line 79:		Line 76:
	\|}		\|}

	===Annotations===		=Annotations=
	Annotations can be attached to Suites or Test Cases and provide supplemental information about the item they annotate. Annotations have a name, description, and a level that will be familiar to users of standard logging systems.		Annotations can be attached to Suites or Test Cases and provide supplemental information about the item they annotate. Annotations have a name, description, and a level that will be familiar to users of standard logging systems.

Line 109:		Line 106:
	\|}		\|}

	===Comments===		=Comments=

	Comments are lightweight informational entities that can be attached to Annotations. Comments have only a label and comment values and do not need to be uniquely named within the context of the object they belong to. Comments are not currently used by any of the Framework macros but they can be added with the [[Runtime Test Services\|Runtime API]]		Comments are lightweight informational entities that can be attached to Annotations. Comments have only a label and comment values and do not need to be uniquely named within the context of the object they belong to. Comments are not currently used by any of the Framework macros but they can be added with the [[Runtime Test Services\|Runtime API]]
Line 130:		Line 127:

	\|}		\|}

	* [[Notes, Comments and Logs]]

	~~[[Category:Reference]]~~

Ivailop: /* Reporting Entities */

2014-09-11T22:21:03Z

Reporting Entities

← Older revision		Revision as of 22:21, 11 September 2014
Line 14:		Line 14:
	To summarize the relationships in this diagram:		To summarize the relationships in this diagram:
	* A Suite has 0 or more Test Cases and Annotations		* A Suite has 0 or more Test Cases and Annotations
	* A Test Case has 0 or more ~~Comments and~~ Annotations		* A Test Case has 0 or more Annotations
	* An Annotation can also have 0 or more Comments.		* An Annotation can also have 0 or more Comments.
	* Comments are a simple leaf object and consist only of a label and a value (string).		* Comments are a simple leaf object and consist only of a label and a value (string).

Ivailop: /* Comments */

2014-09-11T22:20:23Z

Comments

← Older revision		Revision as of 22:20, 11 September 2014
Line 111:		Line 111:
	===Comments===		===Comments===

	Comments are lightweight informational entities that can be attached to ~~Test Cases or~~ Annotations. Comments have only a label and comment values and do not need to be uniquely named within the context of the object they belong to. Comments are not currently used by any of the Framework macros but they can be added with the [[Runtime Test Services\|Runtime API]]		Comments are lightweight informational entities that can be attached to Annotations. Comments have only a label and comment values and do not need to be uniquely named within the context of the object they belong to. Comments are not currently used by any of the Framework macros but they can be added with the [[Runtime Test Services\|Runtime API]]

	Comment attributes are shown in the table below:		Comment attributes are shown in the table below:

Ivailop: /* Suites */

2014-09-11T22:17:46Z

Suites

← Older revision		Revision as of 22:17, 11 September 2014
Line 21:		Line 21:

	===Suites===		===Suites===
	A Suite acts as a container holding Test Cases ~~(and possibly, other Suites)~~. Suites serve to group tests together in a report, but also--importantly--provide subtotals of Pass/Fail counts, Duration, and Error count values, thus allowing you to better pinpoint where problems are occurring.		A Suite acts as a container holding Test Cases. Suites serve to group tests together in a report, but also--importantly--provide subtotals of Pass/Fail counts, Duration, and Error count values, thus allowing you to better pinpoint where problems are occurring.

	There are ~~three~~ ways to create a Suite in the STRIDE Framework.		There are two ways to create a Suite in the STRIDE Framework.

	;Implicit Test Unit Creation		;Implicit Test Unit Creation
	:The declaration of a STRIDE test unit in your source code implicitly creates a Suite at runtime. The suite's name is set to the name of the test unit.		:The declaration of a STRIDE test unit in your source code implicitly creates a Suite at runtime. The suite's name is set to the name of the test unit.

	~~;Dynamic Creation on the Target~~
	:Suites can be created by your test code at runtime using the [[Runtime_Test_Services\|STRIDE API]]. This technique is used primarily when executing data-driven tests where input data drives the composition of tests. A Suite created in this way must be the child of a Test Unit-created Suite.

	;Dynamic Creation on the Host		;Dynamic Creation on the Host

Timd at 18:57, 12 July 2012

2012-07-12T18:57:20Z

← Older revision		Revision as of 18:57, 12 July 2012
Line 134:		Line 134:
	\|}		\|}

			* [[Notes, Comments and Logs]]

	[[Category:Reference]]		[[Category:Reference]]

Ivailop: /* Suites */

2012-02-27T23:57:31Z

Suites

← Older revision		Revision as of 23:57, 27 February 2012
Line 50:		Line 50:
	\|		\|
	* This string is written to the test output describing the Suite (and possibly containing links to supplemental information)		* This string is written to the test output describing the Suite (and possibly containing links to supplemental information)
	* if the Suite corresponds to a source Test Unit the Description value can be automatically filled in from ~~Doxygen~~ comments in source code		* if the Suite corresponds to a source Test Unit the Description value can be automatically filled in from doxygen formatted comments in source code
	\|}		\|}

Marku: /* Annotations */

2010-06-02T22:30:19Z

Annotations

← Older revision		Revision as of 22:30, 2 June 2010
Line 85:		Line 85:
	Annotations can be attached to Suites or Test Cases and provide supplemental information about the item they annotate. Annotations have a name, description, and a level that will be familiar to users of standard logging systems.		Annotations can be attached to Suites or Test Cases and provide supplemental information about the item they annotate. Annotations have a name, description, and a level that will be familiar to users of standard logging systems.

	Annotations are added automatically to results by the Framework when using any of the [[Test Code Macros]]. [[Using Test Points]] will also create annotations in your Test Cases when they are active.		Annotations are added automatically to results by the Framework when using any of the [[Test Code Macros]]. [[Test Point \| Using Test Points]] will also create annotations in your Test Cases when they are active.

	Annotation attributes are shown in the table below:		Annotation attributes are shown in the table below:

Mikee: Created page with 'The following describes the entities that we represent internally for the purpose of capturing and storing test results. ==Reporting Entities== The STRIDE Framework has in in…'

2010-05-05T22:39:37Z

Created page with 'The following describes the entities that we represent internally for the purpose of capturing and storing test results. ==Reporting Entities== The STRIDE Framework has in in…'

New page

The following describes the entities that we represent internally for the purpose of capturing and storing test results.

==Reporting Entities==

The STRIDE Framework has in internal reporting model that is manifested in the [[Runtime_Test_Services|Runtime Test Services API]] and in the navigable [[STRIDE_Test_Space#Results_View|results view]] of STRIDE Test Space. The primary reporting entities are:
* '''Suites'''
* '''Test Cases'''
* '''Annotations'''
* '''Comments'''
This diagram shows how these items are related:

[[Image:TestEntities.png]]

To summarize the relationships in this diagram:
* A Suite has 0 or more Test Cases and Annotations
* A Test Case has 0 or more Comments and Annotations
* An Annotation can also have 0 or more Comments.
* Comments are a simple leaf object and consist only of a label and a value (string).

To see how these entities are represented in Test Space, refer to [[Reporting_Entities|this article]]

===Suites===
A Suite acts as a container holding Test Cases (and possibly, other Suites). Suites serve to group tests together in a report, but also--importantly--provide subtotals of Pass/Fail counts, Duration, and Error count values, thus allowing you to better pinpoint where problems are occurring.

There are three ways to create a Suite in the STRIDE Framework.

;Implicit Test Unit Creation
:The declaration of a STRIDE test unit in your source code implicitly creates a Suite at runtime. The suite's name is set to the name of the test unit.

;Dynamic Creation on the Target
:Suites can be created by your test code at runtime using the [[Runtime_Test_Services|STRIDE API]]. This technique is used primarily when executing data-driven tests where input data drives the composition of tests. A Suite created in this way must be the child of a Test Unit-created Suite.

;Dynamic Creation on the Host
:You can further organize implicitly-created (Test Unit) Suites under parent suites you designate when you run the tests. The [[STRIDE Runner|stride runner]] makes it possible to specify a "suite path" for each implicitly-created Suite into which results are to be reported. This technique is generally used where there are many test units being run, since it allows for more detailed organization and value roll-ups.

Suite attributes are shown in the table below:

{| class="wikitable"
|+ Suite Attributes
! Suite<br/> Attribute !! Value Required !! Comments
|-
| '''Name'''
| Yes
|
* The Suite is identified by its name in the test report output
* if the Suite corresponds to a source Test Unit its default Name value is the Test Unit name
|-
| '''Description'''
| No
|
* This string is written to the test output describing the Suite (and possibly containing links to supplemental information)
* if the Suite corresponds to a source Test Unit the Description value can be automatically filled in from Doxygen comments in source code
|}

===Test Cases===
Test Cases are the individual units of pass/fail in the STRIDE Framework. Each Test Case has a single status value and an optional description. When relying on the default Framework behavior of test units, (i.e. '''not''' using dyanimic test case creation), a test case is automatically created by the Framework for each Test Method when it is executed.

Test Case attributes are shown in the table below:

{| class="wikitable"
|+ Test Case Attributes
! Test Case<br/> Attribute !! Value Required !! Comments
|-
| '''Name'''
| Yes
|
* The Test Case is identified by its name in the test report output
* if the Test Case corresponds to a source Test Method its default Name value is the Test Method name
|-
| '''Description'''
| No
|
* This string is written to the test output describing the Test Case (and possibly containing links to supplemental information)
* if the Test Case corresponds to a source Test Method the Description value can be automatically filled in
|-
| '''Status'''
| Yes
|
* The Test Case status is one of the following: '''pass''', '''fail''', '''in progress''', or '''not applicable'''
* if the Test Case corresponds to a source Test Method the status defaults to '''pass''', but is overridden if the test case fails for any reason.

|}

===Annotations===
Annotations can be attached to Suites or Test Cases and provide supplemental information about the item they annotate. Annotations have a name, description, and a level that will be familiar to users of standard logging systems.

Annotations are added automatically to results by the Framework when using any of the [[Test Code Macros]]. [[Using Test Points]] will also create annotations in your Test Cases when they are active.

Annotation attributes are shown in the table below:

{| class="wikitable"
|+ Annotation Attributes
! Annotation<br/> Attribute !! Value Required !! Comments
|-
| '''Name'''
| Yes
|
* The Annotation is identified by its name in the test report output
|-
| '''Description'''
| No
|
* This string is written to the test output describing the Annotation
* Annotations that are created by Framework macros will have a description value with more detailed information about the condition that triggered the annotation.
|-
| '''Level'''
| Yes
|
* The Annotation level is one of the following: '''trace''', '''debug''', '''info''', '''warn''', '''error''', or '''fatal'''
* Annotations for Suites can only be created using the [[Runtime Test Services|Runtime API]]. Annotations for Test Cases can also be created using the Runtime, but they are also automatically created by various Framework macros, particularly when test failures occur.

|}

===Comments===

Comments are lightweight informational entities that can be attached to Test Cases or Annotations. Comments have only a label and comment values and do not need to be uniquely named within the context of the object they belong to. Comments are not currently used by any of the Framework macros but they can be added with the [[Runtime Test Services|Runtime API]]

Comment attributes are shown in the table below:

{| class="wikitable"
|+ Comment Attributes
! Comment<br/> Attribute !! Value Required !! Comments
|-
| '''Label'''
| Yes
|
* This is a simple (typically short) string label for the comment.
|-
| '''Comment'''
| Yes
|
* This string represents the body of the comment.

|}

[[Category:Reference]]