Reporting Model

The following describes the entities that we represent internally for the purpose of capturing and storing test results.

Reporting Entities

The STRIDE Framework has in internal reporting model that is manifested in the Runtime Test Services API and in the navigable results view of STRIDE Test Space. The primary reporting entities are:

• Suites
• Test Cases
• Annotations
• Comments

This diagram shows how these items are related:

To summarize the relationships in this diagram:

• A Suite has 0 or more Test Cases and Annotations
• A Test Case has 0 or more Comments and Annotations
• An Annotation can also have 0 or more Comments.
• Comments are a simple leaf object and consist only of a label and a value (string).

To see how these entities are represented in Test Space, refer to this article

Suites

A Suite acts as a container holding Test Cases (and possibly, other Suites). Suites serve to group tests together in a report, but also--importantly--provide subtotals of Pass/Fail counts, Duration, and Error count values, thus allowing you to better pinpoint where problems are occurring.

There are three ways to create a Suite in the STRIDE Framework.

Implicit Test Unit Creation

The declaration of a STRIDE test unit in your source code implicitly creates a Suite at runtime. The suite's name is set to the name of the test unit.

Dynamic Creation on the Target

Suites can be created by your test code at runtime using the STRIDE API. This technique is used primarily when executing data-driven tests where input data drives the composition of tests. A Suite created in this way must be the child of a Test Unit-created Suite.

Dynamic Creation on the Host

You can further organize implicitly-created (Test Unit) Suites under parent suites you designate when you run the tests. The stride runner makes it possible to specify a "suite path" for each implicitly-created Suite into which results are to be reported. This technique is generally used where there are many test units being run, since it allows for more detailed organization and value roll-ups.

Suite attributes are shown in the table below:

Suite Attributes

Suite Attribute	Value Required	Comments
Name	Yes	The Suite is identified by its name in the test report output if the Suite corresponds to a source Test Unit its default Name value is the Test Unit name
Description	No	This string is written to the test output describing the Suite (and possibly containing links to supplemental information) if the Suite corresponds to a source Test Unit the Description value can be automatically filled in from doxygen formatted comments in source code

Test Cases

Test Cases are the individual units of pass/fail in the STRIDE Framework. Each Test Case has a single status value and an optional description. When relying on the default Framework behavior of test units, (i.e. not using dyanimic test case creation), a test case is automatically created by the Framework for each Test Method when it is executed.

Test Case attributes are shown in the table below:

Test Case Attributes

Test Case Attribute	Value Required	Comments
Name	Yes	The Test Case is identified by its name in the test report output if the Test Case corresponds to a source Test Method its default Name value is the Test Method name
Description	No	This string is written to the test output describing the Test Case (and possibly containing links to supplemental information) if the Test Case corresponds to a source Test Method the Description value can be automatically filled in
Status	Yes	The Test Case status is one of the following: pass, fail, in progress, or not applicable if the Test Case corresponds to a source Test Method the status defaults to pass, but is overridden if the test case fails for any reason.

Annotations

Annotations can be attached to Suites or Test Cases and provide supplemental information about the item they annotate. Annotations have a name, description, and a level that will be familiar to users of standard logging systems.

Annotations are added automatically to results by the Framework when using any of the Test Code Macros. Using Test Points will also create annotations in your Test Cases when they are active.

Annotation attributes are shown in the table below:

Annotation Attributes

Annotation Attribute	Value Required	Comments
Name	Yes	The Annotation is identified by its name in the test report output
Description	No	This string is written to the test output describing the Annotation Annotations that are created by Framework macros will have a description value with more detailed information about the condition that triggered the annotation.
Level	Yes	The Annotation level is one of the following: trace, debug, info, warn, error, or fatal Annotations for Suites can only be created using the Runtime API. Annotations for Test Cases can also be created using the Runtime, but they are also automatically created by various Framework macros, particularly when test failures occur.

Comments

Comments are lightweight informational entities that can be attached to Test Cases or Annotations. Comments have only a label and comment values and do not need to be uniquely named within the context of the object they belong to. Comments are not currently used by any of the Framework macros but they can be added with the Runtime API

Comment attributes are shown in the table below:

Comment Attributes

Comment Attribute	Value Required	Comments
Label	Yes	This is a simple (typically short) string label for the comment.
Comment	Yes	This string represents the body of the comment.