Studio:Reporter: Difference between revisions

From STRIDE Wiki
Jump to navigation Jump to search
 
(One intermediate revision by the same user not shown)
Line 420: Line 420:
<references />
<references />


[[Category: Reference]]
[[Category: Studio]]

Latest revision as of 21:26, 18 August 2009

Introduction

The STRIDE Reporter allows you to track and report test cases for scripts, store test data in XML format, and produce HTML output for viewing of test case results.

The STRIDE Reporter tracks two primary objects: Suites and Tests , which are further described as follows:

  • A suite is a collection of tests and other suites. A suite has a name, a description and children suites and tests. Suites can also be populated with additional information in the form of annotations and can have additional summary fields added via custom data.
  • A test represents a single pass/fail item and it consists of a name, a description, a duration, and a status. Tests can be populated with additional details using comments.

Automation API


reporter

This is the root level object.  It provides directy access to the top level Suites collection as well as report generation and persistence methods.

Member
Type
Description
Load (
    string file
)
Method
This method is a shortcut to reporter.Suites.Load().
RemoveAll ()
Method
This method is a shortcut to reporter.Suites.RemoveAll().
string MakeReport (
    string file [optional],
    bool linked [optional]
)
Method
This method is a shortcut to reporter.Suites.MakeReport().


string Save (
    string file
)
Method
This method is a shortcut to reporter.Suites.Save().
ReportTemplate
Property,
string
Gets/sets the report template file to use when generating HTML reports.
Suites
Object
Top level Suites collection.


reporter.Suites

The interface for a collection of suite objects.  This collection requires uniquely named objects (case insensitive) and items can be accessed by name or numerical index.

Member Type Description
object Item (
    index
)
Method Returns the specified suite object.  The index can be integer or string. When using a string index, If the specified item is not found, this method returns null.  When using an integer index, an exception will be thrown if the requested index is out of range.
Count Property,
integer
read-only
Read only property that returns the number of suites in the collection.
object Add (
    string name [optional]
)
Method Creates a new suite in the collection.  A name may be provided, but will default to TestSuite_# where # is chosen as 1 greater than the current Count of the collection. Returns the new object or throws an exception if an object with the provided name already exists. Any name that contains path separation characters (forward or back slashes) will be treated as a suite path and trigger the creation all child suites represented by the path string.
object Remove (
    index
)
Method Removes a suite from the collection and returns a reference to the removed obejct.  The index can be integer or string.  If the specified string index is not found, nothing is removed and null is returned.  If a provided integer index is out of range, an exception is thrown.
RemoveAll() Method Empties the collection.
string MakeReport (
    string file [optional],
    bool linked [optional]
)
Method Builds an HTML report for all suites. A filename can be passed in so the HTML results can be written to that file -- otherwise, the default filename of SuiteReport.html is used. If a full path is not specified as part of the filename, the report is generated in the STRIDE installation directory. If the linked parameter is set to true (default is false), each suite will be shown as a link in the parent suite's report.
object Find (
    string name,
    bool depth [optional]
)
Method Searches for a suite with the specified name and returns the object if found, null otherwise.  If depth is set to true (the default), all sub-suites will also be searched in depth first order. If a name contains path separation characters (forward or back slash), this method attempts to find the corresponding child suites represented by the path.
string Save (
    string file [optional]
)
Method Saves the collection's data (including all children) as XML.  The persisted file is able to be read using the Load() method or can be upload to the STRIDE Portal. The file parameter, if provided, is the full system path to the file to create -- otherwise defaults to SuiteData.xml located in the STRIDE installation directory.
Load (
    string file
)
Method Loads the results from a saved XML file. This method will not clear the current data in the collection before loading the file. The file parameter is the full system path to the XML file to load.


reporter.Suites.Item

The interface for suite objects. Individual suites hold collections of several types of child objects: sub-suites, tests, annotations, and custom data fields.

Member Type Description
Suites Object Collection of sub-suites.
Tests Object Collection of Tests contained in the suite.
Name Property,
string
Gets/sets the name of the suite.  When setting the name, if another suite already exists with the same name, an exception will be thrown.
Description Property,
string
Gets/sets the description for the suite. Supports URL markdown [1].
string Save (
    string file [optional]
)
Method
Saves the suite's data (including all children) as XML. The persisted file is able to be read using the Load() method or can be upload to the STRIDE Portal. The file parameter, if provided, is the full system path to the file to create -- otherwise defaults to SuiteData.xml located in the STRIDE installation directory.
Load (
    string file
)
Method
Loads the results from a saved XML file. This method will not clear the current data in the collection before loading the file. The file parameter is the full system path to the XML file to load.
string MakeReport (
    string file [optional],
    bool linked [optional]
)
Method
Builds an HTML report for the suite. A filename can be passed in so the HTML results can be written to that file -- otherwise, the default filename of SuiteReport.html is used. If a full path is not specified as part of the filename, the report is generated in the STRIDE installation directory. If the linked parameter is set to true (default is false), each suite will be shown as a link in the parent suite's report.
Linked Property,
bool
Gets/sets the linked property of the suite, used only when generating reports.  If true, this suite will be shown as a link in the parent suite's report.
Parent Object
Returns a reference to the parent suite of the suite.  If the suite is at the root level and thus has no parent, null is returned.
Path Property,
string,
read-only
Read-only property that returns the full path of the suite, with each ancestor suite separated by a forward slash (/).
Annotations Object
Collection of annotations associated with the suite.
CustomData Object
Collection of CustomData fields associated with the suite.


reporter.Suites.Item.CustomData

The interface for a collection of custom data objects. This collection requires uniquely named objects (case insensitive) and items can be accessed by name or numerical index.

Member Type Description
object Item (
    index
)
Method Returns the specified custom data object. The index can be integer or string. When using a string index, If the specified item is not found, this method returns null. When using an integer index, an exception will be thrown if the requested index is out of range.
Count Method Read only property that returns the number of custom data fields in the collection.
object Add (
    string name [optional]
)
Method Creates a new custom data object in the collection. A name may be provided, but will default to CustomData_# where # is chosen as 1 greater than the current Count of the collection. Returns the created object or throws an exception if an item with the specified name already exists.
object Remove (
    index
)
Method Removes a custom data object from the collection and returns a reference to the removed object. The index can be integer or string. If the specified string index is not found, nothing is removed and null is returned. If a provided integer index is out of range, an exception is thrown.
RemoveAll () Method Empties the collection.


reporter.Suites.Item.CustomData.Item

The interface for a custom data field. Custom data objects allow users to extend the summary fields that are displayed for a suite.  These fields will be shown in the suite summary table following the Name, Description and Start Time fields.

Member Type Description
Name Property,
string
Gets/sets the name of the custom data field.
Value Property,
string
Gets/sets the value of the custom data field. Supports URL markdown [1].


reporter.Suites.Item.Annotations

The interface for a collection of annotations. This collection requires uniquely named objects (case insensitive) and items can be accessed by name or numerical index.

Member Type Description
object Item (
    index
)
Method
Returns the specified annotation object. The index can be integer or string. When using a string index, If the specified item is not found, this method returns null. When using an integer index, an exception will be thrown if the requested index is out of range.
Count Method Read only property that returns the number of annotations in the collection.
object Add (
    string name [optional]
)
Method Creates a new annotation in the collection. A name may be provided, but will default to Annotation_# where # is chosen as 1 greater than the current Count of the collection. Returns the new object or throws an exception if an item with the specified name already exists.
object Remove (
    index
)
Method Removes an annotation from the collection and returns a reference to the removed object. The index can be integer or string. If the specified string index is not found, nothing is removed and null is returned. If a provided integer index is out of range, an exception is thrown.
RemoveAll ()
Method Empties the collection.


reporter.Suites.Item.Annotations.Item

The interface for annotations. Annotations provide a means to attach additional information to a suite.  They consist of a name, a description, and a level.  The level corresponds to the default log levels in the log4cplus project. Annotations can optionally contain a file attachment, which is useful for bundling data related to a suite in the report (e.g. screenshot images, or log files).

Member Type Description
Name Property,
string
Gets/sets the name of the annotation. When setting the name, if another item already exists in the collection with the same name, an exception will be thrown.
Description Property,
string
Gets/sets the annotation's description.  Supports URL markdown[1].
Level Property,
string
Gets/sets the level of the annotation.  This property can be one of the following:
  • FATAL
  • ERROR
  • WARN
  • INFO
  • DEBUG
  • TRACE
File Property,
string
Gets/sets the optional file to attach to this annotation.  This can be a local system path to a file or a URL (http: or ftp:).  When this property is set, the file will be read from the source.
MIMEType Property,
string
Gets/sets the MIME type associated with the file.  This allows users to override the default MIME type detection based on extension.
FileName Property,
string
Gets/sets the filename to use for the annotation file.  This is the name of the file that will be provided back to the user when the annotation file is downloaded via the STRIDE Portal.
LinkFile Property,
bool
Gets/sets the LinkFile property for the annotation.  This property, if true, indicates that the contents of File should NOT be embedded in the reporter XML (i.e. the file should not be copied from the source).  If true, the STRIDE Portal and HTML reports will display the file annotation as an external link to the source set in the File property.  The default value is false.
Comments Object Collection of comments for the annotation.


reporter.Suites.Item.Tests

The interface for a collection of tests. This collection requires uniquely named objects (case insensitive) and items can be accessed by name or numerical index.

Member Type Description
object Item (
    index
)
Method Returns the specified test object. The index can be integer or string. When using a string index, If the specified item is not found, this method returns null. When using an integer index, an exception will be thrown if the requested index is out of range.
Count Method Read only property that returns the number of tests in the collection.
object Add (
    string name
)
Method Creates a new test in the collection. A name may be provided, but will default to TestCase_# where # is chosen as 1 greater than the current Count of the collection. Returns the new object or throws an exception if an object with the provided name already exists.
object Remove (
    index
)
Method Removes a test from the collection and returns a reference to the removed obejct. The index can be integer or string. If the specified string index is not found, nothing is removed and null is returned. If a provided integer index is out of range, an exception is thrown.
RemoveAll ()
Method Empties the collection.


reporter.Suites.Item.Tests.Item

The interface for test objects.  Tests comprise a name, a description, a duration, and a status. Tests can also have an optional metric value associated with them (and corresponding label) as well as one or more comments.

Member Type Description
Comments Object Collection of comments for this test.
Name Property,
string
Gets/sets the name of the test. When setting the name, if another item already exists in the collection with the same name, an exception will be thrown.
Description Property,
string
Gets/sets the test's description. Supports URL markdown[1].
Status Property,
string
Gets/sets the status of the annotation. This property can assigned one of the following:
  • the string values (case insensitive) "PASS"[2], "FAIL"[3], "IN PROGRESS", or "N/A"[4]
  • a boolean value of true or false
  • an integer value (0 implies PASS, nonzero implies FAIL)
Parent Object Returns the parent suite object for this test.
Path Property,
string,
read-only
Read-only property that returns the full path of the test, with each ancestor suite separated by a forward slash (/).
Duration Property,
double
Gets/sets the duration of the test in milliseconds.  If this value is not explicitly set by the user, it will default to the elapsed time from creation to the last time the Status value was updated.
Metric Property,
double
Gets/sets the secondary metric value for the test.  This value is a generic floating point value that can be associated with the test, but it has no inherent units.  An implied units label can be set using the MetricUnits property.
MetricUnits Property,
string
Gets/sets the secondary metric value label.  This value is for report display only and does not affect the actual values stored in Metric, if any.


reporter.Suites.Item.Tests.Item.Comments

The interface for a collection of comments. This collection does NOT require uniquely labeled items (multiple items can have the same label). Because of this, items in the collection can only be accessed by numerical index.

Member Type Description
object Item (
    integer index
)
Method Returns the specified comment object. Since this collection does not enforce name uniqueness, the index can be integer only. An exception will be thrown if the requested index is out of range.
Count Method Read only property that returns the number of comments in the collection.
object Add (
    string label
)
Method Creates a new comment in the collection. A label may be provided, but will default to Comment_# where # is chosen as 1 greater than the current Count of the collection. Returns the new object.
object Remove (
    integer index
)
Method Removes a comment from the collection and returns a reference to the removed obejct. The index can be integer only. If a provided integer index is out of range, an exception is thrown.
RemoveAll ()
Method Empties the collection


reporter.Suites.Item.Tests.Item.Comments.Item

The interface for comment objects.  Comments are simple child objects of tests that provide a means for adding additional information to the test case.  Comments are displayed as child rows for each test case.

Member Type Description
Label Property,
string
Gets/sets the label for the comment.
Comment Property,
string
Gets/sets the comment text for the comment object. Supports URL markdown[1].

Notes

  1. 1.0 1.1 1.2 1.3 1.4 These properties support MediaWiki markdown format for urls. Specifically, text of the form [proto://URI] (where proto can be http, https, mailto, or ftp) will be munged so that a link is displayed when viewing in a browser.
  2. "TRUE" is supported as a synonym for "PASS"
  3. "FALSE" is supported as a synonym for "FAIL"
  4. "NOT APPLICABLE" and "NOT IN USE" are also supported as synonyms for "N/A"