Studio:Reporter: Difference between revisions
No edit summary |
m (Reporter moved to Studio:Reporter) |
||
(2 intermediate revisions by 2 users not shown) | |||
Line 257: | Line 257: | ||
*FATAL | *FATAL | ||
*ERROR | *ERROR | ||
* | *WARN | ||
*INFO | *INFO | ||
*DEBUG | *DEBUG | ||
Line 420: | Line 420: | ||
<references /> | <references /> | ||
[[Category: | [[Category: Studio]] |
Latest revision as of 21:26, 18 August 2009
Introduction
The STRIDE Reporter allows you to track and report test cases for scripts, store test data in XML format, and produce HTML output for viewing of test case results.
The STRIDE Reporter tracks two primary objects: Suites and Tests , which are further described as follows:
- A suite is a collection of tests and other suites. A suite has a name, a description and children suites and tests. Suites can also be populated with additional information in the form of annotations and can have additional summary fields added via custom data.
- A test represents a single pass/fail item and it consists of a name, a description, a duration, and a status. Tests can be populated with additional details using comments.
Automation API
reporter
This is the root level object. It provides directy access to the top level Suites collection as well as report generation and persistence methods.
Member |
Type |
Description |
Load ( string file ) |
Method |
This method is a shortcut to reporter.Suites.Load(). |
RemoveAll () |
Method |
This method is a shortcut to reporter.Suites.RemoveAll(). |
string MakeReport ( string file [optional], bool linked [optional] ) |
Method |
This method is a shortcut to reporter.Suites.MakeReport().
|
string Save ( string file ) |
Method |
This method is a shortcut to reporter.Suites.Save(). |
ReportTemplate |
Property, string |
Gets/sets the report template file to use when generating HTML reports. |
Suites |
Object |
Top level Suites collection. |
reporter.Suites
The interface for a collection of suite objects. This collection requires uniquely named objects (case insensitive) and items can be accessed by name or numerical index.
Member | Type | Description |
object Item ( index ) |
Method | Returns the specified suite object. The index can be integer or string. When using a string index, If the specified item is not found, this method returns null. When using an integer index, an exception will be thrown if the requested index is out of range. |
Count | Property, integer read-only |
Read only property that returns the number of suites in the collection. |
object Add ( string name [optional] ) |
Method | Creates a new suite in the collection. A name may be provided, but will default to TestSuite_# where # is chosen as 1 greater than the current Count of the collection. Returns the new object or throws an exception if an object with the provided name already exists. Any name that contains path separation characters (forward or back slashes) will be treated as a suite path and trigger the creation all child suites represented by the path string. |
object Remove ( index ) |
Method | Removes a suite from the collection and returns a reference to the removed obejct. The index can be integer or string. If the specified string index is not found, nothing is removed and null is returned. If a provided integer index is out of range, an exception is thrown. |
RemoveAll() | Method | Empties the collection. |
string MakeReport ( string file [optional], bool linked [optional] ) |
Method | Builds an HTML report for all suites. A filename can be passed in so the HTML results can be written to that file -- otherwise, the default filename of SuiteReport.html is used. If a full path is not specified as part of the filename, the report is generated in the STRIDE installation directory. If the linked parameter is set to true (default is false), each suite will be shown as a link in the parent suite's report. |
object Find ( string name, bool depth [optional] ) |
Method | Searches for a suite with the specified name and returns the object if found, null otherwise. If depth is set to true (the default), all sub-suites will also be searched in depth first order. If a name contains path separation characters (forward or back slash), this method attempts to find the corresponding child suites represented by the path. |
string Save ( string file [optional] ) |
Method | Saves the collection's data (including all children) as XML. The persisted file is able to be read using the Load() method or can be upload to the STRIDE Portal. The file parameter, if provided, is the full system path to the file to create -- otherwise defaults to SuiteData.xml located in the STRIDE installation directory. |
Load ( string file ) |
Method | Loads the results from a saved XML file. This method will not clear the current data in the collection before loading the file. The file parameter is the full system path to the XML file to load. |
reporter.Suites.Item
The interface for suite objects. Individual suites hold collections of several types of child objects: sub-suites, tests, annotations, and custom data fields.
Member | Type | Description |
Suites | Object | Collection of sub-suites. |
Tests | Object | Collection of Tests contained in the suite. |
Name | Property, string |
Gets/sets the name of the suite. When setting the name, if another suite already exists with the same name, an exception will be thrown. |
Description | Property, string |
Gets/sets the description for the suite. Supports URL markdown [1]. |
string Save ( string file [optional] ) |
Method |
Saves the suite's data (including all children) as XML. The persisted file is able to be read using the Load() method or can be upload to the STRIDE Portal. The file parameter, if provided, is the full system path to the file to create -- otherwise defaults to SuiteData.xml located in the STRIDE installation directory. |
Load ( string file ) |
Method |
Loads the results from a saved XML file. This method will not clear the current data in the collection before loading the file. The file parameter is the full system path to the XML file to load. |
string MakeReport ( string file [optional], bool linked [optional] ) |
Method |
Builds an HTML report for the suite. A filename can be passed in so the HTML results can be written to that file -- otherwise, the default filename of SuiteReport.html is used. If a full path is not specified as part of the filename, the report is generated in the STRIDE installation directory. If the linked parameter is set to true (default is false), each suite will be shown as a link in the parent suite's report. |
Linked | Property, bool |
Gets/sets the linked property of the suite, used only when generating reports. If true, this suite will be shown as a link in the parent suite's report. |
Parent | Object |
Returns a reference to the parent suite of the suite. If the suite is at the root level and thus has no parent, null is returned. |
Path | Property, string, read-only |
Read-only property that returns the full path of the suite, with each ancestor suite separated by a forward slash (/). |
Annotations | Object |
Collection of annotations associated with the suite. |
CustomData | Object |
Collection of CustomData fields associated with the suite. |
reporter.Suites.Item.CustomData
The interface for a collection of custom data objects. This collection requires uniquely named objects (case insensitive) and items can be accessed by name or numerical index.
Member | Type | Description |
object Item ( index ) |
Method | Returns the specified custom data object. The index can be integer or string. When using a string index, If the specified item is not found, this method returns null. When using an integer index, an exception will be thrown if the requested index is out of range. |
Count | Method | Read only property that returns the number of custom data fields in the collection. |
object Add ( string name [optional] ) |
Method | Creates a new custom data object in the collection. A name may be provided, but will default to CustomData_# where # is chosen as 1 greater than the current Count of the collection. Returns the created object or throws an exception if an item with the specified name already exists. |
object Remove ( index ) |
Method | Removes a custom data object from the collection and returns a reference to the removed object. The index can be integer or string. If the specified string index is not found, nothing is removed and null is returned. If a provided integer index is out of range, an exception is thrown. |
RemoveAll () | Method | Empties the collection. |
reporter.Suites.Item.CustomData.Item
The interface for a custom data field. Custom data objects allow users to extend the summary fields that are displayed for a suite. These fields will be shown in the suite summary table following the Name, Description and Start Time fields.
Member | Type | Description |
Name | Property, string |
Gets/sets the name of the custom data field. |
Value | Property, string |
Gets/sets the value of the custom data field. Supports URL markdown [1]. |
reporter.Suites.Item.Annotations
The interface for a collection of annotations. This collection requires uniquely named objects (case insensitive) and items can be accessed by name or numerical index.
Member | Type | Description |
object Item ( index ) |
Method |
Returns the specified annotation object. The index can be integer or string. When using a string index, If the specified item is not found, this method returns null. When using an integer index, an exception will be thrown if the requested index is out of range. |
Count | Method | Read only property that returns the number of annotations in the collection. |
object Add ( string name [optional] ) |
Method | Creates a new annotation in the collection. A name may be provided, but will default to Annotation_# where # is chosen as 1 greater than the current Count of the collection. Returns the new object or throws an exception if an item with the specified name already exists. |
object Remove ( index ) |
Method | Removes an annotation from the collection and returns a reference to the removed object. The index can be integer or string. If the specified string index is not found, nothing is removed and null is returned. If a provided integer index is out of range, an exception is thrown. |
RemoveAll () |
Method | Empties the collection. |
reporter.Suites.Item.Annotations.Item
The interface for annotations. Annotations provide a means to attach additional information to a suite. They consist of a name, a description, and a level. The level corresponds to the default log levels in the log4cplus project. Annotations can optionally contain a file attachment, which is useful for bundling data related to a suite in the report (e.g. screenshot images, or log files).
Member | Type | Description |
Name | Property, string |
Gets/sets the name of the annotation. When setting the name, if another item already exists in the collection with the same name, an exception will be thrown. |
Description | Property, string |
Gets/sets the annotation's description. Supports URL markdown[1]. |
Level | Property, string |
Gets/sets the level of the annotation. This property can be one of the following:
|
File | Property, string |
Gets/sets the optional file to attach to this annotation. This can be a local system path to a file or a URL (http: or ftp:). When this property is set, the file will be read from the source. |
MIMEType | Property, string |
Gets/sets the MIME type associated with the file. This allows users to override the default MIME type detection based on extension. |
FileName | Property, string |
Gets/sets the filename to use for the annotation file. This is the name of the file that will be provided back to the user when the annotation file is downloaded via the STRIDE Portal. |
LinkFile | Property, bool |
Gets/sets the LinkFile property for the annotation. This property, if true, indicates that the contents of File should NOT be embedded in the reporter XML (i.e. the file should not be copied from the source). If true, the STRIDE Portal and HTML reports will display the file annotation as an external link to the source set in the File property. The default value is false. |
Comments | Object | Collection of comments for the annotation. |
reporter.Suites.Item.Tests
The interface for a collection of tests. This collection requires uniquely named objects (case insensitive) and items can be accessed by name or numerical index.
Member | Type | Description |
object Item ( index ) |
Method | Returns the specified test object. The index can be integer or string. When using a string index, If the specified item is not found, this method returns null. When using an integer index, an exception will be thrown if the requested index is out of range. |
Count | Method | Read only property that returns the number of tests in the collection. |
object Add ( string name ) |
Method | Creates a new test in the collection. A name may be provided, but will default to TestCase_# where # is chosen as 1 greater than the current Count of the collection. Returns the new object or throws an exception if an object with the provided name already exists. |
object Remove ( index ) |
Method | Removes a test from the collection and returns a reference to the removed obejct. The index can be integer or string. If the specified string index is not found, nothing is removed and null is returned. If a provided integer index is out of range, an exception is thrown. |
RemoveAll () |
Method | Empties the collection. |
reporter.Suites.Item.Tests.Item
The interface for test objects. Tests comprise a name, a description, a duration, and a status. Tests can also have an optional metric value associated with them (and corresponding label) as well as one or more comments.
Member | Type | Description |
Comments | Object | Collection of comments for this test. |
Name | Property, string |
Gets/sets the name of the test. When setting the name, if another item already exists in the collection with the same name, an exception will be thrown. |
Description | Property, string |
Gets/sets the test's description. Supports URL markdown[1]. |
Status | Property, string |
Gets/sets the status of the annotation. This property can assigned one of the following: |
Parent | Object | Returns the parent suite object for this test. |
Path | Property, string, read-only |
Read-only property that returns the full path of the test, with each ancestor suite separated by a forward slash (/). |
Duration | Property, double |
Gets/sets the duration of the test in milliseconds. If this value is not explicitly set by the user, it will default to the elapsed time from creation to the last time the Status value was updated. |
Metric | Property, double |
Gets/sets the secondary metric value for the test. This value is a generic floating point value that can be associated with the test, but it has no inherent units. An implied units label can be set using the MetricUnits property. |
MetricUnits | Property, string |
Gets/sets the secondary metric value label. This value is for report display only and does not affect the actual values stored in Metric, if any. |
reporter.Suites.Item.Tests.Item.Comments
The interface for a collection of comments. This collection does NOT require uniquely labeled items (multiple items can have the same label). Because of this, items in the collection can only be accessed by numerical index.
Member | Type | Description |
object Item ( integer index ) |
Method | Returns the specified comment object. Since this collection does not enforce name uniqueness, the index can be integer only. An exception will be thrown if the requested index is out of range. |
Count | Method | Read only property that returns the number of comments in the collection. |
object Add ( string label ) |
Method | Creates a new comment in the collection. A label may be provided, but will default to Comment_# where # is chosen as 1 greater than the current Count of the collection. Returns the new object. |
object Remove ( integer index ) |
Method | Removes a comment from the collection and returns a reference to the removed obejct. The index can be integer only. If a provided integer index is out of range, an exception is thrown. |
RemoveAll () |
Method | Empties the collection |
reporter.Suites.Item.Tests.Item.Comments.Item
The interface for comment objects. Comments are simple child objects of tests that provide a means for adding additional information to the test case. Comments are displayed as child rows for each test case.
Member | Type | Description |
Label | Property, string |
Gets/sets the label for the comment. |
Comment | Property, string |
Gets/sets the comment text for the comment object. Supports URL markdown[1]. |
Notes
- ↑ 1.0 1.1 1.2 1.3 1.4 These properties support MediaWiki markdown format for urls. Specifically, text of the form [proto://URI] (where proto can be http, https, mailto, or ftp) will be munged so that a link is displayed when viewing in a browser.
- ↑ "TRUE" is supported as a synonym for "PASS"
- ↑ "FALSE" is supported as a synonym for "FAIL"
- ↑ "NOT APPLICABLE" and "NOT IN USE" are also supported as synonyms for "N/A"