STRIDE Test Space

What is STRIDE Test Space?

STRIDE Test Space is a hosted web application for storing and analyzing your test results. Test Space accepts data that results from the execution of STRIDE Test Units or Test Scripts. Data is uploaded to Test Space manually (using the web interface) or automatically from one of the STRIDE execution tools (STRIDE Runner or STRIDE Studio). Once data is uploaded, it is retained until it is manually removed or automatically deleted (depending on the space configuration).

STRIDE Test Space organizes results in a hierarchy of projects, test spaces and result sets. Each time you upload data to a test space, a new result set is created (unless you explicitly add your data to an existing one). With a given result set, test results are further organized into test suites containing test cases. Any number of these organizational entities can be used to create a fluid hierarchy of test results that adapts to shifting needs during a product lifecycle.

Test Space is primarily a repository for your test results. Whether you are doing ad-hoc testing with the STRIDE Framework or running fully-automated continuous integration of your STRIDE-enabled code base, Test Space provides a central place to store all the test data that is produced by the tests. Results are uploaded into specific test spaces, which allows the maintainer to control access and notifications for the results.

Test Space also provides easy regression analysis in the form of baseline comparison. Users can create one or more fixed baseline data sets from existing results and further specify that all result sets in a test space should be compared with the fixed set of data. This is sometimes known as "gold standard" comparison and is very useful for detecting regressions in a set of stable tests. Similarly, individual test spaces can be configured to automatically compare each new result set with the previous result. This kind of comparison can also be helpful in detecting regressions in stable code bases. Baseline comparison data can also optionally include timing thresholds so as to enable automatic comparison of test case durations.

Collaboration and communication are built-in to Test Space in the form of messaging and notifications. Once users are granted access to a specific test space, they have full access to view and manage test results. The test space properties can also be configured to notify all users of potential problems with new result sets - specifically regressions against baselines, log errors, and timing threshold violations (if your baseline was configured to compare durations).

Test Space enables focused communication about test results by allowing users to create simple message threads. Messages can be associated with a test space, a result set, or even with a specific test suite. The latter can be very useful when users need to discuss specific test failures, while messages at the space or result set level might be used, for example, to discuss general trends and goals.

Uploading Results

New results set can be easily uploaded using the automatic upload features in the STRIDE Runner or STRIDE Studio. You can also upload XML results data files manually using the web interface by navigating to the detail view for a specific test case and clicking the upload data button.

Viewing Results

STRIDE Test Space presents several distinct views of your data.

Glossary

Test Case

This is the unit of measure for test results - and single pass/fail entity. Test cases can be supplemented with additional information in the form of annotations and comments.

Test Suite

TBD

Annotation

TBD

Comments

TBD

Result Set

a result set is a collection of test suites and test cases that represent a complete set of test results for a given test space. In it's simplest form, is the output from a single execution of the STRIDE Runner for a given set of test units.

Test Space

A test space is a logical grouping of test results. Although STRIDE Test Space does not enforce any relationship between result sets, test spaces are only useful for analysis when you use them to hold sequential result set data that represent the same set of tests. That is, meaningful comparison between subsequent results sets can only be done when each result set represents execution of the same set of tests.

Each test space has properties that allow you to control who has access, if and how the results are compared to other results, whether to notify users of potential regressions, and how many result sets to keep in the space.

Project

A project is logical grouping of test spaces. Every test space must be assigned to one and only one project.

Baseline

TBD

Messages

TBD