STRIDE Test Space: Difference between revisions
No edit summary |
No edit summary |
||
| Line 5: | Line 5: | ||
STRIDE Test Space organizes results in a hierarchy of ''projects'' and ''test spaces''. Each time you upload data to a test space, a new result set is created (unless you explicitly add your data to an existing result set). With a given result set, test results are further organized into test suites containing test cases. Any number of these organizational entities can be used to create a fluid hierarchy of test results that adapts to shifting needs during a product lifecycle. | STRIDE Test Space organizes results in a hierarchy of ''projects'' and ''test spaces''. Each time you upload data to a test space, a new result set is created (unless you explicitly add your data to an existing result set). With a given result set, test results are further organized into test suites containing test cases. Any number of these organizational entities can be used to create a fluid hierarchy of test results that adapts to shifting needs during a product lifecycle. | ||
Test Space is primarily a repository for your test results. Whether you are doing ad-hoc testing with | Test Space is primarily a repository for your test results. Whether you are doing ad-hoc testing with the STRIDE Framework or running fully-automated continuous integration of your STRIDE-enabled code base, Test Space provides a central place to store all the test data that results from the execution of the tests. Results are uploaded into specific test spaces, which allows the maintainer to control access and notifications for the results. | ||
Test Space also provides easy regression analysis in the form of baseline comparison. Users can easily create one or more fixed baseline data sets from existing results and further specify that all result sets in a test space should be compared with the fixed set of data. This is sometimes known as "gold standard" comparison and is very useful for detecting regressions in a set of stable tests. Similarly, individual test spaces can be configured to automatically compare each new result set with the previous result. This kind of comparison can also be helpful in detecting regressions in stable code bases. | Test Space also provides easy regression analysis in the form of baseline comparison. Users can easily create one or more fixed baseline data sets from existing results and further specify that all result sets in a test space should be compared with the fixed set of data. This is sometimes known as "gold standard" comparison and is very useful for detecting regressions in a set of stable tests. Similarly, individual test spaces can be configured to automatically compare each new result set with the previous result. This kind of comparison can also be helpful in detecting regressions in stable code bases. Baseline comparison data can also optionally be created so as to automatically compare execution times for test cases. | ||
Collaboration and communication are built-in to Test Space in the form of messaging and notifications. Once users are granted access to a specific test space, they have full access to view and manage test results. The test space properties can also be configured to notify all users of potential problems with new result sets - specifically regressions against baselines, any errors that were logged in the tests, and timing threshold violations (if your baseline was configured to compare test case durations). | |||
Test Space enables focused communication about test results by allowing users to create simple message threads. Messages can be associated with a test space, a result set, or even with a specific test suite. The latter can be very useful when users need to discuss specific test failures, while messages at the space or result set level might be used, for example, to discuss general trends and goals. | |||
==Uploading Results== | ==Uploading Results== | ||
Revision as of 17:14, 5 June 2009
What is STRIDE Test Space?
STRIDE Test Space is a hosted web application for storing and analyzing your test results. Test Space accepts data that results from the execution of STRIDE Test Units or Test Scripts. Data is uploaded to Test Space manually (using the web interface) or automatically from one of the STRIDE execution tools (STRIDE Runner or STRIDE Studio). Once data is uploaded, it is retained until it is manually removed or automatically deleted (depending on the space configuration).
STRIDE Test Space organizes results in a hierarchy of projects and test spaces. Each time you upload data to a test space, a new result set is created (unless you explicitly add your data to an existing result set). With a given result set, test results are further organized into test suites containing test cases. Any number of these organizational entities can be used to create a fluid hierarchy of test results that adapts to shifting needs during a product lifecycle.
Test Space is primarily a repository for your test results. Whether you are doing ad-hoc testing with the STRIDE Framework or running fully-automated continuous integration of your STRIDE-enabled code base, Test Space provides a central place to store all the test data that results from the execution of the tests. Results are uploaded into specific test spaces, which allows the maintainer to control access and notifications for the results.
Test Space also provides easy regression analysis in the form of baseline comparison. Users can easily create one or more fixed baseline data sets from existing results and further specify that all result sets in a test space should be compared with the fixed set of data. This is sometimes known as "gold standard" comparison and is very useful for detecting regressions in a set of stable tests. Similarly, individual test spaces can be configured to automatically compare each new result set with the previous result. This kind of comparison can also be helpful in detecting regressions in stable code bases. Baseline comparison data can also optionally be created so as to automatically compare execution times for test cases.
Collaboration and communication are built-in to Test Space in the form of messaging and notifications. Once users are granted access to a specific test space, they have full access to view and manage test results. The test space properties can also be configured to notify all users of potential problems with new result sets - specifically regressions against baselines, any errors that were logged in the tests, and timing threshold violations (if your baseline was configured to compare test case durations).
Test Space enables focused communication about test results by allowing users to create simple message threads. Messages can be associated with a test space, a result set, or even with a specific test suite. The latter can be very useful when users need to discuss specific test failures, while messages at the space or result set level might be used, for example, to discuss general trends and goals.
Uploading Results
Viewing Results
Baseline Comparison
Notifications
Messages
Glossary
Test Case
This is the unit of measure for test results - and single pass/fail entity. Test cases can be supplemented with additional information in the form of annotations and comments.
Test Suite
TBD
Annotation
TBD
Comments
TBD
Result Set
a result set is a collection of test suites and test cases that represent a complete set of test results for a given test space. In it's simplest form, is the output from a single execution of the STRIDE Runner for a given set of test units.
Test Space
A test space is a logical grouping of test results. Although STRIDE Test Space does not enforce any relationship between result sets, test spaces are only useful for analysis when you use them to hold sequential result set data that represent the same set of tests. That is, meaningful comparison between subsequent results sets can only be done when each result set represents execution of the same set of tests.
Each test space has properties that allow you to control who has access, if and how the results are compared to other results, whether to notify users of potential regressions, and how many result sets to keep in the space.
Project
A project is logical grouping of test spaces. Every test space must be assigned to one and only one project.
Baseline
TBD
Messages
TBD