Creating And Using Baselines

Introduction

Baselines represent expections of test results. Basline data consists of suites and test cases that are used for comparing to other result sets to determine the quality of the results. In a simple test-bed that has only passing tests, baselines are not typically needed since any test failure can be considered a regression - thus regression is easy to determine by simple inspection. As test coverage becomes more extensive, however, it's inevitable that some test failures will be deemed acceptable, at least for a defined period of time (say for an upcoming release cycle). In these cases, it's necessary to define a baseline that includes the desired minimum results. This baseline data can then be compared with each new result set to quickly determine if any result sets are different than expected.

Types

STRIDE Testspace supports two basic types of baseline comparison - fixed and sequential.

Sequential baseline comparison simply compares each result set to the previous result set (previous based on upload order).

How the comparison is computed

ETC