Studio:Test Spec Template

This document describes the details of the test design such that test assets can be created by Test Engineers.

Workspace Organization

<
How will the test workspace be organized into test suites, test files, and test cases?

How will test case implementations tie back to their specific test spec? For instance:
>

Workspace Naming Conventions

<
What naming conventions will be used to aid in organization and ensure uniqueness of test asset names? this includes:

• Workspace folder names (test suite names)
• Workspace file names
• Test function/subroutine names
• Test case names

>

Mapping Test Items to Basecamp

Suggested mapping is to use To-Do Lists to represent test suites, and the To_Do items within them to represent their component test cases (or test files).

Test Suite Specifications

Any test suite-related information should be included in the corresponding Basecamp To-Do list description.

If needed, create a Basecamp message and provide a link to in in the To-Do list description, or create specific to-do items if Test Engineer activity is needed to set up suite setup/teardown.

Specifications for Individual Tests

Specifications for individual tests should be documented in comments added to the relevant Basecamp To-Do item. Where possible, include the complete specification/instructions in a single comment.

Note that you can apply a lot of formatting on these comment pages (info here). (Unfortunately, the markup is different from wiki markup.)

A disadvantage of using a To-Do comment here is that existing comments can't be edited. However, you can simulate editing by adding a new comment with all of the updated information, then deleting the existing comment. An advantage over the wiki here is that Basecamp will email a change notification automatically to affected participants.

Common Spec Information

Where you have specification information that is common across several test specs, create a Basecamp Message to hold this information and provide a link in the relevant To-Do comments.

These messages can be edited as needed and notification emails can be automatically sent.

Individual Spec Contents

The goal of each Basecamp test spec is to contain enough information to allow a competent Test Engineer to create the target test asset. Different tests will require different sets of information, and it is up to the author of the test spec to judge what's best here. Experience has shown that a "one size fits all" approach to test specs doesn't work, however one should strive for consistency between a project's individual test specs to the extent it is possible.

Following is an outline of topics that you may want to consider including in the test specs that you write:

References

<
Link to relevant documentation or source code that is specific to this test specification.
>

Test Strategy / Approach

<
Describe the test strategy for this specification if it isn't straightforward or obvious.

Information may include:

• Testing technologies to be used ( target test code, remote functions, trace views, etc. )
• SCL extensions to be used ( wrapper or helper functions, broadcast events, etc)
• Metrics and annotations to be collected
• Coverage goals

>

Test Case Breakdown

<
Provide a breakdown of the test cases to be covered by this specification. Specify the name and a brief description for each case listed.
>

Pass / Fail Criteria

<
Define the pass / fail criteria for each of the items to be tested if not obvious.
>

Assumptions and Pre-test Conditions

<
Specify all assumptions made or pre-test conditions required for the items to be tested. List requirements that are not explicity handled by Test Suite Setup

For example:

• Build or platform configuration
• Initial device or component state
• External test equipment
• Resource dependencies such as media files or available memory

>

Post-test Conditions

<
Specify all post-test conditions required to ensure that other follow on tests are not affected. List requirements that are not explicitly handled by Test Suite Teardown
>

Suspension Criteria and Resumption Requirements

<
Specify what constitutes stoppage for a test or the suite of tests, and what failures are acceptable and should allow testing to proceed.

Testing after a truly fatal error will generate conditions that may be identified as failures but are in fact ghost errors caused by the earlier failures that were ignored.
>