Organizing Tests into Suites

Overview

Unless you specify otherwise, stride will publish results from all of your test units into a single top-level test suite when run. This is a satisfactory arrangement as long as the number of tests is relatively small. As the number of test units grows, or test units from disparate groups are aggregated (as in a Continuous Integration arrangement) this flat report organization becomes hard to use.

The software under test itself is typically broken up into separate units (e.g. libraries, components, objects, etc.) and organized into a hierarchy that reflects the functional organization of the software. If the results from STRIDE tests could be organized identically to the software under test, ****.

• Subtotals of Time Under Test, Pass/Fail and Error counts are provided for each suite

Organizing Into Suites at Test Runtime

The stride test runner allows you to specify a "path" under which a test unit's results will be published. The path is a forward-slash delimited list of suite names. The path is optionally specified in the argument to the -r command line option.

Following are some examples. Many more examples can be seen here:.

[-r omitted]

If you omit the -r argument, stride will run all of the test units specified in the database and publish them into a single top-level root test suite

-r /(TestUnit1 TestUnit2)

Here we specify two test units to be run and published to the root suite

-r /suite1(TestUnit1) -r /suite2(TestUnit2)

Here the two test units are each published to their own suite. (Note that -r can be specified any number of times on a stride command line.)

-r /suite1/suite1.1/suite1.1.1(TestUnit1 TestUnit2)

Here, the two test units are published to a suite that is several levels deep in a hierarchy. Note that stride will create suites as needed to fulfill your publishing requirements.

Techniques for Organization

• Name your test units such that their organization is implicit

level1_level2_level3_name

• Have each subgroup manage their own file.

OrganizeIntoSuites.pl

#!/usr/bin/perl
use strict;
use warnings;

# set this to your separator character or string
my $token = '::';

while( <> ) {
    chomp;
    my @segments = split(/$token/);
    # remove the last segment from the array
    my $testunit = pop @segments;
    print "-r /";
    foreach my $segment (@segments) {
        print "$segment/";
    }
    print "($_)\n";
}

Batch File or Shell Script

stride --list --database TestApp.sidb | perl OrganizeIntoSuites.pl > TestsInSuites.txt
stride --database TestApp.sidb --device TCP:localhost:8000 -O TestsInSuites.txt