Studio:Test Runners

TestRun.pl

TestRun is a light-weight utility for executing the scripts and/or folders in a workspace. This utility automates the execution of scripts and/or folders in one or more workspaces. If no workspace file is specified, it will search the current directory tree and run all ssw files that are found. If no top-level folders are specified for exclusion, the entire workspace will be executed.

Usage

 TestRun.pl [--workspace=tests.ssw --exclude=myFolder]

Parameters

-w | --workspace = WorkspaceName.ssw

Sets the workspace to be opened. If no workspace is specified, the current directory is searched and all ssw files will be opened, in file-system order. Multiple workspace switches can be used to specify more than one workspace to open.

-f | --folder = path/my/folder

Sets a logical folder in studio to be executed. Multiple folder switches can be used to specify more than one folder to run.

-s | --script = myscript.pl

Sets a script file in studio to be executed. Multiple script switches can be used to specify more than one script to run.

STRIDE_TestRunner.pl

This utility automates the execution of multiple workspaces using Stride Studio's automation and testing framework. It searches in the current directory tree for all runnable files (STRIDE workspaces and .s2tc files - see below) and attempts to execute the found items. This utility is designed for robust execution and can continue even when errors occur in the execution. Workspace timeouts are monitored and the entire STRIDE infrastructure is restarted if workspaces fail to complete execution in the specified amount of time. STRIDE_TestRunner maintains the test results and provides options to control uploading the results to a STRIDE Portal. There is also preliminary support for executing stand-alone scripts - currently only perl, jscript, vbscript and batch files are supported (.pl, .js, .vbs, and .bat respectively).

Usage

 STRIDE_TestRunner.pl [OPTIONS]

PREREQUISITES

   This script uses several non-standard modules - they are:

       Win32
       Win32::API
       Win32::Process
       XML::DOM
       S2S::TestUtils
       S2S::PortalUtils

   You will need to have these modules installed and working before running
   this script. The script has only be tested with ActiveState's perl
   distribution, therefore we recommend that you use any current perl
   release from ActiveState. If you are using an ActiveState distribution,
   you can use ppm (perl package manager) to install these required
   packages. The S2S::TestUtils package should be found automatically by
   the script as long ad STRIDE is properly installed.

   If you intend to use the --screenshot option (see below), you will also
   need to install the ImageMagick libraries and the Win32::Screenshot
   module. A binary distribution of the ImageMagick libraries is available
   at http://www.imagemagick.org/script/binary-releases.php. When
   installing the ImageMagick distribution, be sure to select the option to
   install the perl modules and the development headers (we recommend just
   installing everything in the distribution). The Win32::Screenshot
   package can be obtained from CPAN (www.cpan.org). Win32::Screenshot does
   require one source file to be compiled, so you must have a working
   development environment on your machine (VisualStudio or the
   VisualStudio .NET Toolkit, e.g.) when you make/install it.

STANDALONE SCRIPT EXECUTION

   This application contains provisional support for standalone script
   execution. In addition to looking for all STRIDE Studio workspaces to
   execute (files with .ssw extension), we also look for files with an
   .s2tc extension (S2 test configuration files).

   These s2tc files are expected to contain xml of the following form:

       <?xml version="1.0" encoding="UTF-8"?>
       <StrideTestConfiguration version="1.0">
           <file path="test.js" command="cmd to exec" command_arguments="addl args" script_arguments="addl args" timeout="20"/>
           ...
       </StrideTestConfiguration>

 FILE NODE ATTRIBUTES
   path="script.pl"
       This is the only attribute that is required as it indicates what
       script file to attempt to run. This file can be specified as a full
       system path (absolute path) or a relative path (relative to the
       location of the .s2tc file).

   command="C:\path\to\script\engine.exe"
       Optional command to use to execute the given script file. If no
       command is explicitly specified, we will attempt to infer the
       interpreter to use from the extension of the script file given in
       the path attribute. Currently we only can infer the interpreter for
       perl, jscript, vbscript and batch files.

   command_arguments="args"
       Optional additional arguments to specify to the command (script
       engine). These arguments will be passed to the script engine before
       the script file.

   script_arguments="args"
       Optional additional arguments to pass to the script. These arguments
       will be passed to the script file before the final argument which is
       the path of the current s2tc file in the test hierarchy. This test
       hierarchy path is automatically passed to every script by the
       framework.

   timeout="10"
       Optional timeout value (in seconds). Each script will be allowed to
       run for the specified timeout (or a default of 10 minutes) and will
       be killed if it runs for longer than the timeout.

OPTIONS

   --reportfile="reportfilename.html"
       Overrides the default report file name (TestResults.html). If you
       specify a full path, it will be honored, otherwise the file will be
       created in a time-stamped subdirectory of the working (starting)
       directory.

   --datafile="datafilename.xml"
       Overrides the default data file name (TestData.xml). If you specify
       a full path, it will be honored, otherwise the file will be created
       in a time-stamped subdirectory of the working (starting) directory).

   --directory="some\dir"
       Specifies which directory to start running workspaces in. Defaults
       to the current directory.

   --file="some\file.ssw"
       Specifies a specific file to execute. If this option is specified,
       only the specified file will be executed. This option only supports
       runnable file types: stride workspaces and s2tc files, currently.

   --resultdir="some\dir"
       Specifies an alternate directory for the test results (data and
       reports). Defaults to a time-stamped directory under the current
       directory.

   --database="C:\\workspaces\\master\\master.sidb"
       Allows user to specify a single database to use for all workspaces.
       The full path to a database file must be given. If this option is
       not used, the default database is used for each workspace.

   --filter=path\to\filter.pl
       Specifies a perl script file that will be used at runtime to further
       filter the file-set that is executed. If this script is specified
       and exists, for each candidate workspace file that the script finds,
       it will attempt to call the UseFile function in that script (the
       script is loaded using 'require' runtime loading). If the UseFile
       function exists and returns 0, the file will be skipped - otherwise
       the workspace will be executed.

   --timeout=n
       Allows user to specify a default workspace timeout value to use (in
       seconds). This value is only used for any workspace that does not
       already have a specified timeout value. Specify a value of 0 if you
       want this script to wait forever on workspaces.

   --cleanup-delay=n
       Allows user to specify the time to wait after each workspace
       exucution before checking for processes to cleanup. If this is not
       specified, a default delay of 2 seconds will be used.

   --nocontinue
       Indicates that the script should cease execution upon a timeout
       error. The default behavior is to report timeout errors and
       continue.

   --watchlog
       Indicates that the script should check the StrideErrLog.txt file for
       any error messages after each workspace and add any error messages
       found to the report.

   --full-restart
       Causes all relevant S2 processes to be shut-down in between each
       workspaces that is executed. The default behavior is that we leave
       the S2 Reporter and Host processes running between tests. This
       option assures the cleanest possible process environment at the
       start of each test at the expense of some time lost in between each
       test as the reporter results are saved and reloaded.

   --screenshot
       Causes the script to generate screen shot images whenever the
       workspace exceeds the allowable run time (timeout value). This will
       allow you to see any messages being displayed by S2 executables
       prior to being killed. Please note the special installation
       requirements in PREREQUISITES above.

   --portal-location="servername"
       Specifies a resolvable address of a valid STRIDE Portal instance (a
       webserver running the STRIDE Portal software). Defaults to
       'localhost'.

   --portal-path="/some/path"
       Specifies the portal path to use when uploading results to the
       STRIDE Portal.

   --portal-space="myspace"
       Specifies the portal space name to use when posting results to the
       STRIDE Portal. Defaults to public.

   --portal-user="username"
       Specifies the portal user name to use when posting results to the
       STRIDE Portal. Defaults to public.

   --portal-pass="password"
       Specifies the portal password to use when posting results to the
       STRIDE Portal. Defaults to public.

   --noportal
       Causes the script to skip the uploading of test results. The default
       behavior is to attempt to upload results whenever they are created.