Training Runtime API

Objectives

This Training Module is focused leveraging the Runtime APIs in the context of writing a test. The module covers the following topics:

• How to set test status
• Setting test case name and description directly via the API
• Setting test suite name and description directly via the API
• Dynamically creating test suites and test cases

There are two new files used -- TestRuntime.cpp & TestRuntime.h --- that implement two test Units:

• TestRuntime_Static
• TestRuntime_Dynamic

All of the Test Units have test cases already implemented (used for reference) and have one test method that you are required to implement: one is called Exercise and the other is called dynamic_Exercise. Currently the exercise methods return a NOT IN USE status.

Instructions

Build and Run TestApp

• Build TestApp using SDK makefile
• Startup TestApp
• If not already done, create an option file (myoptions.txt) using the following content (Windows example)

 ##### Command Line Options ######
 --device "TCP:localhost:8000"
 --database %STRIDE_DIR%\SDK\Windows\out\TestApp.sidb
 --output %STRIDE_DIR%\SDK\Windows\sample_src\TestApp.xml
 --log_level all

• Execute Test Runtime APIs Test Units only

 > stride -O myoptions.txt --run TestRuntime_Static TestRuntime_Dynamic

 Loading database...
 Connecting to device...
 Executing...
  test unit "TestRuntime_Static"
    > 2 passed, 0 failed, 0 in progress, 1 not in use.
  test unit "TestRuntime_Dynamic"
    > 4 passed, 0 failed, 0 in progress, 1 not in use.
 ------------------------------------------------------------
 Summary: 6 passed, 0 failed, 0 in progress, 2 not in use.
 
 Disconnecting from device...
 Saving result file...

• Review the details of the test results using a Browser. Open TestApp.xml which can be found in the sample_src directory (based on the output option). By opening the xml file in a web browser the xsl is automatically applied to create html.

Implement Exercise

• TestRuntime_Static::Exercise
  • Validate the sut_mult() routine using some simple data input
  • Set the test method name to Mult
  • Use direct Runtime APIs to:
    • Set the test case description
    • Capture logging information via comments
    • Set the status of the test case

• TestRuntime_Dynamic->Exercise
  • Pass in 5 via command line for the number of test cases to generate
  • Check using srEXIT_XX that the number of test cases has been passed in correctly
  • Add a new Test Suite using Exercise for the name
  • Write a loop generating a dynamic test case using NumberOfTestCases
    • Test Case name shall be Test_n where "n" is the loop count
    • Add a description for each test case
    • Validate sut_foo() using the loop count

• note- The dynamic Test Unit is using CClass packaging

• Execute Test Runtime API Test Units only (NOTE - requires passing parameter)

 > stride -O myoptions.txt --run TestRuntime_Static TestRuntime_Dynamic(5)

 Loading database...
 Connecting to device...
 Executing...
   test unit "TestRuntime_Static"
     > 3 passed, 0 failed, 0 in progress, 0 not in use.
   test unit "TestRuntime_Dynamic"
     > 10 passed, 0 failed, 0 in progress, 0 not in use.
   ---------------------------------------------------------------------
   Summary: 13 passed, 0 failed, 0 in progress, 0 not in use.

 Disconnecting from device...
 Saving result file...

Run and Publish Results

When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following if not already done:

 #### Test Space options (partial) #####
 #### Note - make sure to change username, etc. ####
 --testspace https://username:password@yourcompany.stridetestspace.com
 --project Training
 --name YOURNAME

  > stride -O myoptions.txt --run TestRuntime_Static TestRuntime_Dynamic --space TestRuntime --upload

Note: This space has been set up with a Baseline of expected test results that you can use to validate your results.

Reference

The following reference information is related to passing parameters to Test Units.

Wiki

• Runtime Test Services with special attention to the following:
  • Review srTestCaseSetDescription()
  • Review srTestCaseSetStatus()
  • Review srTestCaseSetName()
  • Review srTestCaseAddComment()
  • Review srTestSuiteAddCase()
  • Review srTestSuiteAddSuite()
  • Review srTestSuiteSetDescription()
  • Review srTestSuiteSetName()

Samples

• Test Class Sample - Specifically the Runtime Services Test Units
• Test C Class Sample - Specifically the Runtime Services Test Units
• Test Function List Sample - Specifically the Runtime Services Test Units