Training Runtime API: Difference between revisions

From STRIDE Wiki
Jump to navigation Jump to search
 
(6 intermediate revisions by 2 users not shown)
Line 3: Line 3:
This Training Module is focused leveraging the [[Runtime_Test_Services | Runtime APIs]] in the context of writing a test. The module covers the following topics:
This Training Module is focused leveraging the [[Runtime_Test_Services | Runtime APIs]] in the context of writing a test. The module covers the following topics:
* How to set [[Runtime_Test_Services#srTestCaseSetStatus | test status]]
* How to set [[Runtime_Test_Services#srTestCaseSetStatus | test status]]
* Setting [[Runtime_Test_Services#srTestCaseSetName | test case name]] and [[Runtime_Test_Services#srTestCaseSetDescription | description]] directly via the API
* Adding [[Runtime_Test_Services#srTestCaseAddAnnotation | test annotations]] and attaching [[Runtime_Test_Services#srTestAnnotationAddComment | comments]] to them
* Setting [[Runtime_Test_Services#srTestSuiteSetName | test suite name]] and [[Runtime_Test_Services#srTestSuiteSetDescription | description]] directly via the API
* Dynamically creating [[Runtime_Test_Services#srTestSuiteAddCase | test cases]]
* Dynamically creating [[Runtime_Test_Services#srTestSuiteAddSuite | test suites]] and [[Runtime_Test_Services#srTestSuiteAddCase | test cases]]




Line 21: Line 20:
* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
* Startup TestApp
* Startup TestApp
* If not already done, create an [[Stride_Runner#Options | option file]] (myoptions.txt) using the following content (Windows example)
* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]  
 
  ##### Command Line Options ######
  --device "TCP:localhost:8000"
  --database "%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb"
  --output "%STRIDE_DIR%\SDK\Windows\sample_src\TestApp.xml"
  --log_level all


* Execute ''Test Runtime APIs'' Test Units only  
* Execute ''Test Runtime APIs'' Test Units only  
Line 52: Line 45:
* '''TestRuntime_Static::Exercise'''
* '''TestRuntime_Static::Exercise'''
** Validate the ''sut_mult()'' routine using some simple data input
** Validate the ''sut_mult()'' routine using some simple data input
*** ''Hint:'' You will probably want to read about [[Runtime Test Services]]
** Set the ''test method'' name to '''Mult'''
** Set the ''test method'' name to '''Mult'''
** Use direct Runtime APIs to:
** Use direct Runtime APIs to:
Line 90: Line 84:
=== Run and Publish Results ===
=== Run and Publish Results ===


When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following if not already done:
When you have completed the Exercise(s) publish your results to Test Space. If you have not added test space options to your options file (<tt>myoptions.txt</tt>) please see [[Training_Getting_Started#Test_Space_Access| testspace access]].
 
  #### Test Space options (partial) #####
  #### Note - make sure to change username, etc. ####
  --testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
  --project Training
  --name YOURNAME


   > stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic --space TestRuntime --upload
   > stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic --space TestRuntime --upload
Line 108: Line 96:


* [[Runtime_Test_Services#C_Test_Functions | Runtime Test Services]] with special attention to the following:
* [[Runtime_Test_Services#C_Test_Functions | Runtime Test Services]] with special attention to the following:
** Review srTestCaseSetDescription()
** Review srTestCaseSetStatus()
** Review srTestCaseSetStatus()
** Review srTestCaseSetName()
** Review srTestCaseAddAnnotation()
** Review srTestCaseAddComment()
** Review srTestAnnotationAddComment()
** Review srTestSuiteAddCase()
** Review srTestSuiteAddCase()
** Review srTestSuiteAddSuite()
** Review srTestSuiteSetDescription()
** Review srTestSuiteSetName()


=== Samples ===
=== Samples ===

Latest revision as of 19:43, 5 August 2014

Objectives

This Training Module is focused leveraging the Runtime APIs in the context of writing a test. The module covers the following topics:


There are two new files used -- TestRuntime.cpp & TestRuntime.h --- that implement two test Units:

  • TestRuntime_Static
  • TestRuntime_Dynamic


All of the Test Units have test cases already implemented (used for reference) and have one test method that you are required to implement: one is called Exercise and the other is called dynamic_Exercise. Currently the exercise methods return a NOT IN USE status.

Instructions

Build and Run TestApp

  • Build TestApp using SDK makefile
  • Startup TestApp
  • If you have not created an option file, please refer to setup
  • Execute Test Runtime APIs Test Units only
 > stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic

 Loading database...
 Connecting to device...
 Executing...
  test unit "TestRuntime_Static"
    > 2 passed, 0 failed, 0 in progress, 1 not in use.
  test unit "TestRuntime_Dynamic"
    > 4 passed, 0 failed, 0 in progress, 1 not in use.
 ------------------------------------------------------------
 Summary: 6 passed, 0 failed, 0 in progress, 2 not in use.
 
 Disconnecting from device...
 Saving result file...
  • Review the details of the test results using a Browser. Open TestApp.xml which can be found in the sample_src directory (based on the output option). By opening the xml file in a web browser the xsl is automatically applied to create html.

Implement Exercise

  • TestRuntime_Static::Exercise
    • Validate the sut_mult() routine using some simple data input
    • Set the test method name to Mult
    • Use direct Runtime APIs to:
      • Set the test case description
      • Capture logging information via comments
      • Set the status of the test case
  • TestRuntime_Dynamic->Exercise
    • Pass in 5 via command line for the number of test cases to generate
    • Check using srEXIT_XX that the number of test cases has been passed in correctly
    • Add a new Test Suite using Exercise for the name
    • Write a loop generating a dynamic test case using NumberOfTestCases
      • Test Case name shall be Test_n where "n" is the loop count as each case must have a unique name.
      • Add a description for each test case
      • Validate sut_foo() using the loop count


  • note- The dynamic Test Unit is using CClass packaging


  • Execute Test Runtime API Test Units only (NOTE - requires passing parameter)
 > stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic(5)

 Loading database...
 Connecting to device...
 Executing...
   test unit "TestRuntime_Static"
     > 3 passed, 0 failed, 0 in progress, 0 not in use.
   test unit "TestRuntime_Dynamic"
     > 10 passed, 0 failed, 0 in progress, 0 not in use.
   ---------------------------------------------------------------------
   Summary: 13 passed, 0 failed, 0 in progress, 0 not in use.

 Disconnecting from device...
 Saving result file...

Run and Publish Results

When you have completed the Exercise(s) publish your results to Test Space. If you have not added test space options to your options file (myoptions.txt) please see testspace access. 

  > stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic --space TestRuntime --upload

Note: This space has been set up with a Baseline of expected test results that you can use to validate your results.

Reference

The following reference information is related to passing parameters to Test Units.

Wiki

  • Runtime Test Services with special attention to the following:
    • Review srTestCaseSetStatus()
    • Review srTestCaseAddAnnotation()
    • Review srTestAnnotationAddComment()
    • Review srTestSuiteAddCase()

Samples

Retrieved from "https://www.stridewiki.com/index.php?title=Training_Runtime_API&oldid=14057"