Training Runtime API: Difference between revisions

Latest revision as of 19:43, 5 August 2014

Objectives

This Training Module is focused leveraging the Runtime APIs in the context of writing a test. The module covers the following topics:

How to set test status
Adding test annotations and attaching comments to them
Dynamically creating test cases

There are two new files used -- TestRuntime.cpp & TestRuntime.h --- that implement two test Units:

TestRuntime_Static
TestRuntime_Dynamic

All of the Test Units have test cases already implemented (used for reference) and have one test method that you are required to implement: one is called Exercise and the other is called dynamic_Exercise. Currently the exercise methods return a NOT IN USE status.

Instructions

Build and Run TestApp

Build TestApp using SDK makefile
Startup TestApp
If you have not created an option file, please refer to setup

Execute Test Runtime APIs Test Units only

 > stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic

 Loading database...
 Connecting to device...
 Executing...
  test unit "TestRuntime_Static"
    > 2 passed, 0 failed, 0 in progress, 1 not in use.
  test unit "TestRuntime_Dynamic"
    > 4 passed, 0 failed, 0 in progress, 1 not in use.
 ------------------------------------------------------------
 Summary: 6 passed, 0 failed, 0 in progress, 2 not in use.
 
 Disconnecting from device...
 Saving result file...

Review the details of the test results using a Browser. Open TestApp.xml which can be found in the sample_src directory (based on the output option). By opening the xml file in a web browser the xsl is automatically applied to create html.

Implement Exercise

TestRuntime_Static::Exercise
- Validate the sut_mult() routine using some simple data input
  - Hint: You will probably want to read about Runtime Test Services
- Set the test method name to Mult
- Use direct Runtime APIs to:
  - Set the test case description
  - Capture logging information via comments
  - Set the status of the test case

TestRuntime_Dynamic->Exercise
- Pass in 5 via command line for the number of test cases to generate
- Check using srEXIT_XX that the number of test cases has been passed in correctly
- Add a new Test Suite using Exercise for the name
- Write a loop generating a dynamic test case using NumberOfTestCases
  - Test Case name shall be Test_n where "n" is the loop count as each case must have a unique name.
  - Add a description for each test case
  - Validate sut_foo() using the loop count

note- The dynamic Test Unit is using CClass packaging

Execute Test Runtime API Test Units only (NOTE - requires passing parameter)

 > stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic(5)

 Loading database...
 Connecting to device...
 Executing...
   test unit "TestRuntime_Static"
     > 3 passed, 0 failed, 0 in progress, 0 not in use.
   test unit "TestRuntime_Dynamic"
     > 10 passed, 0 failed, 0 in progress, 0 not in use.
   ---------------------------------------------------------------------
   Summary: 13 passed, 0 failed, 0 in progress, 0 not in use.

 Disconnecting from device...
 Saving result file...

Run and Publish Results

When you have completed the Exercise(s) publish your results to Test Space. If you have not added test space options to your options file (myoptions.txt) please see testspace access.

  > stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic --space TestRuntime --upload

Note: This space has been set up with a Baseline of expected test results that you can use to validate your results.

Reference

The following reference information is related to passing parameters to Test Units.

Wiki

Runtime Test Services with special attention to the following:
- Review srTestCaseSetStatus()
- Review srTestCaseAddAnnotation()
- Review srTestAnnotationAddComment()
- Review srTestSuiteAddCase()

Samples

Test Class Sample - Specifically the Runtime Services Test Units
Test C Class Sample - Specifically the Runtime Services Test Units
Test Function List Sample - Specifically the Runtime Services Test Units

@@ Line 3: / Line 3: @@
 This Training Module is focused leveraging the [[Runtime_Test_Services | Runtime APIs]] in the context of writing a test. The module covers the following topics:
 * How to set [[Runtime_Test_Services#srTestCaseSetStatus | test status]]
-* Setting [[Runtime_Test_Services#srTestCaseSetName | test case name]] and [[Runtime_Test_Services#srTestCaseSetDescription | description]] directly via the API
+* Adding [[Runtime_Test_Services#srTestCaseAddAnnotation | test annotations]] and attaching [[Runtime_Test_Services#srTestAnnotationAddComment | comments]] to them
-* Setting [[Runtime_Test_Services#srTestSuiteSetName | test suite name]] and [[Runtime_Test_Services#srTestSuiteSetDescription | description]] directly via the API
+* Dynamically creating [[Runtime_Test_Services#srTestSuiteAddCase | test cases]]
-* Dynamically creating [[Runtime_Test_Services#srTestSuiteAddSuite | test suites]] and [[Runtime_Test_Services#srTestSuiteAddCase | test cases]]
@@ Line 19: / Line 18: @@
 === Build and Run TestApp ===
-* Build TestApp using SDK makefile
+* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using SDK makefile
 * Startup TestApp
-* If not already done, create an [[Stride_Runner#Options | option file]] (myoptions.txt) using the following content (Windows example)
+* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]
-  ##### Command Line Options ######
-  --device "TCP:localhost:8000"
-  --database %STRIDE_DIR%\SDK\Windows\out\TestApp.sidb
-  --output %STRIDE_DIR%\SDK\Windows\sample_src\TestApp.xml
-  --log_level all
 * Execute ''Test Runtime APIs'' Test Units only
-   > stride -O myoptions.txt --run TestRuntime_Static TestRuntime_Dynamic
+   > stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic
    Loading database...
@@ Line 46: / Line 39: @@
    Saving result file...
-* Review the details of the test results using a Browser. Open '''TestApp.xml''' which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.
+* Review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the ''sample_src'' directory (based on the ''output'' option). By opening the xml file in a web browser the xsl is automatically applied to create html.
 === Implement Exercise ===
@@ Line 52: / Line 45: @@
 * '''TestRuntime_Static::Exercise'''
 ** Validate the ''sut_mult()'' routine using some simple data input
+*** ''Hint:'' You will probably want to read about [[Runtime Test Services]]
 ** Set the ''test method'' name to '''Mult'''
 ** Use direct Runtime APIs to:
@@ Line 63: / Line 57: @@
 ** Add a new Test Suite using '''Exercise''' for the name
 ** Write a loop generating a dynamic test case using ''NumberOfTestCases''
-*** Test Case name shall be '''Test_n''' where "n" is the loop count
+*** Test Case name shall be '''Test_n''' where "n" is the loop count as each case must have a unique name.
 *** Add a description for each test case
 *** Validate ''sut_foo()'' using the loop count
@@ Line 73: / Line 67: @@
 * Execute ''Test Runtime API'' Test Units only (NOTE - requires passing parameter)
-   > stride -O myoptions.txt --run TestRuntime_Static TestRuntime_Dynamic(5)
+   > stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic(5)
    Loading database...
@@ Line 87: / Line 81: @@
    Disconnecting from device...
    Saving result file...
 === Run and Publish Results ===
-When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (myoptions.txt) with the following if not already done:
+When you have completed the Exercise(s) publish your results to Test Space. If you have not added test space options to your options file (<tt>myoptions.txt</tt>) please see [[Training_Getting_Started#Test_Space_Access| testspace access]].
-  #### Test Space options (partial) #####
-  #### Note - make sure to change username, etc. ####
-  --testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
-  --project Training
-  --name YOURNAME
-    > stride -O myoptions.txt --run TestRuntime_Static TestRuntime_Dynamic --space TestRuntime --upload
+    > stride --options_file myoptions.txt --run TestRuntime_Static --run TestRuntime_Dynamic --space TestRuntime --upload
 Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.
@@ Line 109: / Line 96: @@
 * [[Runtime_Test_Services#C_Test_Functions | Runtime Test Services]] with special attention to the following:
-** Review srTestCaseSetDescription()
 ** Review srTestCaseSetStatus()
-** Review srTestCaseSetName()
+** Review srTestCaseAddAnnotation()
-** Review srTestCaseAddComment()
+** Review srTestAnnotationAddComment()
 ** Review srTestSuiteAddCase()
-** Review srTestSuiteAddSuite()
-** Review srTestSuiteSetDescription()
-** Review srTestSuiteSetName()
 === Samples ===

Training Runtime API: Difference between revisions

Latest revision as of 19:43, 5 August 2014

Contents

Objectives

Instructions

Build and Run TestApp

Implement Exercise

Run and Publish Results

Reference

Wiki

Samples

Navigation menu

Training Runtime API: Difference between revisions

Latest revision as of 19:43, 5 August 2014

Objectives

Instructions

Build and Run TestApp

Implement Exercise

Run and Publish Results

Reference

Wiki

Samples

Navigation menu

Search