Training Test Macros: Difference between revisions

Latest revision as of 14:50, 22 November 2014

Objectives

This Training Module focuses on the basics of writing and executing tests. It covers the following topics:

The C++ class Test Unit packaging option
How to leverage Test Macros
Using Notes and Test Logs, what is the difference?
Creating Test Documentation via the build process
Executing Tests using the Runner

The test unit is implemented in two source files: TestBasic.cpp and TestBasic.h. The comments and descriptions are contained in TestBasic.h.

The Test Unit has test cases already implemented (used for reference) and has a test method that you are required to implement (called Exercise). Currently this method is empty and returns a NOT IN USE status.

Instructions

Build and Run TestApp

Build TestApp using the SDK makefile
Invoke TestApp found in the /out/bin directory
If you have not created an option file, please refer to setup

Execute the Test Basic Test Units

 > stride --options_file myoptions.txt --run TestBasic
 Loading database...
 Connecting to device...
 Executing...
   test unit "TestBasic"
     > 3 passed, 1 failed, 0 in progress, 1 not in use.
   --------------------------------------------------------------
   Summary: 3 passed, 1 failed, 0 in progress, 1 not in use.

 Disconnecting from device...
 Saving result file...

You can also review the details of the test results using a Browser. Open TestApp.xml which can be found in the sample_src directory (based on the output option). By opening the xml file in a web browser the xsl is automatically applied to create html.

Implement Exercises

Now edit the training source code to complete the following exercises:

Exercise()

Assignment 1: Add an srNOTE to Exercise() that will add a simple message to the test report (e.g. "Exercise ...")
- Hint: You will probably want to read about Test Macros
- Hint: srTestCaseSetStatus(srTEST_CASE_DEFAULT, srTEST_NOTINUSE, 0) is a placeholder to be removed when you add your test code.
Assignment 2: Within Exercise() validate that sut_mult(1,1) does NOT equal sut_add(1,1)
- Hint: You will use a Test Macro.

Check Results

Before you rebuild TestApp.exe, you will need to shut it down by entering 'q' in its console window or closing the window directly.

After rebuilding, invoke TestApp.exe once again.

Execute only TestBasic using the stride runner:

 > stride --options_file myoptions.txt --run TestBasic

 Loading database...
 Connecting to device...
 Executing...
   test unit "TestBasic"
     > 4 passed, 1 failed, 0 in progress, 0 not in use.
   --------------------------------------------------------------
   Summary: 4 passed, 1 failed, 0 in progress, 0 not in use.

 Disconnecting from device...
 Saving result file...

If you have TestApp.xml already open in your browser, you can simply refresh (F5), to view the latest test results.

Run and Publish Results

When you have completed the Exercise(s) publish your results to Test Space(--space TestBasic). If you have not added test space options to your options file (myoptions.txt) please see testspace access.

  > stride --options_file myoptions.txt --run TestBasic --space TestBasic --upload

 Loading database...
 Connecting to device...
 Executing...
   test unit "TestBasic"
     > 4 passed, 1 failed, 0 in progress, 0 not in use.
   ------------------------------------------------------------
   Summary: 4 passed, 1 failed, 0 in progress, 0 not in use.

 Disconnecting from device...
 Saving result file...
 Uploading to test space...

Validate Uploaded Results

Navigate to Test Space using your browser, then validate your results against the pre-configured baseline. For details, please see Confirming Training Exercise Results

Note: This space has been set up with a Baseline of expected test results that you can use to validate your results.

You can view the results on test space by pointing your browser at https://yourcompany.stridetestspace.com. Log in using the credentials you supplied earlier in the options file.

Reference

The following reference information is related to Test Unit basics.

Wiki

Test Units Overview - Provides a general overview Test Units (i.e. writing tests in C and C++)
Test Unit Packaging - Discusses the three types of packaging that can be used for Test Units (we prefer Test Classes even for c programmers)
Test Macros - Optional macros that provide shortcuts for testing assertions and automatic report annotation (you will want to use these).
Notes - Used to add logging information to your test logic (automatically added to test reports)
Test Logs - Used to add logging information to your source code (added to test reports if enabled)
Tracing - The Runner allows tracing on logs and test points.

Samples

Test Class Sample - Specifically the Basic Simple Test Unit
Test Macro Sample - This sample covers simple uses of each of the Test Macros.

Note - Other Test Unit Samples related to packaging that are useful are the following:

@@ Line 2: / Line 2: @@
 This Training Module focuses on the basics of writing and executing tests. It covers the following topics:
-* The different [[Test_Units | Test Unit packaging]] options
+* The C++ class [[Test_Units | Test Unit packaging]] option
 * How to leverage [[Test_Macros | Test Macros]]
 * Using [[Test_Macros#Notes | Notes]] and [[Test_Log | Test Logs]], what is the difference?
@@ Line 9: / Line 9: @@
-This training module comprises three separate Test Units, each of which demonstrates an available packaging option:
+The test unit is implemented in two source files: '''TestBasic.cpp''' and '''TestBasic.h'''. The comments and descriptions are contained in '''TestBasic.h'''.
-;TestBasic_FList
-: Packages a group of free functions into a Test Unit
-;TestBasic_CClass
-: Uses a C structure as a Test Unit container
-;TestBasic_Class
-: Uses a C++ class as a Test Unit container
-These test units are implemented in two source files: '''TestBasic.cpp''' and '''TestBasic.h'''.
+The Test Unit has test cases already implemented (used for reference) and has a test method that you are required to implement (called '''Exercise'''). Currently this method is empty and returns a ''NOT IN USE'' status.
-All of the test methods in each Test Unit have already been implemented (you can use these for reference) except for the test method named'''TestExercise'''.  This method simply sets a ''NOT IN USE'' status for the test method.
-'''IMPORTANT NOTE''' - As you look at this module's source code, you will note that the test methods (cases) within each Test Unit are '''exactly''' the same, regardless of packaging. (In fact the ''CClass'' and ''Class'' Test Units just call the ''C-based'' test functions used by the ''FList'' Test Unit.) The only difference is in the packaging of the test methods into each Test Unit.
 == Instructions ==
@@ Line 27: / Line 17: @@
 === Build and Run TestApp ===
-* Build TestApp.exe using the SDK makefile
+* [[Building_an_Off-Target_Test_App#Build_Steps | Build TestApp]] using the SDK makefile
-** If you have trouble building, please see [[Troubleshooting Build Problems]].
+* Invoke TestApp found in the ''/out/bin'' directory
-* Invoke TestApp.exe
+* If you have not created an option file, please refer to [[Training_Getting_Started#Run_Training_Tests| setup]]
-** After build, the file will be found in <tt>%STRIDE_DIR%\SDK\Windows\out\bin</tt>.
-** You can double-click in the File Explorer and the program's console window will open.
-* Create an [[Stride_Runner#Options | option file]] (<tt>myoptions.txt</tt>) that includes the following content (Windows example)
-** You can create this file anywhere that's convenient.
-  ##### Command Line Options ######
-  --device "TCP:localhost:8000"
-  --database %STRIDE_DIR%\SDK\Windows\out\TestApp.sidb
-  --output %STRIDE_DIR%\SDK\Windows\sample_src\TestApp.xml
-  --log_level all
 * Execute the ''Test Basic'' Test Units
-   > stride -O myoptions.txt --run TestBasic_FList TestBasic_CClass TestBasic_Class
+   > stride --options_file myoptions.txt --run TestBasic
    Loading database...
    Connecting to device...
    Executing...
-     test unit "TestBasic_FList"
+     test unit "TestBasic"
-       > 3 passed, 2 failed, 0 in progress, 1 not in use.
+       > 3 passed, 1 failed, 0 in progress, 1 not in use.
-    test unit "TestBasic_CClass"
-      > 3 passed, 2 failed, 0 in progress, 1 not in use.
-    test unit "TestBasic_Class"
-      > 3 passed, 2 failed, 0 in progress, 1 not in use.
      --------------------------------------------------------------
-     Summary: 9 passed, 6 failed, 0 in progress, 3 not in use.
+     Summary: 3 passed, 1 failed, 0 in progress, 1 not in use.
    Disconnecting from device...
    Saving result file...
-* You can now review the details of the test results using a Web Browser.
+You can also review the details of the test results using a Browser. Open [[Building_an_Off-Target_Test_App#Interpreting_Results | TestApp.xml ]] which can be found in the sample_src directory (based on the output option). By opening the xml file in a web browser the xsl is automatically applied to create html.
-* Locate '''TestApp.xml''' (which was written to the <tt>sample_src</tt> directory via the <tt>--output</tt> command line option).
-* Open this file in your browser (dragging the file from file explorer to your browser is usually the easiest way to do this).
-** By opening the xml file in a web browser an xsl style sheet is automatically applied to create html that is ultimately displayed.
-** If you use Google Chrome, please see [[Browser Compatibility]].
 === Implement Exercises ===
 Now edit the training source code to complete the following exercises:
-;TestBadFailure()
-*''' Your assignment:''' The test macro currently used in the test method <tt>TestBadFailure()</tt> can prevent subsequent test methods in the Test Unit from executing. This is sometimes useful, but here we want to change the Test Macro type that's used such that subsequent test methods will execute.
-** ''Hint:'' This function can be found in <tt>TestBasic.cpp</tt> in the <tt>sample_src</tt> directory.
-** ''Hint:'' You will probably want to read about [[Test Macros]]
-** ''Hint:'' Reading about [[Test Units]] is a good idea too
-;TestExercise()
+;Exercise()
-* '''Assignment 1:''' Add an ''srNOTE'' to <tt>TestExercise()</tt> that will add a simple message to the test report (e.g. "Test Exercise ...")
+* '''Assignment 1:''' Add an ''srNOTE'' to <tt>Exercise()</tt> that will add a simple message to the test report (e.g. "Exercise ...")
 ** ''Hint:'' You will probably want to read about [[Test Macros]]
+** ''Hint:'' srTestCaseSetStatus(srTEST_CASE_DEFAULT, srTEST_NOTINUSE, 0) is a placeholder to be removed when you add your test code.
-* '''Assignment 2:''' Within <tt>TestExercise()</tt> validate that <tt>sut_mult(1,1)</tt> does NOT equal <tt>sut_add(1,1)</tt>
+* '''Assignment 2:''' Within <tt>Exercise()</tt> validate that <tt>sut_mult(1,1)</tt> does NOT equal <tt>sut_add(1,1)</tt>
 ** ''Hint:'' You will use a [[Test Macros | Test Macro]].
-====Checking Results====
+=== Check Results ===
 * Before you rebuild <tt>TestApp.exe</tt>, you will need to shut it down by entering 'q' in its console window or closing the window directly.
@@ Line 89: / Line 54: @@
 * After rebuilding, invoke <tt>TestApp.exe</tt> once again.
-* Execute only ''TestBasic_FList'' using the stride runner:
+* Execute only ''TestBasic'' using the stride runner:
-   > stride -O myoptions.txt --run TestBasic_FList
+   > stride --options_file myoptions.txt --run TestBasic
    Loading database...
    Connecting to device...
    Executing...
-     test unit "TestBasic_FList"
+     test unit "TestBasic"
-       > 4 passed, 2 failed, 0 in progress, 0 not in use.
+       > 4 passed, 1 failed, 0 in progress, 0 not in use.
      --------------------------------------------------------------
-     Summary: 4 passed, 2 failed, 0 in progress, 0 not in use.
+     Summary: 4 passed, 1 failed, 0 in progress, 0 not in use.
    Disconnecting from device...
@@ Line 108: / Line 73: @@
 === Run and Publish Results ===
-When you have completed the Exercise(s) publish your results to Test Space. To make it easier for now we recommend that you update your existing option file (<tt>myoptions.txt</tt>) with the following:
+When you have completed the Exercise(s) publish your results to Test Space(--space TestBasic). If you have not added test space options to your options file (<tt>myoptions.txt</tt>) please see [[Training_Getting_Started#Test_Space_Access| testspace access]].
-  #### Test Space options (partial) #####
+    > stride --options_file myoptions.txt --run TestBasic --space TestBasic --upload
-  #### Note - make sure to change username, etc. ####
-  --testspace <nowiki>https://username:password@yourcompany.stridetestspace.com</nowiki>
-  --project Training
-  --name YOURNAME
-    > stride -O myoptions.txt --run TestBasic_FList TestBasic_CClass TestBasic_Class --space TestBasic --upload
    Loading database...
    Connecting to device...
    Executing...
-     test unit "TestBasic_FList"
+     test unit "TestBasic"
-       > 4 passed, 2 failed, 0 in progress, 0 not in use.
+       > 4 passed, 1 failed, 0 in progress, 0 not in use.
-    test unit "TestBasic_CClass"
-      > 4 passed, 2 failed, 0 in progress, 0 not in use.
-    test unit "TestBasic_Class"
-      > 4 passed, 2 failed, 0 in progress, 0 not in use.
      ------------------------------------------------------------
-     Summary: 12 passed, 6 failed, 0 in progress, 0 not in use.
+     Summary: 4 passed, 1 failed, 0 in progress, 0 not in use.
    Disconnecting from device...
    Saving result file...
    Uploading to test space...
+=== Validate Uploaded Results ===
+Navigate to Test Space using your browser, then validate your results against the pre-configured baseline. For details, please see [[Training_Getting_Started#Confirming_Training_Exercise_Results | Confirming Training Exercise Results]]
 Note: This space has been set up with a Baseline of [[Training_Getting_Started#Test_Space_Access | ''expected test results'']] that you can use to validate your results.
@@ Line 152: / Line 111: @@
 === Samples ===
-* [[Test_Function_List_Sample|Test Function List Sample]] - Specifically the [[Test_Function_List_Sample#basic_simple|Basic Simple]] Test Unit
+* [[Test_Class_Sample|Test Class Sample]] -  Specifically the [[Test_Class_Sample#Basic|Basic Simple]] Test Unit
 * [[Test_Macros_Sample|Test Macro Sample]] - This sample covers simple uses of each of the Test Macros.
 Note - Other Test Unit Samples related to packaging that are useful are the following:
-* [[Test_Class_Sample|Test Class Sample]]
 * [[Test_CClass_Sample|Test C Class Sample]]
+* [[Test_Function_List_Sample|Test Function List Sample]]
 [[Category: Training]]

Training Test Macros: Difference between revisions

Latest revision as of 14:50, 22 November 2014

Contents

Objectives

Instructions

Build and Run TestApp

Implement Exercises

Check Results

Run and Publish Results

Validate Uploaded Results

Reference

Wiki

Samples

Navigation menu

Training Test Macros: Difference between revisions

Latest revision as of 14:50, 22 November 2014

Objectives

Instructions

Build and Run TestApp

Implement Exercises

Check Results

Run and Publish Results

Validate Uploaded Results

Reference

Wiki

Samples

Navigation menu

Search