Training Advanced: Difference between revisions

From STRIDE Wiki
Jump to navigation Jump to search
No edit summary
 
(7 intermediate revisions by 2 users not shown)
Line 1: Line 1:
The '''Advanced''' training focuses on using ''Test Doubles'', ''Test Points'', and ''Fixtures''.  
The '''Advanced''' training focuses on using ''Test Doubles'', ''Test Points'', and ''Fixtures''.  


The ''Test Doubles'' will demonstrate how global functions can be dynamically intercepted and replaced with test logic (stub, fake, mock). The following routines will be used:
The ''Test Doubles'' will demonstrate how global functions can be dynamically intercepted and replaced with test logic (stub, fake, mock). The following functions contained in ''software_under_test'' will be used to demonstrate how to leverage interception when testing:  


  int sut_foo(int x);
  int sut_foo(int x);
Line 8: Line 8:
  int sut_strcheck(const char *s);
  int sut_strcheck(const char *s);


''Test Points'' enable a different style of testing using source instrumentation. This is especially applicable when normal API testing is not practical. The following ''sequencing'' routines will be called that issue Test Points calls (again very simple but used to show the technique).  
''Test Points'' enable a different style of testing using source instrumentation. This is especially applicable when normal API testing is not practical. The following ''sequencing'' functions will be called that issue Test Points. Note that the sequencing routines are very simple, keeping the focus on the technique that can be applied.


  void sut_Sequence1(void);
  void sut_Sequence1(void);
Line 14: Line 14:
  void sut_Sequence3(void);
  void sut_Sequence3(void);


''Fixturing'' shows how setup and teardown routines per test method can be incorporated within a Test Suite. It will rely on the following routines to control thread:
''Fixturing'' shows how setup and teardown routines per test method can be incorporated within a Test Unit. The following functions will be used to demonstrate fixturing:  


  void sut_start_thread(void);
  void sut_start_thread(void);
  void sut_stop_thread(void);
  void sut_stop_thread(void);
Example option file ''my.opt'' used below for running
  --database "%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb"
  --device TCP:localhost:8000


== Test Doubles ==
== Test Doubles ==


This test suite focuses on leveraging '''Test Doubles''' in the context of executing a test.  
This Test Unit focuses on leveraging '''Test Doubles''' in the context of executing a test.  


The following articles are related to this example:
The following articles are related to this example:


* Using [[Using_Test_Doubles | Test Doubles]]  
* Using [[Test Double | Test Doubles]]  
** ''Definition'' verses ''Reference''
** ''Definition'' verses ''Reference''
** ''Explicit'' verses ''Implicit''
** ''Explicit'' verses ''Implicit''
** Setting and Resetting the Double implementation
** Setting and Resetting the Double implementation
* How to apply pragmas for [[Function_Capturing | function intercepting]]
* How to apply pragmas for [[Scl function | function intercepting]]


The Test Suite is called '''TestDoubles''' and is implemented in two source files: '''testdoubles.cpp''' and '''testdoubles.h'''. The comments and descriptions are contained in the header file. Three test cases (methods) are already implemented and one test method that can be used to make changes to called '''TryStuff'''. Currently this test method is set to ''not in use''.  
The Test Unit is called '''TestDoubles''' and is implemented in two source files: '''testdoubles.cpp''' and '''testdoubles.h'''. The comments and descriptions are contained in the header file. Three test cases (methods) are already implemented and one test method that can be used to make changes to is called '''TryStuff'''. Currently the ''TryStuff'' test method is set to ''not in use''.  


First thing to do is run the ''TestDoubles'' Suite. Start the '''TestApp.exe''' and use the following command:
First thing to do is run the ''TestDoubles'' Test Unit. Start the '''TestApp.exe''' and use the following command:


   stride --options_file my.opt --run "TestDoubles"
   stride --options_file my.opt --run "TestDoubles"


You can take a look at the results in '''TestApp.xml'''.
You can take a look at the results by opening the generated '''stride.html''' in your browser.


Consider the following for '''TryStuff:'''
Consider the following for '''TryStuff:'''
Line 44: Line 49:
* Double sut_boo()
* Double sut_boo()
** call sut_2xboo() with a number
** call sut_2xboo() with a number
** confirm with a Test Macro within the double the passed in number (mocking example)
** confirm within the test double (using a [[Test Macros | Test Macro]]) the expected passed in number (mocking example)


== Test Points ==
== Test Points ==
This test suite focuses on '''Test Points''' and how to validate them.  
This Test Unit focuses on '''Test Points''' and how to validate them.  


The following articles are related to this example:  
The following articles are related to this example:  
* Presentation of a [[Expectations | validation]] technique based on ''code sequencing'' and ''state data''
* Presentation of a [[Expectations | validation]] technique based on ''code sequencing'' and ''state data''
* Overview of [[Source_Instrumentation_Overview#Instrumentation | source instrumentation]]
* Overview of a [[Test Point]]
* Review of [[Test_Point_Testing_in_C/C%2B%2B | expectation tables and predicates]]
* [[Test_Point_Testing_in_C/C%2B%2B | Writing Tests]] leveraging Test Points
* [[Test Point]] Macro definition


The Test Suite is called '''TestPoints''' and is implemented in two source files: '''testpoints.cpp''' and '''testpoints.h'''. The comments and descriptions are contained in the header file. Three test cases (methods) are already implemented and one test method that can be used to make changes to called '''TryStuff'''. Currently this test method is set to ''not in use''.  
The Test Unit is called '''TestPoints''' and is implemented in two source files: '''testpoints.cpp''' and '''testpoints.h'''. The comments and descriptions are contained in the header file. Three test cases (methods) are already implemented and one test method that can be used to make changes to is called '''TryStuff'''. Currently the ''TryStuff'' test method is set to ''not in use''.  


First thing to do is run the ''TestPoints'' Suite. Start the '''TestApp.exe''' and use the following command:
First thing to do is run the ''TestPoints'' Test Unit. Start the '''TestApp.exe''' and use the following command:


   stride --options_file my.opt --run "TestPoints"
   stride --options_file my.opt --run "TestPoints"


You can take a look at the results in '''TestApp.xml'''.
You can take a look at the results by opening the generated '''stride.html''' in your browser.


Consider trying the following:
Consider the following for '''TryStuff:'''
* Write a test to validate a subset of Test Points  
* Write a test to validate a subset of Test Points  
* Write a test to validate "E", "F", and "G", and a predicate to verify data associated with "E"
* Write a test to validate "E", "F", and "G", and a predicate to verify data associated with "E"


== Fixtures ==
== Fixtures ==
This test suite focuses on using Fixutres in the context of an executing Test Suite.   
This Test Unit focuses on using Fixutures in the context of an executing Test Unit.   
 
The following articles are related to this example:
* [[Test_Unit_Pragmas#Fixturing_Pragmas | Fixture]] logic for each test method
* [[What_is_Unique_About_STRIDE#Fixturing | fixturing]]


The Test Suite is called '''TestFixtures''' and is implemented in two source files: '''testfixtures.cpp''' and '''testfixtures.h'''. The comments and descriptions are contained in the header file. One test cases (methods) is already implemented and one test method that can be used to make changes to '''TryStuff'''. Currently this test method is set to ''not in use''.  
The Test Unit is called '''TestFixtures''' and is implemented in two source files: '''testfixtures.cpp''' and '''testfixtures.h'''. The comments and descriptions are contained in the header file. One test cases (methods) is already implemented and one test method that can be used to make changes to is called '''TryStuff'''. Currently the ''TryStuff'' test method is set to ''not in use''.  


First thing to do is run the ''TestFixtures'' Suite. Start the '''TestApp.exe''' and use the following command:
First thing to do is run the ''TestFixtures'' Test Unit. Start the '''TestApp.exe''' and use the following command:


   stride --options_file my.opt --run "TestFixtures"
   stride --options_file my.opt --run "TestFixtures"


You can take a look at the results in '''TestApp.xml'''.
You can take a look at the results by opening the generated '''stride.html''' in your browser.


Consider trying the following:
Consider the following for '''TryStuff:'''
* TBD
* Write a test to validate "A", "B", and "C" using the "TRIGGER"
* TBD

Latest revision as of 21:49, 6 July 2015

The Advanced training focuses on using Test Doubles, Test Points, and Fixtures.

The Test Doubles will demonstrate how global functions can be dynamically intercepted and replaced with test logic (stub, fake, mock). The following functions contained in software_under_test will be used to demonstrate how to leverage interception when testing:

int sut_foo(int x);
int sut_boo(int x);
int sut_2xboo(int x);
int sut_strcheck(const char *s);

Test Points enable a different style of testing using source instrumentation. This is especially applicable when normal API testing is not practical. The following sequencing functions will be called that issue Test Points. Note that the sequencing routines are very simple, keeping the focus on the technique that can be applied.

void sut_Sequence1(void);
void sut_Sequence2(void);
void sut_Sequence3(void);

Fixturing shows how setup and teardown routines per test method can be incorporated within a Test Unit. The following functions will be used to demonstrate fixturing:

void sut_start_thread(void);
void sut_stop_thread(void);

Example option file my.opt used below for running

 --database "%STRIDE_DIR%\SDK\Windows\out\TestApp.sidb"
 --device TCP:localhost:8000

Test Doubles

This Test Unit focuses on leveraging Test Doubles in the context of executing a test.

The following articles are related to this example:

The Test Unit is called TestDoubles and is implemented in two source files: testdoubles.cpp and testdoubles.h. The comments and descriptions are contained in the header file. Three test cases (methods) are already implemented and one test method that can be used to make changes to is called TryStuff. Currently the TryStuff test method is set to not in use.

First thing to do is run the TestDoubles Test Unit. Start the TestApp.exe and use the following command:

 stride --options_file my.opt --run "TestDoubles"

You can take a look at the results by opening the generated stride.html in your browser.

Consider the following for TryStuff:

  • Create your own stub for foo() and intercept it
    • confirm it returns what you expect
  • Double sut_boo()
    • call sut_2xboo() with a number
    • confirm within the test double (using a Test Macro) the expected passed in number (mocking example)

Test Points

This Test Unit focuses on Test Points and how to validate them.

The following articles are related to this example:

The Test Unit is called TestPoints and is implemented in two source files: testpoints.cpp and testpoints.h. The comments and descriptions are contained in the header file. Three test cases (methods) are already implemented and one test method that can be used to make changes to is called TryStuff. Currently the TryStuff test method is set to not in use.

First thing to do is run the TestPoints Test Unit. Start the TestApp.exe and use the following command:

 stride --options_file my.opt --run "TestPoints"

You can take a look at the results by opening the generated stride.html in your browser.

Consider the following for TryStuff:

  • Write a test to validate a subset of Test Points
  • Write a test to validate "E", "F", and "G", and a predicate to verify data associated with "E"

Fixtures

This Test Unit focuses on using Fixutures in the context of an executing Test Unit.

The Test Unit is called TestFixtures and is implemented in two source files: testfixtures.cpp and testfixtures.h. The comments and descriptions are contained in the header file. One test cases (methods) is already implemented and one test method that can be used to make changes to is called TryStuff. Currently the TryStuff test method is set to not in use.

First thing to do is run the TestFixtures Test Unit. Start the TestApp.exe and use the following command:

 stride --options_file my.opt --run "TestFixtures"

You can take a look at the results by opening the generated stride.html in your browser.

Consider the following for TryStuff:

  • Write a test to validate "A", "B", and "C" using the "TRIGGER"