Indicating Pass/Fail in a Test Method - Revision history

Timd at 22:05, 9 February 2012

2012-02-09T22:05:31Z

← Older revision		Revision as of 22:05, 9 February 2012
Line 1:		Line 1:
	==Test Method Return Values==		==Test Method Return Values==
	As described elsewhere, [[Test_Units#~~Test_Units_2~~\|Stride test methods]] ~~<ref>or free functions in the case of scl_flist test units</ref>~~ must conform to ~~[[Scl_test_flist#Notes\|~~certain criteria]]. Among other things, these criteria specify allowable signatures for test methods.		As described elsewhere, [[Test_Units#Test_Methods\|Stride test methods]] must conform to certain criteria. Among other things, these criteria specify allowable signatures for test methods.

	Three distinct return return types are permitted, each with their own behavior with respect to how return values are interpreted by the Stride runtime in setting PASS/FAIL status.		Three distinct return return types are permitted, each with their own behavior with respect to how return values are interpreted by the Stride runtime in setting PASS/FAIL status.

Timd at 17:29, 9 February 2012

2012-02-09T17:29:04Z

← Older revision		Revision as of 17:29, 9 February 2012
Line 211:		Line 211:

	<references/>		<references/>

			[[Category : Test Units]]

Timd at 22:55, 8 February 2012

2012-02-08T22:55:46Z

← Older revision		Revision as of 22:55, 8 February 2012
Line 55:		Line 55:
	==Examples==		==Examples==
	The following example implementations validate the operation of a function named DoAdd() (not shown). Required header file declarations are assumed to be in place.		The following example implementations validate the operation of a function named DoAdd() (not shown). Required header file declarations are assumed to be in place.

			<small>c++ examples are shown, but techniques are applicable to c as well.</small>

	===void return===		===void return===
	This is the return type that is most often used as it provides the most convenience.		This is the return type that is most often used as it provides the most convenience.

	~~c++ examples are shown, but techniques are applicable to c as well.~~

	<source lang=c>		<source lang=c>

Timd at 22:48, 8 February 2012

2012-02-08T22:48:43Z

← Older revision		Revision as of 22:48, 8 February 2012
Line 1:		Line 1:
	==Test Method Return Values==		==Test Method Return Values==
	[[Test_Units#Test_Units_2\|Stride test methods]] <ref>or free functions in the case of scl_flist test units</ref> must conform to [[Scl_test_flist#Notes\|certain criteria]]. ~~These~~ criteria ~~allow~~ for ~~several choices~~ in return type ~~from the test method~~.		As described elsewhere, [[Test_Units#Test_Units_2\|Stride test methods]] <ref>or free functions in the case of scl_flist test units</ref> must conform to [[Scl_test_flist#Notes\|certain criteria]]. Among other things, these criteria specify allowable signatures for test methods.

			Three distinct return return types are permitted, each with their own behavior with respect to how return values are interpreted by the Stride runtime in setting PASS/FAIL status.

			Following is a table summarizing the details of this behavior and code examples illustrating the use of each return type.


Line 204:		Line 208:
	}		}
	</source>		</source>

			<references/>

Timd: /* Test Method Return Values */

2012-02-08T22:33:02Z

Test Method Return Values

← Older revision		Revision as of 22:33, 8 February 2012
Line 1:		Line 1:
	==Test Method Return Values==		==Test Method Return Values==
	Stride test methods (or free functions in the case of scl_flist test units) must conform to [[Scl_test_flist#Notes\|certain criteria]]. These criteria allow for several choices in return type from the test method.		[[Test_Units#Test_Units_2\|Stride test methods]] <ref>or free functions in the case of scl_flist test units</ref> must conform to [[Scl_test_flist#Notes\|certain criteria]]. These criteria allow for several choices in return type from the test method.


	{\| border="1" cellspacing="0" cellpadding="10" style="~~align~~:~~left~~;"		{\| border="1" cellspacing="0" cellpadding="10" style="background-color:#ffffcc;font-family:arial;font-size:9pt;"
	\| '''Return Type'''		\| '''Return Type'''
	\| '''Description'''		\| '''Description'''
Line 23:		Line 23:
	\| integer type		\| integer type
	\| Integer types include:		\| Integer types include:
	* int		* <tt>int</tt>
	* short		* <tt>short</tt>
	* char		* <tt>char</tt>
	* long		* <tt>long</tt>
	''may include signed/unsigned/const qualifiers''		<small>''may include signed/unsigned/const qualifiers''</small>
	\|		\|
	* If you explicitly set the status in the method (using a test macro or a runtime call), this status will override the status that would otherwise be set by the return value		* If you explicitly set the status in the method (using a test macro or a runtime call), this status will override the status that would otherwise be set by the return value
Line 37:		Line 37:
	* If the method does not run to completion (due to crash or hang in software under test), the status is srTEST_INPROGRESS		* If the method does not run to completion (due to crash or hang in software under test), the status is srTEST_INPROGRESS
	\|-		\|-
	\| bool		\| <tt>bool</tt>
	\| c++ only		\| c++ only
	\|		\|

Timd: /* Examples */

2012-02-08T22:13:10Z

Examples

@@ Line 91: / Line 91: @@
      // ASSERT macro validates and will set status to srTEST_FAIL and
      // immediately return if the condition is not met
-     srASSERT_NEQ(h, 0);
+     srASSERT_NE(h, 0);
      int rc = h->DoSomething();
@@ Line 113: / Line 113: @@
 #include <srtest.h>
 int MyTest::Test1()
+{
      int sum = DoAdd(2, 2);
-     // return 0 -> PASS, non-zero -> FAIL
+     // return 0 -> srTEST_PASS, non-zero -> srTEST_FAIL
      return (sum == 4) ? 0 : 1;
+}
 int MyTest::Test2()
+{
      int sum = DoAdd(3, 3);
      srEXPECT_EQ(sum, 6);
      // return value does not affect test status since the status
      // was set explicitly using a test macro above
      return 0;
+}
@@ Line 150: / Line 178: @@
      // return value does not affect test status since the status
      // was set explicitly using a test macro above
      return true;
+}
 </source>

Timd: /* integer return */

2012-02-08T22:01:12Z

integer return

← Older revision		Revision as of 22:01, 8 February 2012
Line 116:		Line 116:
	{		{
	int sum = DoAdd(2, 2);		int sum = DoAdd(2, 2);
	// return ~~true~~ -> PASS, ~~false~~ -> FAIL		// return 0 -> PASS, non-zero -> FAIL
	return (sum == 4);		return (sum == 4) ? 0 : 1;
	}		}

Line 127:		Line 127:
	// return value does not affect test status since the status		// return value does not affect test status since the status
	// was set explicitly using a test macro above		// was set explicitly using a test macro above
	return ~~true~~;		return 0;
	}		}
	</source>		</source>

Timd: /* void return */

2012-02-08T21:58:12Z

void return

← Older revision		Revision as of 21:58, 8 February 2012
Line 60:		Line 60:
	#include <srtest.h>		#include <srtest.h>

			/////////////////////////////
			// shows use of a test macro to set status
			/////////////////////////////
	void MyTest::Test1()		void MyTest::Test1()
	{		{
	int sum = DoAdd(2, 2);		int sum = DoAdd(2, 2);
	// test macro validates condition and sets test status.		// test macro validates condition and sets test status.
	~~// the srEXPECT macros continue~~
	srEXPECT_EQ(sum, 4);		srEXPECT_EQ(sum, 4);
	}		}

			/////////////////////////////
			// shows use a a runtime call to set status
			/////////////////////////////
	void MyTest::Test2()		void MyTest::Test2()
	{		{
Line 77:		Line 82:
	}		}

			/////////////////////////////
			// shows use of ASSERT macro to "short circuit" the test
			/////////////////////////////
	void MyTest::Test3()		void MyTest::Test3()
			{
			HANDLE h = GetResource();
			// if the resource is not acquired, there is no point in continuing; the
			// ASSERT macro validates and will set status to srTEST_FAIL and
			// immediately return if the condition is not met
			srASSERT_NEQ(h, 0);

			int rc = h->DoSomething();
			srEXPECT_EQ(rc, 0);
			}

			/////////////////////////////
			// shows default status
			/////////////////////////////
			void MyTest::Test4()
	{		{
	// status is not set, so default srTEST_PASS is in force		// status is not set, so default srTEST_PASS is in force
	}		}


	</source>		</source>

Timd: /* Examples */

2012-02-08T21:06:03Z

Examples

← Older revision		Revision as of 21:06, 8 February 2012
Line 54:		Line 54:
	===void return===		===void return===
	This is the return type that is most often used as it provides the most convenience.		This is the return type that is most often used as it provides the most convenience.

			c++ examples are shown, but techniques are applicable to c as well.

	<source lang=c>		<source lang=c>
	#include <srtest.h>		#include <srtest.h>

	void Test1()		void MyTest::Test1()
	{		{
	int sum = DoAdd(2, 2);		int sum = DoAdd(2, 2);
Line 66:		Line 68:
	}		}

	void Test2()		void MyTest::Test2()
	{		{
	int sum = DoAdd(3, 3);		int sum = DoAdd(3, 3);
Line 72:		Line 74:
	// using the macro, but same result		// using the macro, but same result
	srTestStatus_e eStatus = (sum == 6) ? srTEST_PASS : srTEST_FAIL;		srTestStatus_e eStatus = (sum == 6) ? srTEST_PASS : srTEST_FAIL;
	~~srTestCaseSetStatus~~(~~srTEST_CASE_DEFAULT,~~		testCase.SetStatus(eStatus, 0);
	eStatus,
	0);
	}		}

	void Test3()		void MyTest::Test3()
	{		{
	// status is not set, so default srTEST_PASS is in force		// status is not set, so default srTEST_PASS is in force
Line 85:		Line 85:

	===integer return===		===integer return===

			<source lang=c>
			#include <srtest.h>

			int MyTest::Test1()
			{
			int sum = DoAdd(2, 2);
			// return true -> PASS, false -> FAIL
			return (sum == 4);
			}

			int MyTest::Test2()
			{
			int sum = DoAdd(3, 3);
			//
			srEXPECT_EQ(sum, 6);
			// return value does not affect test status since the status
			// was set explicitly using a test macro above
			return true;
			}
			</source>

	===bool return===		===bool return===

Timd: /* void return */

2012-02-08T20:47:26Z

void return

← Older revision		Revision as of 20:47, 8 February 2012
Line 69:		Line 69:
	{		{
	int sum = DoAdd(3, 3);		int sum = DoAdd(3, 3);
	~~srASSERT_EQ~~(sum, 6);		// set the status using a runtime call; much less convenient than
			// using the macro, but same result
			srTestStatus_e eStatus = (sum == 6) ? srTEST_PASS : srTEST_FAIL;
			srTestCaseSetStatus(srTEST_CASE_DEFAULT,
			eStatus,
			0);
			}

			void Test3()
			{
			// status is not set, so default srTEST_PASS is in force
	}		}

	</source>		</source>

	===integer return===		===integer return===