Creating And Using Baselines - Revision history

Ivailop at 21:08, 15 October 2014

2014-10-15T21:08:53Z

Marku: /* How the baseline comparison is computed */

2010-06-04T18:25:06Z

How the baseline comparison is computed

← Older revision		Revision as of 18:25, 4 June 2010
Line 127:		Line 127:

	[[Category:Test Space]]		[[Category:Test Space]]
	~~[[Category:STRIDE Test Space]]~~

Mikee: /* Fixed */

2009-10-08T22:02:14Z

Fixed

← Older revision		Revision as of 22:02, 8 October 2009
Line 33:		Line 33:
	Once you have creates baseline data, you can configure the testspace to automatically compare new result sets with one or more of the baselines. To do this, edit the testspace properties and select the ''add fixed baseline compare'' under the ''auto compare'' settings. Using the drop-list of existing fixed baselines, select the desired baseline and submit. You can also remove previously set baseline comparisons (if you no longer want to compare with certain baselines) using the testspace properties page.		Once you have creates baseline data, you can configure the testspace to automatically compare new result sets with one or more of the baselines. To do this, edit the testspace properties and select the ''add fixed baseline compare'' under the ''auto compare'' settings. Using the drop-list of existing fixed baselines, select the desired baseline and submit. You can also remove previously set baseline comparisons (if you no longer want to compare with certain baselines) using the testspace properties page.

	~~VIDEO HERE -~~ how to ~~activate~~ fixed baselines		This screencast shows how to apply fixed baselines:

			<dhflashplayer>file=TS_BaselineFixedApply.mp4\|width=640\|height=480\| path=http://www.s2technologies.com/video/wiki/\|more_than_one=true</dhflashplayer>

	== How the baseline comparison is computed ==		== How the baseline comparison is computed ==

Mikee: /* Fixed */

2009-10-08T19:40:30Z

Fixed

← Older revision		Revision as of 19:40, 8 October 2009
Line 15:		Line 15:
	Fixed baseline comparisons can be used when you want to define a fixed set of data that represents your expected results for each run of the test-bed. The only way to create a fixed baseline currently is by copying it from an existing result set or by importing XML data from an external file -- which would have been either exported from a different test space OR generated by the stride test runner. The snapshot icon ([[image: Camera.png]]) shown in the [[STRIDE_Test_Space#Space_View\|testspace view]] allows you to create a baseline from a particular result set. When you select this button, a form will be displayed to allow you to specify a name and optional duration limit percent value to use to calculate duration upper bounds for all test cases. For example, if you enter 20 in this field, the duration upper bound will be calculated by multiplying the duration value for each test case by 1.2 (20% above the nominal value). If you leave this duration field blank when creating a baseline, no upper bound values will be assigned to the baseline. This baseline creation process ''copies'' the data in the result set to the fixed baseline - once the baseline is created, any changes to the original result set have no impact on the fixed baseline data.		Fixed baseline comparisons can be used when you want to define a fixed set of data that represents your expected results for each run of the test-bed. The only way to create a fixed baseline currently is by copying it from an existing result set or by importing XML data from an external file -- which would have been either exported from a different test space OR generated by the stride test runner. The snapshot icon ([[image: Camera.png]]) shown in the [[STRIDE_Test_Space#Space_View\|testspace view]] allows you to create a baseline from a particular result set. When you select this button, a form will be displayed to allow you to specify a name and optional duration limit percent value to use to calculate duration upper bounds for all test cases. For example, if you enter 20 in this field, the duration upper bound will be calculated by multiplying the duration value for each test case by 1.2 (20% above the nominal value). If you leave this duration field blank when creating a baseline, no upper bound values will be assigned to the baseline. This baseline creation process ''copies'' the data in the result set to the fixed baseline - once the baseline is created, any changes to the original result set have no impact on the fixed baseline data.

	This screencast ~~demonstrates this concept~~:		This screencast shows how to create fixed baselines:

	<dhflashplayer>file=TS_BaselineFixedCreate.mp4\|width=640\|height=480\| path=http://www.s2technologies.com/video/wiki/\|more_than_one=true</dhflashplayer>		<dhflashplayer>file=TS_BaselineFixedCreate.mp4\|width=640\|height=480\| path=http://www.s2technologies.com/video/wiki/\|more_than_one=true</dhflashplayer>

Mikee at 19:09, 8 October 2009

2009-10-08T19:09:28Z

← Older revision		Revision as of 19:09, 8 October 2009
Line 7:		Line 7:
	=== Sequential ===		=== Sequential ===
	Sequential baseline comparison simply compares each result set to the previous result set (previous based on upload order). This kind of comparison can be useful for stable test-beds that are expected to produce the same results with each run of the tests. If your test-bed is changing regularly, has inherent instability (variation in results), or is under active development, then this kind of comparison offers little value. Sequential comparison can be activated for any test space by editing the testspace settings so as to enable the ''auto compare sequentially'' property.		Sequential baseline comparison simply compares each result set to the previous result set (previous based on upload order). This kind of comparison can be useful for stable test-beds that are expected to produce the same results with each run of the tests. If your test-bed is changing regularly, has inherent instability (variation in results), or is under active development, then this kind of comparison offers little value. Sequential comparison can be activated for any test space by editing the testspace settings so as to enable the ''auto compare sequentially'' property.


	This screencast demonstrates this concept:		This screencast demonstrates this concept:
Line 28:		Line 27:
	In addition, the individual test cases that make up the baseline data can be modified - specifically the status can be changed and the duration upper bound can be set or modified. These values are modified by first selecting the baseline name in the list view to get to the result set view for the baseline data. Once you have navigated to the desired test case (using hyperlink/tree navigation), you can hover near one of these values (status or duration limit) and select the ''edit'' link that appears. Make your changes using the displayed form and submit. When editing the duration limit value, you can enter floating point values representing the upper bound (in milliseconds) OR a percentage string (including the percent sign, e.g. "20%") and the duration limit by applying this percentage increase to the test case's duration value.		In addition, the individual test cases that make up the baseline data can be modified - specifically the status can be changed and the duration upper bound can be set or modified. These values are modified by first selecting the baseline name in the list view to get to the result set view for the baseline data. Once you have navigated to the desired test case (using hyperlink/tree navigation), you can hover near one of these values (status or duration limit) and select the ''edit'' link that appears. Make your changes using the displayed form and submit. When editing the duration limit value, you can enter floating point values representing the upper bound (in milliseconds) OR a percentage string (including the percent sign, e.g. "20%") and the duration limit by applying this percentage increase to the test case's duration value.

	~~VIDEO HERE - editing~~ baseline data		This screencast shows how to edit baseline data:

			<dhflashplayer>file=TS_BaselineFixedEdit.mp4\|width=640\|height=480\| path=http://www.s2technologies.com/video/wiki/\|more_than_one=true</dhflashplayer>

	Once you have creates baseline data, you can configure the testspace to automatically compare new result sets with one or more of the baselines. To do this, edit the testspace properties and select the ''add fixed baseline compare'' under the ''auto compare'' settings. Using the drop-list of existing fixed baselines, select the desired baseline and submit. You can also remove previously set baseline comparisons (if you no longer want to compare with certain baselines) using the testspace properties page.		Once you have creates baseline data, you can configure the testspace to automatically compare new result sets with one or more of the baselines. To do this, edit the testspace properties and select the ''add fixed baseline compare'' under the ''auto compare'' settings. Using the drop-list of existing fixed baselines, select the desired baseline and submit. You can also remove previously set baseline comparisons (if you no longer want to compare with certain baselines) using the testspace properties page.

Mikee at 17:33, 8 October 2009

2009-10-08T17:33:25Z

← Older revision		Revision as of 17:33, 8 October 2009
Line 16:		Line 16:
	Fixed baseline comparisons can be used when you want to define a fixed set of data that represents your expected results for each run of the test-bed. The only way to create a fixed baseline currently is by copying it from an existing result set or by importing XML data from an external file -- which would have been either exported from a different test space OR generated by the stride test runner. The snapshot icon ([[image: Camera.png]]) shown in the [[STRIDE_Test_Space#Space_View\|testspace view]] allows you to create a baseline from a particular result set. When you select this button, a form will be displayed to allow you to specify a name and optional duration limit percent value to use to calculate duration upper bounds for all test cases. For example, if you enter 20 in this field, the duration upper bound will be calculated by multiplying the duration value for each test case by 1.2 (20% above the nominal value). If you leave this duration field blank when creating a baseline, no upper bound values will be assigned to the baseline. This baseline creation process ''copies'' the data in the result set to the fixed baseline - once the baseline is created, any changes to the original result set have no impact on the fixed baseline data.		Fixed baseline comparisons can be used when you want to define a fixed set of data that represents your expected results for each run of the test-bed. The only way to create a fixed baseline currently is by copying it from an existing result set or by importing XML data from an external file -- which would have been either exported from a different test space OR generated by the stride test runner. The snapshot icon ([[image: Camera.png]]) shown in the [[STRIDE_Test_Space#Space_View\|testspace view]] allows you to create a baseline from a particular result set. When you select this button, a form will be displayed to allow you to specify a name and optional duration limit percent value to use to calculate duration upper bounds for all test cases. For example, if you enter 20 in this field, the duration upper bound will be calculated by multiplying the duration value for each test case by 1.2 (20% above the nominal value). If you leave this duration field blank when creating a baseline, no upper bound values will be assigned to the baseline. This baseline creation process ''copies'' the data in the result set to the fixed baseline - once the baseline is created, any changes to the original result set have no impact on the fixed baseline data.

	~~VIDEO HERE - how to create fixed baseline from existing result set~~		This screencast demonstrates this concept:

			<dhflashplayer>file=TS_BaselineFixedCreate.mp4\|width=640\|height=480\| path=http://www.s2technologies.com/video/wiki/\|more_than_one=true</dhflashplayer>

	Fixed baselines can be viewed and edited by going to the baseline tab in your testspace view. The baseline list view supports the following operations for each baseline:		Fixed baselines can be viewed and edited by going to the baseline tab in your testspace view. The baseline list view supports the following operations for each baseline:

Mikee: /* Fixed */

2009-10-07T23:53:30Z

Fixed

← Older revision		Revision as of 23:53, 7 October 2009
Line 14:		Line 14:

	=== Fixed ===		=== Fixed ===
	Fixed baseline comparisons can be used when you want to define a fixed set of data that represents your expected results for each run of the test-bed. The only way to create a fixed baseline currently is by copying it from an existing result set or by importing XML data from an external file -- which would have been either exported from a different test space OR generated by the stride test runner. The snapshot (~~camera icon~~) ~~show in the result set table~~ in the [[STRIDE_Test_Space#Space_View\|testspace view]] allows you to create a baseline from a particular result set. When you select this button, a form will be displayed to allow you to specify a name and optional duration limit percent value to use to calculate duration upper bounds for all test cases. For example, if you enter 20 in this field, the duration upper bound will be calculated by multiplying the duration value for each test case by 1.2 (20% above the nominal value). If you leave this duration field blank when creating a baseline, no upper bound values will be assigned to the baseline. This baseline creation process ''copies'' the data in the result set to the fixed baseline - once the baseline is created, any changes to the original result set have no impact on the fixed baseline data.		Fixed baseline comparisons can be used when you want to define a fixed set of data that represents your expected results for each run of the test-bed. The only way to create a fixed baseline currently is by copying it from an existing result set or by importing XML data from an external file -- which would have been either exported from a different test space OR generated by the stride test runner. The snapshot icon ([[image: Camera.png]]) shown in the [[STRIDE_Test_Space#Space_View\|testspace view]] allows you to create a baseline from a particular result set. When you select this button, a form will be displayed to allow you to specify a name and optional duration limit percent value to use to calculate duration upper bounds for all test cases. For example, if you enter 20 in this field, the duration upper bound will be calculated by multiplying the duration value for each test case by 1.2 (20% above the nominal value). If you leave this duration field blank when creating a baseline, no upper bound values will be assigned to the baseline. This baseline creation process ''copies'' the data in the result set to the fixed baseline - once the baseline is created, any changes to the original result set have no impact on the fixed baseline data.

	VIDEO HERE - how to create fixed baseline from existing result set		VIDEO HERE - how to create fixed baseline from existing result set

Mikee: /* Fixed */

2009-10-07T23:49:09Z

Fixed

← Older revision		Revision as of 23:49, 7 October 2009
Line 14:		Line 14:

	=== Fixed ===		=== Fixed ===
	Fixed baseline comparisons can be used when you want to define a fixed set of data that represents your expected results for each run of the test-bed. The only way to create a fixed baseline currently is by copying it from an existing result set or by importing XML data from an external file (which would have been either exported from a different test space OR generated by the stride test runner. The snapshot (camera icon) show in the result set table in the [[STRIDE_Test_Space#Space_View\|testspace view]] allows you to create a baseline from a particular result set. When you select this button, a form will be displayed to allow you to specify a name and optional duration limit percent value to use to calculate duration upper bounds for all test cases. For example, if you enter 20 in this field, the duration upper bound will be calculated by multiplying the duration value for each test case by 1.2 (20% above the nominal value). If you leave this duration field blank when creating a baseline, no upper bound values will be assigned to the baseline. This baseline creation process ''copies'' the data in the result set to the fixed baseline - once the baseline is created, any changes to the original result set have no impact on the fixed baseline data.		Fixed baseline comparisons can be used when you want to define a fixed set of data that represents your expected results for each run of the test-bed. The only way to create a fixed baseline currently is by copying it from an existing result set or by importing XML data from an external file -- which would have been either exported from a different test space OR generated by the stride test runner. The snapshot (camera icon) show in the result set table in the [[STRIDE_Test_Space#Space_View\|testspace view]] allows you to create a baseline from a particular result set. When you select this button, a form will be displayed to allow you to specify a name and optional duration limit percent value to use to calculate duration upper bounds for all test cases. For example, if you enter 20 in this field, the duration upper bound will be calculated by multiplying the duration value for each test case by 1.2 (20% above the nominal value). If you leave this duration field blank when creating a baseline, no upper bound values will be assigned to the baseline. This baseline creation process ''copies'' the data in the result set to the fixed baseline - once the baseline is created, any changes to the original result set have no impact on the fixed baseline data.

	VIDEO HERE - how to create fixed baseline from existing result set		VIDEO HERE - how to create fixed baseline from existing result set

Mikee: /* Sequential */

2009-10-07T23:43:55Z

Sequential

← Older revision		Revision as of 23:43, 7 October 2009
Line 9:		Line 9:


	This screencast demonstrates ~~these concepts~~:		This screencast demonstrates this concept:

	<dhflashplayer>file=TS_BaselineSequential.mp4\|width=640\|height=480\| path=http://www.s2technologies.com/video/wiki/\|more_than_one=true</dhflashplayer>		<dhflashplayer>file=TS_BaselineSequential.mp4\|width=640\|height=480\| path=http://www.s2technologies.com/video/wiki/\|more_than_one=true</dhflashplayer>

Mikee: /* Sequential */

2009-10-07T23:43:35Z

Sequential

← Older revision		Revision as of 23:43, 7 October 2009
Line 8:		Line 8:
	Sequential baseline comparison simply compares each result set to the previous result set (previous based on upload order). This kind of comparison can be useful for stable test-beds that are expected to produce the same results with each run of the tests. If your test-bed is changing regularly, has inherent instability (variation in results), or is under active development, then this kind of comparison offers little value. Sequential comparison can be activated for any test space by editing the testspace settings so as to enable the ''auto compare sequentially'' property.		Sequential baseline comparison simply compares each result set to the previous result set (previous based on upload order). This kind of comparison can be useful for stable test-beds that are expected to produce the same results with each run of the tests. If your test-bed is changing regularly, has inherent instability (variation in results), or is under active development, then this kind of comparison offers little value. Sequential comparison can be activated for any test space by editing the testspace settings so as to enable the ''auto compare sequentially'' property.

	~~VIDEO HERE - how to activate sequential baselines~~
			This screencast demonstrates these concepts:

			<dhflashplayer>file=TS_BaselineSequential.mp4\|width=640\|height=480\| path=http://www.s2technologies.com/video/wiki/\|more_than_one=true</dhflashplayer>

	=== Fixed ===		=== Fixed ===

← Older revision		Revision as of 21:08, 15 October 2014
Line 10:		Line 10:
	This screencast demonstrates this concept:		This screencast demonstrates this concept:

	<~~dhflashplayer>file=TS_BaselineSequential.mp4\|~~width=640\|height=480~~\| path=~~http://www.s2technologies.com/video/wiki/~~\|more_than_one=true~~</~~dhflashplayer~~>		<html5media width="640" height="480">http://www.s2technologies.com/video/wiki/TS_BaselineSequential.mp4</html5media>

	=== Fixed ===		=== Fixed ===
Line 17:		Line 17:
	This screencast shows how to create fixed baselines:		This screencast shows how to create fixed baselines:

	<~~dhflashplayer>file=TS_BaselineFixedCreate.mp4\|~~width=640\|height=480~~\| path=~~http://www.s2technologies.com/video/wiki/~~\|more_than_one=true~~</~~dhflashplayer~~>		<html5media width="640" height="480">http://www.s2technologies.com/video/wiki/TS_BaselineFixedCreate.mp4</html5media>

	Fixed baselines can be viewed and edited by going to the baseline tab in your testspace view. The baseline list view supports the following operations for each baseline:		Fixed baselines can be viewed and edited by going to the baseline tab in your testspace view. The baseline list view supports the following operations for each baseline:
Line 29:		Line 29:
	This screencast shows how to edit baseline data:		This screencast shows how to edit baseline data:

	<~~dhflashplayer>file=TS_BaselineFixedEdit.mp4\|~~width=640\|height=480~~\| path=~~http://www.s2technologies.com/video/wiki/~~\|more_than_one=true~~</~~dhflashplayer~~>		<html5media width="640" height="480">http://www.s2technologies.com/video/wiki/TS_BaselineFixedEdit.mp4</html5media>

	Once you have creates baseline data, you can configure the testspace to automatically compare new result sets with one or more of the baselines. To do this, edit the testspace properties and select the ''add fixed baseline compare'' under the ''auto compare'' settings. Using the drop-list of existing fixed baselines, select the desired baseline and submit. You can also remove previously set baseline comparisons (if you no longer want to compare with certain baselines) using the testspace properties page.		Once you have creates baseline data, you can configure the testspace to automatically compare new result sets with one or more of the baselines. To do this, edit the testspace properties and select the ''add fixed baseline compare'' under the ''auto compare'' settings. Using the drop-list of existing fixed baselines, select the desired baseline and submit. You can also remove previously set baseline comparisons (if you no longer want to compare with certain baselines) using the testspace properties page.
Line 35:		Line 35:
	This screencast shows how to apply fixed baselines:		This screencast shows how to apply fixed baselines:

	<~~dhflashplayer>file=TS_BaselineFixedApply.mp4\|~~width=640\|height=480~~\| path=~~http://www.s2technologies.com/video/wiki/~~\|more_than_one=true~~</~~dhflashplayer~~>		<html5media width="640" height="480">http://www.s2technologies.com/video/wiki/TS_BaselineFixedApply.mp4</html5media>

	== How the baseline comparison is computed ==		== How the baseline comparison is computed ==