Stride Overview: Difference between revisions
No edit summary |
|||
Line 1: | Line 1: | ||
__NOTOC__ | __NOTOC__ | ||
Stride™ is a test framework for | '''Stride™''' is a [http://en.wikipedia.org/wiki/XUnit xUnit-style] test framework for C/C++ software comprised of an external test ''Runner'' , ''Runtime'' library, and a test ''Compiler''. It was designed with a specific focus on enabling tests to be executed on a device using a fully functional build. This means that the software works the same as before, but also has built-in tests that can be invoked on-demand by the test runner. Stride has also been integrated with [http://www.testspace.com Testspace] for test content management. | ||
Stride's unique host-target architecture allows easier and better On-Target/Device White-box testing. Software builds automatically become more testable; facilitating deeper code coverage opportunities. | |||
Revision as of 14:30, 6 July 2015
Stride™ is a xUnit-style test framework for C/C++ software comprised of an external test Runner , Runtime library, and a test Compiler. It was designed with a specific focus on enabling tests to be executed on a device using a fully functional build. This means that the software works the same as before, but also has built-in tests that can be invoked on-demand by the test runner. Stride has also been integrated with Testspace for test content management.
Stride's unique host-target architecture allows easier and better On-Target/Device White-box testing. Software builds automatically become more testable; facilitating deeper code coverage opportunities.
Runner
Stride also contains a lightweight Runner application that is a host-based command-line utility for interactive and automated test execution. It also integrates seamlessly with Testspace for test content management.
Runtime
Stride includes a Runtime software source package that supports connectivity with the host system. It is integrated with embedded application software to enable testability to be compiled into the software with minimal impact on performance or the size of the application. The software is agnostic to the RTOS, CPU, and transport. It can be configured to support multi-processes, multi-threads, user/kernel spaces, or a simpler single application process. Typically the runtime is configured as a background thread and only executes when running tests.
Compiler
The Stride Compiler (aka Build Tools) are a set of command-line utilities that integrate with your target software build process. They preprocess standard C/C++ header files that contain STRIDE test declarations to auto-generate source code comprising a custom Intercept Module. The intercept module code is then built with the Stride Runtime to provide fixturing and harnessing test logic as part of your build image.
Cross Platform
Stride works on virtually any target platform. Stride's cross-platform framework facilitates a unified testing approach for all team members, enabling organizations to standardize on a test workflow that is independent of the target platform being used or what branch of software is being changed.
Runtime written in standard C
The Stride Runtime is written in standard C on top of a simple platform abstraction layer that enables it to work across platforms. It can be configured for a single process multi-threading environment or multiple process environments (i.e. Linux, Windows, Embedded RTOS, etc.). It is delivered as source code to be included in the application's build system. The transport between the host and target is configurable and supports Serial and TCP/IP by default. Custom transports are also available. The runtime has also been tailored specifically to embedded applications, overhead is minimal. It consumes very little memory for table and control block storage. Resource usage is configurable and can be tailored to the limitations of the target platform.
Integrates With Your Existing Build System
Stride also auto-generates intercept module used for harnessing and remote logic as source code during the make process, removing any dependencies on specific compilers and / or processors.
Supports Off-Target Testing
Testing can also be conducted using a standard desktop machine (Windows, Linux, or FreeBSD). The Stride SDKs allow for a seamless transition between the real target and an Off-Target host environment.
Rich set of Assertions
Test cases are typically constructed leveraging a large set of available assertion macros. On failures macros provide such details as file name, line number, and why the condition failed. Stride provides are large set of optional macros available for automatic report annotation in the case of failures.
#include <mytest.h>
void MyTest::CheckFoo() {
..
srEXPECT_EQ(foo(2 + 2), 4);
}
void MyTest::CheckBoo() {
..
srEXPECT_GT(boo(3 * 3), 7);
}
Extensive Parameter Support
Parameterized testing is key to reducing test code duplication. Constructor arguments, name-value collections, and host file remoting are all supported. Stride provides an easy way for passing and accessing these types of parameters, allowing customization of test behavior at runtime.
Create an INI-type of file (i.e. myinput.ini)
Loopsize = 10 InputFile = ${MYPATH}/datacontent.bin Toogle = On
Pass the name-value collection using the Stride Runner
$ stride .. --run="MyTest(/path/to/myinput.ini)"
Now simply access your input within test logic using built-in services
{
// .. doing stuff ..
GetParam("Loopsize", buffer, size);
//.. using parameters for test logic
}
Test Doubles & Test Points
Stride offers advanced testing techniques that enable deeper and more effective testing with less effort.
Test Doubles
For more advanced testing scenarios, dependencies can be doubled. This feature provides a means for intercepting C/C++ global functions on the target and substituting a stub, fake, or mock. The substitution is all controllable via the runtime, allowing the software to continue executing normally when not running a test.
For example something like the following could be useful:
void MySuite::test1()
{
// inserting my own version of malloc()
srDOUBLE_SET(malloc, my_malloc_stub);
// calling a routine that uses malloc(). checking handled null correctly
ret_code = FuncThatUsersMalloc(42);
srEXPECT_EQ(ret_code, -1)
// restoring the real malloc()
srDOUBLE_RESET(malloc)
}
Test Points
Stride leverages source instrumentation to provide an additional validation technique called Expectation Testing that can be applied to the executing software application. The execution sequencing of the code, along with state data, can be automatically validated based on what is expected. This validation technique does not focus on calling functions / methods but rather verifies code sequencing. This can span threads, process boundaries, and even multiple targets, as the application(s) is running. Leveraging simple macros -- called Test Points -- developers strategically instrument the source under test.
/* a test point with formatted string payload */
srTEST_POINT_STR("IMPORTANT", "important date= %d", myVar);
The test validation is done without impacting the application's performance (the on-target test code is executed in a background thread and scripts are executed on the host machine). When failures do occur, context is provided with the file name and associated line number of the failed expectations. This type of validation can be applied to a wide-range of testing scenarios:
- State Machines
- Data flow through system components
- Sequencing between threads
- Drivers that don't return values
- and much more ...
The following is a snippet of what the test code might look like
srTestPointExpect_t expected[]={
{"IMPORTANT"}, myCheckData,
{"ANOTHER_TP"},
{0}};
// setup the expections
srTestPointSetup(..);
// start the operations
srTestPointWait(handle, 1000);
Test Implementation
Stride based tests are integrated with the existing build system enabling your software to be both fully functional and testable at the same time. The test logic is separated from the application source code and is not executed unless invoked via the Stride Runner. Any source instrumentation is only active when executing tests. The impact of built-in testability to the software application is nominal.
The test validation can be implemented in both C/C++ on the target and Perl on the host.
C/C++ Test Unit Example
#include <srtest.h>
class MyTest {
public:
void CheckFoo();
void CheckBoo();
};
#pragma scl_test_class(MyTest)
#include <mytest.h>
void MyTest::CheckFoo() {
..
srEXPECT_EQ(foo(2 + 2), 4);
}
void MyTest::CheckBoo() {
..
srEXPECT_GT(boo(3 * 3), 7);
}
Perl Test Script Example
use strict;
use warnings;
package MyTests;
use base qw(STRIDE::Test);
use STRIDE::Test;
sub CheckFoo : Test
{
EXPECT_EQ(Remote->foo(2 + 2), 4);
}
sub CheckBoo : Test
{
EXPECT_GT(Remote->boo(3 * 3), 7);
}
1;
Works with Testspace
Stride have been designed to leverage Testspace features such as test design, execution control, and test analysis. Managing test content and status become better organized and easier to assess current progress. Refer to Testspace website for more details.
For more details refer to our FAQ