What is Unique About STRIDE

From STRIDE Wiki
Revision as of 23:14, 11 June 2010 by Mikee (talk | contribs) (→‎STRIDE facilitates deeper API/Unit Testing)
Jump to navigation Jump to search

STRIDE is a cross-platform framework

The STRIDE Runtime is written in standard C on top of a simple platform abstraction layer that enables it to work on virtually any target platform. It is delivered as source code to be included in the application's build system. STRIDE also auto-generates harnessing and remoting logic as source code during the make process, removing any dependencies on specific compilers and / or processors. The transport between the host and target is configurable and supports serial/USB and TCP/IP by default. Custom transports are also available. STRIDE is easily configured for single process or multiple process environments (i.e. Linux, Windows CE, etc.). Testing can also be conducted using an off-target environment which is provided for Windows and Linux host machines.

The cross-platform framework facilitates a unified testing approach for all team members, enabling organizations to standardize on a test workflow that is independent of the target platform being used or what branch of software is being changed.

STRIDE software builds are both functional and testable

STRIDE enables software builds to be both fully functional and testable at the same time. The software works exactly the same as before. Whatever the software image was used for in the past -- system testing, developer debugging, etc. -- is still applicable. The STRIDE test logic is separated from the application source code and is NOT executed unless invoked via the runner. The source instrumentation is only active when executing tests. The impact of built-in testability to the software application is nominal. The application can easily be switched back to a non-testable build by simply removing the STRIDE_ENABLED preprocessor directive. This automatically controls all STRIDE related source code and macros. There are no changes required to the build process to enable or disable this functionality.

The built-in testability is similar in concept to a debug build except the focus is on testing. The testable build leverages the existing software build process by integrating into the same make system used by the development team (no one-off or special builds). Automatically included in the build is test automation controllable from the host. When tests are executed, timing analysis is also included with the generated test report. Developers can easily pre-flight test their source code changes before committing them to a baseline. Builds can be automatically regression tested as part of the daily build process.

STRIDE facilitates deeper API/Unit Testing

STRIDE offers unique techniques that can be leveraged for deeper API/Unit test coverage. STRIDE provides support for this type of testing in C/C++. There are no special APIs required to register tests, suites, etc. Just write your test in any combination of C/C++ and the auto-generated intercept module via the STRIDE build tools takes care of everything. Tests from separate teams are automatically aggregated by the system -- no coordination is required.

Writing API/Unit tests in native code is the simplest way to begin validating the software. There is no new language to learn, no proprietary editor, and your normal programming workflow is not interrupted. This type of testing works well for:

  • calling APIs directly
  • validating C++ classes
  • isolating modules
  • critical processing / timing
  • and much more ...

For more advanced testing scenarios, dependencies can be doubled. This feature provides a means for intercepting C/C++ global functions on the target and substituting a stub, fake, or mock. The substitution is all controllable via the runtime, allowing the software to continue executing normally when not running a test.

File fixturing is another technique that can be leveraged to drive better testing. Test code executing on the target platform can perform file operations on the host remotely. This enables opening, reading, writing, etc. files while executing target-based test logic. File fixturing allows bypassing external input into the system for more controlled and isolated testing.

There are numerous other features that can be leveraged to facilitate deeper API/Unit test coverage:

STRIDE includes behavior-based testing techniques

STRIDE leverages source instrumentation to provide behavior-based testing techniques that can be applied to the executing software application. The execution sequencing of the code, along with data, can be automatically validated based on expectations. Behavior-based testing is different than unit testing or API testing. Behavior testing does not focus on calling functions / methods and validating their return values. Behavior testing validates the expected sequencing of the software executing under normal operating conditions. This can span threads and process boundaries, and even multiple targets, as the application(s) is running. Leveraging macros -- called Test Points -- domain experts strategically instrument the source under test. Data can also optionally be associated with a Test Point , which is often critical to validating that the software is behaving as expected. This type of validation can be applied to a wide-range of testing scenerios:

  • State Machines
  • Data flow through system components
  • Sequencing between threads
  • Drivers that don't return values
  • and much more ..

Another key element to STRIDE based behavior testing is that the expected code sequencing is automatically validated. This automatic validation can be used for on-going regression testing. The expectations are realized by simple table definitions along with hooks for customization related to optional data. When failures do occur, context is provided with the file name and associated line number of the unexpected Test Point(s) behavior. The test validation can be implemented in both native target code and scripting on the host. In either case, the validation is done without impacting the application's performance (i.e. target code executed in a background thread and host script executed on desktop).

Software Quality Assurance(SQA) can also add behavior-based testing as part of their existing functional / system testing. Because the runner is a command line utility, it is easily controlled from existing test infrastructure. SQA can also leverage the instrumentation to create their own test suites using scripting that executes in concert with existing test automation.

STRIDE optimizes failure resolution

When executing behavior-based tests or API/unit tests , results can be automatically uploaded to STRIDE Test Space for storing and analyzing. Result data is uploaded manually (using the web interface) or automatically using the runner. Hosting test results in a central location with easy access via a browser enables the entire team to better participate in ensuring quality as apart of the development process. Reports containing test results, timing, tracing logs, and built-in test documentation (supported in both target code and script) all correlated together enhances the context for all team members to manage testing and optimize resolving failures. Team collaboration on testing activities is also facilitated with auto-generated email notifications and messaging. Emails contain links to test failures where file names and line numbers are provided to aid in resolution. Messaging allows team members to more effectively communicate on the specifics of test results while minimizing information required in traditional emails, status reports, etc.

Retrieved from "https://www.stridewiki.com/index.php?title=What_is_Unique_About_STRIDE&oldid=12891"