A Systematic Approach to Risk-Based Testing Using Risk-annotated Requirements Models

Nowadays, software-intensive systems continuously pervade several areas of daily life, even critical ones, and replace established mechanical or manual solutions. Development and quality assurance methods have to ensure that these software-intensive systems are delivered both with adequate quality, optimized resources and within the scheduled time frame. The idea of risk-based testing is to prioritize testing activities to what is deemed critical for the software-intensive system. Although there is a common agreement that risk-based testing techniques ought to be rigorously applied, especially for safety- and security-critical systems, there is actually little knowledge available on how to systematically come to risk-optimized test suites. This paper presents a novel approach to risk-based testing that deals with the transition from risk management and requirements engineering to test design activities and test case generation by using models. The main contribution of the paper is the description of a methodology that allows an easy combination of test generation directives and risk level in order to generate risk-optimized test suites.

[1]  Marc-Florian Wendland,et al.  Requirements-Driven Testing with Behavior Trees , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[2]  Toby Myers The Foundation for a Scaleable Methodology for Systems Design , 2010 .

[3]  Yanping Chen,et al.  Specification-based regression test selection with risk analysis , 2002, CASCON.

[4]  R. Eschbach,et al.  From Requirements to Statistical Testing of Embedded Systems , 2007, Fourth International Workshop on Software Engineering for Automotive Systems (SEAS '07).

[5]  Felix Redmill Theory and practice of risk‐based testing , 2005, Softw. Test. Verification Reliab..

[6]  Ståle Amland Hulda Garborgsv Risk Based Testing and Metrics Risk Analysis Fundamentals and Metrics for software testing including a Financial Application case study , 2002 .

[7]  Klaus Pohl,et al.  An automated technique for risk-based test case generation and prioritization , 2008, AST '08.

[8]  Peter A. Lindsay,et al.  Execution of natural language requirements using State Machines synthesised from Behavior Trees , 2012, J. Syst. Softw..

[9]  Shari Lawrence Pfleeger Risky business: what we have yet to learn about risk management , 2000, J. Syst. Softw..

[10]  Ståle Amland Risk-based testing: : Risk analysis fundamentals and metrics for software testing including a financial application case study , 2000, J. Syst. Softw..

[11]  Jürgen Großmann,et al.  Establishing a Service-Oriented Tool Chain for the Development of Domain-Independent MBT Scenarios , 2010, 2010 17th IEEE International Conference and Workshops on Engineering of Computer Based Systems.

[12]  R. Geoff Dromey Genetic Design: Amplifying Our Ability to Deal With Requirements Complexity , 2003, Scenarios: Models, Transformations and Tools.

[13]  Philipp Zech Risk-Based Security Testing in Cloud Computing Environments , 2011, 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation.

[14]  Peter A. Lindsay,et al.  An Automated Failure Mode and Effect Analysis Based on High-Level Design Specification with Behavior Trees , 2005, IFM.

[15]  Andreas Metzger,et al.  Employing Requirements Metrics for Automating Early Risk Assessment , 2007 .

[16]  Felix Redmill,et al.  Exploring risk‐based testing and its implications , 2004, Softw. Test. Verification Reliab..

[17]  Bran Selic,et al.  A Systematic Approach to Domain-Specific Language Design Using UML , 2007, 10th IEEE International Symposium on Object and Component-Oriented Real-Time Distributed Computing (ISORC'07).

[18]  James Bach,et al.  Heuristic Risk-Based Testing , 1999 .

[19]  R. Geoff Dromey,et al.  From requirements to design: formalizing the key steps , 2003, First International Conference onSoftware Engineering and Formal Methods, 2003.Proceedings..

[20]  Robert Eschbach,et al.  Risk-Based Testing of Safety-Critical Embedded Systems Driven by Fault Tree Analysis , 2011, 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops.

[21]  Thomas Bauer,et al.  Risk-based Statistical Testing: A Refinement- based Approach to the Reliability Analysis of Safety-Critical Systems , 2009 .