Experiences and Challenges of Introducing Risk-Based Testing in an Industrial Project

Risk-based testing has a high potential to improve the software test process as it helps to optimize the allocation of resources and provides decision support for the management. But for many organizations the integration of risk-based testing into an existing test process is a challenging task. In this paper we present a generic risk-based testing methodology and a procedure how it can be introduced in a test process. Based on this procedure we derive four stages of risk-based test integration, i.e., initial risk-based testing, risk-based test reporting, risk-based test planning, and optimization of risk-based testing. We then discuss how this procedure could be applied based on an industrial project and identify several challenges and lessons learned in introducing risk-based testing.

[1]  Ruth Breu,et al.  Integrating Manual and Automatic Risk Assessment for Risk-Based Testing , 2012, SWQD.

[2]  Karl E. Wiegers.  First Things First: Prioritizing Requirements , 1999 .

[3]  Barry W. Boehm,et al.  Bridge the Gap between Software Test Process and Business Value: A Case Study , 2009, ICSP.

[4]  Alec Dorling,et al.  SPICE: Software Process Improvement and Capability Determination , 1993, Software Quality Journal.

[5]  Theodorich Kopetzky,et al.  Value-Based Coverage Measurement in Requirements-Based Testing: Lessons Learned from an Approach Implemented in the TOSCA Testsuite , 2012, 2012 38th Euromicro Conference on Software Engineering and Advanced Applications.

[6]  Richard Turner,et al.  CMMI Distilled: A Practical Introduction to Integrated Process Improvement , 2001 .

[7]  Michael Steiner,et al.  Make test process assessment similar to software process assessment—the Test SPICE approach , 2012, J. Softw. Evol. Process..

[8]  Rudolf Ramler,et al.  What Software Repositories Should Be Mined for Defect Predictors? , 2009, 2009 35th Euromicro Conference on Software Engineering and Advanced Applications.

[9]  Rex Black.  Advanced Software Testing - Vol. 1: Guide to the ISTQB Advanced Certification as an Advanced Test Analyst (Rockynook Computing) , 2008 .

[10]  Neil Thompson,et al.  Risk Based E-Business Testing , 2002 .

[11]  Martin Pol,et al.  Test process improvement: a practical step-by-step guide to structured testing , 1999 .

[12]  Ståle Amland.  Risk-based testing: : Risk analysis fundamentals and metrics for software testing including a financial application case study , 2000, J. Syst. Softw..

[13]  Barry W. Boehm,et al.  Improving software testing process: feature prioritization to make winners of success‐critical stakeholders , 2012, J. Softw. Evol. Process..

[14]  Felix Redmill.  Exploring risk-based testing and its implications: Research Articles , 2004 .

[15]  Erik van Veenendaal.  The PRISMA Approach , 2012 .

[16]  Anne Marsden,et al.  International Organization for Standardization , 2014 .

[17]  Raymond A. DeCarlo,et al.  Software release control using defect based quality estimation , 2004, 15th International Symposium on Software Reliability Engineering.

[18]  Leo van der Aalst,et al.  TMap Next, for result-driven testing , 2006 .

[19]  Shari Lawrence Pfleeger.  Risky business: what we have yet to learn about risk management , 2000, J. Syst. Softw..

[20]  Felix Redmill,et al.  Exploring risk‐based testing and its implications , 2004, Softw. Test. Verification Reliab..

[21]  Felix Redmill.  Theory and practice of risk-based testing: Research Articles , 2005 .

[22]  Felix Redmill.  Theory and practice of risk‐based testing , 2005, Softw. Test. Verification Reliab..

[23]  Software Engineering Risk Management: A Just-in-Time Approach , 1995 .

[24]  Joseph P. Cavano,et al.  A framework for the measurement of software quality , 1978 .

[25]  Stefan Biffl,et al.  Value-Based Management of Software Testing , 2006, Value-Based Software Engineering.

[26]  T. Saaty,et al.  The Analytic Hierarchy Process , 1985 .

[27]  Norman E. Fenton,et al.  Quantitative Analysis of Faults and Failures in a Complex Software System , 2000, IEEE Trans. Software Eng..

[28]  Vahid Garousi,et al.  Trustworthy Software Development Processes, International Conference on Software Process, ICSP 2009 Vancouver, Canada, May 16-17, 2009 Proceedings , 2009, ICSP.