Integrating inspection and test processes based on context‐specific assumptions

Inspections and testing are two of the most commonly performed software quality assurance processes today. Typically, these processes are applied in isolation, which, however, fails to exploit the benefits of systematically combining and integrating them. In consequence, tests are not focused on the basis of early defect detection data. Expected benefits of such process integration include higher defect detection rates or reduced quality assurance effort. Moreover, when conducting testing without any prior information regarding the system's quality, it is often unclear how to focus testing. A systematic integration of inspection and testing processes requires context‐specific knowledge about the relationships between inspections and testing. This knowledge is typically not available and needs to be empirically identified and validated. Often, context‐specific assumptions can be seen as a starting point for generating such knowledge. On the basis of the integrated inspection and testing approach, In2Test, which uses inspection data to focus testing, we present in this article how knowledge about the relationship between inspections and testing can be gained, documented, and evolved in an analytical or empirical manner. In addition, this article gives an overview of related work and highlights future research directions. Copyright © 2012 John Wiley & Sons, Ltd.

[1]  Frank Elberzhager,et al.  A systematic mapping study on the combination of static and dynamic quality assurance techniques , 2012, Inf. Softw. Technol..

[2]  Oliver Laitenberger,et al.  Studying the effects of code inspection and structural testing on software quality , 1998, Proceedings Ninth International Symposium on Software Reliability Engineering (Cat. No.98TB100257).

[3]  Khaled El Emam,et al.  Fraunhofer Institute for Experimental Software Engineering , 1997, Softw. Process. Improv. Pract..

[4]  Gary Gack Managing the Black Hole: The Executive's Guide to Software Project Risk , 2010 .

[5]  Nikolai Tillmann,et al.  Automating Software Testing Using Program Analysis , 2008, IEEE Software.

[6]  Andreas Zeller,et al.  Mining metrics to predict component failures , 2006, ICSE.

[7]  RunesonPer,et al.  What Do We Know about Defect Detection Methods , 2006 .

[8]  James Miller,et al.  Comparing and combining software defect detection techniques: a replicated empirical study , 1997, ESEC '97/FSE-5.

[9]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[10]  Ayse Basar Bener,et al.  Data mining source code for locating software bugs: A case study in telecommunication industry , 2009, Expert Syst. Appl..

[11]  Jürgen Münch,et al.  Experiences with Systematic Reuse: Applying the EF/QIP Approach , 1997 .

[12]  Antonia Bertolino,et al.  Software Testing Research: Achievements, Challenges, Dreams , 2007, Future of Software Engineering (FOSE '07).

[13]  T. H. Tse,et al.  Test case prioritization for regression testing of service-oriented business applications , 2009, WWW '09.

[14]  Per Runeson,et al.  Detection or isolation of defects? An experimental comparison of unit testing and code inspection , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[15]  Dale Karolak,et al.  Software engineering risk management , 1995 .

[16]  C. Csallner,et al.  Check 'n' crash: combining static checking and testing , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[17]  Michael Eichberg,et al.  A Handbook of Software and Systems Engineering , 2009 .

[18]  Dale W. Karolak Software Engineering Risk Management: Finding Your Path through the Jungle , 1998 .

[19]  D. Ross Jeffery,et al.  Has twenty-five years of empirical software engineering made a difference? , 2002, Ninth Asia-Pacific Software Engineering Conference, 2002..

[20]  Frank Elberzhager,et al.  Inspection and Test Process Integration Based on Explicit Test Prioritization Strategies , 2012, SWQD.

[21]  Mika Mäntylä,et al.  What Types of Defects Are Really Discovered in Code Reviews? , 2009, IEEE Transactions on Software Engineering.

[22]  Victor R. Basili,et al.  A Validation of Object-Oriented Design Metrics as Quality Indicators , 1996, IEEE Trans. Software Eng..

[23]  Norman E. Fenton,et al.  Quantitative Analysis of Faults and Failures in a Complex Software System , 2000, IEEE Trans. Software Eng..

[24]  Natalia Juristo Juzgado,et al.  Reviewing 25 Years of Testing Technique Experiments , 2004, Empirical Software Engineering.

[25]  Khaled El Emam,et al.  The application of subjective estimates of effectiveness to controlling software inspections , 2000, J. Syst. Softw..

[26]  Lionel C. Briand,et al.  A comparison and integration of capture-recapture models and the detection profile method , 1998, Proceedings Ninth International Symposium on Software Reliability Engineering (Cat. No.98TB100257).

[27]  Frank Elberzhager,et al.  Optimizing cost and quality by integrating inspection and test processes , 2011, ICSSP '11.

[28]  Elaine J. Weyuker,et al.  We're Finding Most of the Bugs, but What are We Missing? , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[29]  Gregg Rothermel,et al.  Selecting a Cost-Effective Test Case Prioritization Technique , 2004, Software Quality Journal.

[30]  Claes Wohlin,et al.  State‐of‐the‐art: software inspections after 25 years , 2002, Softw. Test. Verification Reliab..

[31]  Barbara Paech,et al.  Exploring the relationship of a file's history and its fault-proneness: An empirical method and its application to open source programs , 2010, Inf. Softw. Technol..

[32]  Dieter Rombach,et al.  People-oriented capture, display, and use of process information , 2009 .

[33]  Khaled El Emam,et al.  The Optimal Class Size for Object-Oriented Software , 2002, IEEE Trans. Software Eng..

[34]  Tore Dybå,et al.  The Future of Empirical Methods in Software Engineering Research , 2007, Future of Software Engineering (FOSE '07).

[35]  Elaine J. Weyuker,et al.  The distribution of faults in a large industrial software system , 2002, ISSTA '02.

[36]  Lionel C. Briand,et al.  A systematic and comprehensive investigation of methods to build and evaluate fault prediction models , 2010, J. Syst. Softw..

[37]  Jianguo Chen,et al.  Combining Model Checking and Testing for Software Analysis , 2008, 2008 International Conference on Computer Science and Software Engineering.

[38]  Claes Wohlin,et al.  Capture-recapture in software inspections after 10 years research--theory, evaluation and application , 2004, J. Syst. Softw..

[39]  Stefan Biffl,et al.  Investigating the Temporal Behavior of Defect Detection in Software Inspection and Inspection-Based Testing , 2010, PROFES.

[40]  Norman E. Fenton,et al.  A Critique of Software Defect Prediction Models , 1999, IEEE Trans. Software Eng..

[41]  A. Zeller,et al.  If Your Bug Database Could Talk . . . , 2006 .