Validating the defect detection performance advantage of group designs for software reviews: report of a replicated experiment

It is widely accepted that software development technical reviews (SDTRs) are a useful technique for finding defects in software products. The normative SDTR literature assumes that group reviews are better than individual reviews. Recent debates centre around the need for review meetings. This paper presents the findings of a replicated experiment that was conducted to investigate whether group review meetings are needed and why. We found that an interacting group is the preferred choice over the average individual and artificial (nominal) groups. The source of performance advantage of interacting groups is not synergy as was previously thought, but rather in discriminating between true defects and false positives identified by individual reviewers. As a practical implication, nominal groups may be an alternative review design in situations where individuals exhibit a low level of false positives.

[1]  Edward Yourdon,et al.  Structured walkthroughs , 1978 .

[2]  M. J. Norušis,et al.  SPSS 6.1 Guide to Data Analysis , 1997 .

[3]  Adam A. Porter,et al.  An experiment to assess different defect detection methods for software requirements inspections , 1994, Proceedings of 16th International Conference on Software Engineering.

[4]  Wei-Tek Tsai,et al.  An experimental study of fault detection in user requirements documents , 1992, TSEM.

[5]  Michael E. Fagan.  Design and Code Inspections to Reduce Errors in Program Development , 1976, IBM Syst. J..

[6]  Gudmund J. W. Smith,et al.  The internal consistency of the Humm-Wadsworth Temperament Scale. , 1958 .

[7]  Harvey Siy,et al.  Identifying the mechanisms driving code inspection costs and benefits , 1996 .

[8]  Philip Yetton,et al.  Improving Group Performance by Training in Individual Problem Solving , 1987 .

[9]  Harvey P. Siy,et al.  An experiment to assess cost-benefits of inspection meetings and their alternatives: a pilot study , 1996, Proceedings of the 3rd International Software Metrics Symposium.

[10]  Adam A. Porter,et al.  Comparing Detection Methods for Software Requirements Inspections: A Replicated Experiment , 1995, IEEE Trans. Software Eng..

[11]  John C. Knight,et al.  An improved inspection technique , 1993, CACM.

[12]  Gerald M. Weinberg,et al.  Handbook of Walkthroughs, Inspections, and Technical Reviews: Evaluating Programs, Projects, and Products , 1990 .

[13]  Erik Kamsties,et al.  An Empirical Evaluation of Three Defect-Detection Techniques , 1995, ESEC.

[14]  Chris Sauer,et al.  Validating the defect detection performance advantage of group designs for software reviews: report of a laboratory experiment using program code , 1997, ESEC '97/FSE-5.

[15]  Robert G. Ebenau,et al.  Software Inspection Process , 1993 .

[16]  Lawrence G. Votta,et al.  Does every inspection need a meeting? , 1993, SIGSOFT '93.

[17]  Glenford J. Myers,et al.  A controlled experiment in program testing and code walkthroughs/inspections , 1978, CACM.

[18]  J. Davitz,et al.  A survey of studies contrasting the quality of group performance and individual performance, 1920-1957. , 1958, Psychological bulletin.

[19]  Peter J. Middleton,et al.  Software Inspection , 1994, J. Inf. Technol..

[20]  Chris Sauer,et al.  A Framework for Software Development Technical Reviews , 1994, Software Quality and Productivity.

[21]  Stephen G. Eick,et al.  Estimating software fault content before coding , 1992, International Conference on Software Engineering.

[22]  Eliot R. Smith,et al.  Research methods in social relations , 1962 .

[23]  P. Yetton,et al.  The relationships among group size, member ability, social decision schemes, and performance , 1983 .

[24]  M. E. Shaw.  Group dynamics : the psychology of small group behavior , 1971 .

[25]  Philip M. Johnson,et al.  Assessing software review meetings: A controlled experimental study using CSRS , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[26]  JefferyRoss,et al.  Validating the defect detection performance advantage of group designs for software reviews , 1997 .

[27]  David Lorge Parnas,et al.  Active design reviews: principles and practices , 1985, ICSE '85.

[28]  Philip Yetton,et al.  Individual versus group problem solving: An empirical test of a best-member strategy , 1982 .

[29]  I. Steiner.  Group process and productivity , 1972 .