Technical Reviews: A Behaviorally Motivated Program of Research

Software engineers use a number of different types of software development technical review (SDTR) for the purpose of detecting defects in software products. This paper applies the behavioral theory of group performance to explain the outcomes of software reviews. A program of empirical research is developed, including propositions to both explain review performance and identify ways of improving review performance based on the specific strengths of individuals and groups. Its contributions are to clarify our understanding of what drives defect detection performance in SDTRs and to set an agenda for future research. In identifying individuals' task expertise as the primary driver of review performance, the research program suggests specific points of leverage for substantially improving review performance. It points to the importance of understanding software reading expertise and implies the need for a reconsideration of existing approaches to managing reviews.

[1]  M. E. Shaw.  A comparison of individuals and small groups in the rational solution of complex problems. , 1932 .

[2]  Patrick R. Laughlin,et al.  Group size, member ability, and social decision schemes on an intellective task. , 1975 .

[3]  E. Ramsden.  Group Process and Productivity , 1973 .

[4]  N. Pennington.  Stimulus structures and mental representations in expert comprehension of computer programs , 1987, Cognitive Psychology.

[5]  Edward F. Weller,et al.  Lessons from three years of inspection data (software development) , 1993, IEEE Software.

[6]  Robert D. Galliers,et al.  Coping with information technology? How British executives perceive the key information systems management issues in the mid-1990s , 1994, J. Inf. Technol..

[7]  Philip Yetton,et al.  The Effect Of The Review Process On Auditor Judgments , 1985 .

[8]  I. Lakatos.  Falsification and the Methodology of Scientific Research Programmes , 1976 .

[9]  Steve Williams,et al.  Does training in problem solving improve the quality of group decisions , 1991 .

[10]  Fred Niederman,et al.  Information Systems Management Issues for the 1990s , 1991, MIS Q..

[11]  John C. Knight,et al.  An improved inspection technique , 1993, CACM.

[12]  Herbert Solomon,et al.  Individual Performance and Group Performance in Problem Solving Related to Group Size and Previous Exposure to the Problem , 1959 .

[13]  M. Diehl,et al.  Productivity loss in brainstorming groups: Toward the solution of a riddle. , 1987 .

[14]  Dick Bowman,et al.  Principles of software engineering management , 1989, APLQ.

[15]  J. Davitz,et al.  A survey of studies contrasting the quality of group performance and individual performance, 1920-1957. , 1958, Psychological bulletin.

[16]  B BisantDavid,et al.  A Two-Person Inspection Method to Improve Programming Productivity , 1989 .

[17]  Mark C. Paulk,et al.  Capability Maturity Model , 1991 .

[18]  Philip Yetton,et al.  Group Size and Performance: Prediction of Failure by Loan Officers , 1981 .

[19]  Pei Hsia,et al.  Learning to Put Lessons Into Practice , 1993, IEEE Softw..

[20]  Jay Hall,et al.  The Effects of a Normative Intervention on Group Decision-Making Performance , 1970 .

[21]  A. Frank Ackerman,et al.  Software inspections: an effective verification process , 1989, IEEE Software.

[22]  Philip Yetton,et al.  Individual versus group problem solving: An empirical test of a best-member strategy , 1982 .

[23]  J. Levine,et al.  Progress in Small Group Research , 1990 .

[24]  Robert C. Ziller,et al.  Group Size: A Determinant of the Quality and Stability of Group Decisions , 1957 .

[25]  Gerald M. Weinberg,et al.  Handbook of Walkthroughs, Inspections, and Technical Reviews: Evaluating Programs, Projects, and Products , 1990 .

[26]  Harvey Siy,et al.  Identifying the mechanisms driving code inspection costs and benefits , 1996 .

[27]  A. Tenbrunsel,et al.  Organizational Behavior and Human Decision Processes , 2013 .

[28]  Lionel E. Deimel,et al.  Applying program comprehension techniques to improve software inspections , 1994 .

[29]  Philip Yetton,et al.  Improving Group Performance by Training in Individual Problem Solving , 1987 .

[30]  Priscilla J. Fowler,et al.  Software inspections and the industrial production of software , 1984 .

[31]  Zenon W. Pylyshyn,et al.  Computation and Cognition: Toward a Foundation for Cognitive Science , 1984 .

[32]  Warren E. Watson,et al.  A realistic test of individual versus group consensus decision making. , 1989 .

[33]  Harvey P. Siy,et al.  An experiment to assess the cost-benefits of code inspections in large scale software development , 1995, SIGSOFT '95.

[34]  P. Yetton,et al.  The relationships among group size, member ability, social decision schemes, and performance , 1983 .

[35]  Thomas A. Corbi,et al.  Program Understanding: Challenge for the 1990s , 1989, IBM Syst. J..

[36]  Ken T. Trotman,et al.  The Review Process And The Accuracy Of Auditor Judgments , 1985 .

[37]  Wei-Tek Tsai,et al.  N-Fold inspection: a requirements analysis technique , 1990, Commun. ACM.

[38]  Robert G. Ebenau,et al.  Software Inspection Process , 1993 .

[39]  Charles R. Holloman,et al.  Individual versus Group Effectiveness in Solving Factual and Nonfactual Problems , 1970 .

[40]  Robin M. Hogarth,et al.  Quality of Group Judgment , 1977 .

[41]  Thomas Gilb,et al.  Software Inspection , 1994 .

[42]  I. Lakatos,et al.  Criticism and the Growth of Knowledge: Falsification and the Methodology of Scientific Research Programmes , 1970 .

[43]  Victor R. Basili,et al.  Software Quality: An Overview from the Perspective of Total Quality Management , 1994, IBM Syst. J..

[44]  Raymond Madachy,et al.  Analysis of a successful inspection program , 1993 .

[45]  C E Ehrlich,et al.  The challenge of the 1990s. , 1990, Gynecologic oncology.

[46]  Gerald M. Weinberg,et al.  Reviews, Walkthroughs, and Inspections , 1984, IEEE Transactions on Software Engineering.

[47]  Dewayne E. Perry,et al.  Prototyping a process monitoring experiment , 1994, Proceedings of 1993 15th International Conference on Software Engineering.

[48]  M. E. Shaw.  Group dynamics : the psychology of small group behavior , 1971 .

[49]  Philip M. Johnson,et al.  Assessing software review meetings: A controlled experimental study using CSRS , 1997, Proceedings of the (19th) International Conference on Software Engineering.

[50]  James R. Lyle,et al.  A Two-Person Inspection Method to Improve Prog ramming Productivity , 1989, IEEE Transactions on Software Engineering.

[51]  Roger K. Blashfield,et al.  Performance of a composite as a function of the number of judges , 1978 .

[52]  Warren E. Watson,et al.  Member competence, group interaction, and group decision making: A longitudinal study. , 1991 .

[53]  J. Hackman,et al.  Group tasks, group interaction process, and group performance effectiveness: A review and proposed integration , 1975 .

[54]  F. MacDonald,et al.  A review of tool support for software inspection , 1995, Proceedings Seventh International Workshop on Computer-Aided Software Engineering.

[55]  Chris Sauer,et al.  Validating the defect detection performance advantage of group designs for software reviews: report of a laboratory experiment using program code , 1997, ESEC '97/FSE-5.

[56]  F. J. Lukey.  Understanding and debugging programs , 1980 .

[57]  Philip Yetton,et al.  An integration of process and decision scheme explanations of group problem solving performance , 1988 .

[58]  Lawrence G. Votta,et al.  Does every inspection need a meeting? , 1993, SIGSOFT '93.

[59]  Harvey P. Siy,et al.  An Experiment ot Assess the Cost-Benefits of Code Inspections in Large Scale Software Development , 1997, IEEE Trans. Software Eng..

[60]  P. Yetton,et al.  Individual and Group Judgments of Internal Control Systems , 1983 .

[61]  Glenford J. Myers,et al.  A controlled experiment in program testing and code walkthroughs/inspections , 1978, CACM.

[62]  Joseph E. McGrath.  Small group research , 1966 .

[63]  Michael E. Fagan.  Design and Code Inspections to Reduce Errors in Program Development (Reprint) , 2002, Software Pioneers.

[64]  Wei-Tek Tsai,et al.  Distributed, collaborative software inspection , 1993, IEEE Software.

[65]  Robert B. Grady,et al.  Key lessons in achieving widespread inspection use , 1994, IEEE Software.

[66]  Edward Yourdon,et al.  Structured walkthroughs , 1978 .

[67]  Russell S. Winer,et al.  Group process and decision performance in a simulated marketing environment , 1987 .

[68]  Herbert Solomon,et al.  Two models of group behavior in the solution of eureka-type problems , 1955 .

[69]  Adam A. Porter,et al.  An experiment to assess different defect detection methods for software requirements inspections , 1994, Proceedings of 16th International Conference on Software Engineering.

[70]  J. A. Adams,et al.  Psychological bulletin. , 1962, Psychological bulletin.

[71]  Robert B. Grady,et al.  Practical Software Metrics for Project Management and Process Improvement , 1992 .

[72]  David Lorge Parnas,et al.  Active design reviews: principles and practices , 1985, ICSE '85.

[73]  Michael E. Fagan.  Design and Code Inspections to Reduce Errors in Program Development , 1976, IBM Syst. J..

[74]  Elliot Soloway,et al.  Empirical Studies of Programmers: Second Workshop , 1991 .

[75]  Stephen G. Eick,et al.  Estimating software fault content before coding , 1992, International Conference on Software Engineering.

[76]  Rebecca A. Henry,et al.  Improving Group Judgment Accuracy: Information Sharing and Determining the Best Member , 1995 .

[77]  Wei-Tek Tsai,et al.  An experimental study of fault detection in user requirements documents , 1992, TSEM.

[78]  R. L. Thorndike.  On what type of task will a group do well , 1938 .

[79]  Adam A. Porter,et al.  Comparing Detection Methods for Software Requirements Inspections: A Replicated Experiment , 1995, IEEE Trans. Software Eng..

[80]  Chris Sauer,et al.  Validating the defect detection performance advantage of group designs for software reviews: report of a replicated experiment , 1997, Proceedings of Australian Software Engineering Conference ASWEC 97.

[81]  Akira K. Onoma,et al.  Practical Steps Toward Quality Development , 1995, IEEE Softw..

[82]  Ken T. Trotman,et al.  Member Variation, Recognition of Expertise, and Group Performance , 1987 .

[83]  Priscilla J. Fowler.  In-process inspections of workproducts at AT&T , 1986, AT&T Technical Journal.