TheEffectivenessofPairProgramming:AMeta-Analysis

Several experiments on the effects of pair versus solo programming have been reported in the literature. We present a metaanalysis of these studies. The analysis shows a small significant positive overall effect of pair programming on quality, a medium significant positive overall effect on duration, and a medium significant negative overall effect on effort. However, between-study variance is significant, and there are signs of publication bias among published studies on pair programming. A more detailed examination of the evidence suggests that pair programming is faster than solo programming when programming task complexity is low and yields code solutions of higher quality when task complexity is high. The higher quality for complex tasks comes at a price of considerably greater effort, while the reduced completion time for the simpler tasks comes at a price of noticeably lower quality. We conclude that greater attention should be given to moderating factors on the effects of pair programming.

[1]  A. Ehrenberg,et al.  The Design of Replicated Studies , 1993 .

[2]  Michelene T. H. Chi,et al.  Expertise in Problem Solving. , 1981 .

[3]  Tore Dybå,et al.  Applying Systematic Reviews to Diverse Study Types: An Experience Report , 2007, ESEM 2007.

[4]  N. Charness,et al.  Expert Performance Its Structure and Acquisition , 2002 .

[5]  Jan Chong,et al.  The Social Dynamics of Pair Programming , 2007, 29th International Conference on Software Engineering (ICSE'07).

[6]  Nancy Pennington,et al.  Comprehension strategies in programming , 1987 .

[7]  Barry W. Boehm,et al.  An empirical comparison between pair development and software inspection in Thailand , 2006, ISESE '06.

[8]  Matthias M. Müller,et al.  Two controlled experiments concerning the comparison of pair programming to peer review , 2005, J. Syst. Softw..

[9]  James Miller,et al.  Applying meta-analytical procedures to software engineering experiments , 2000, J. Syst. Softw..

[10]  Kipling D. Williams,et al.  PROCESSES Social Loafing: A Meta-Analytic Review and Theoretical Integration , 2022 .

[11]  S Duval,et al.  Trim and Fill: A Simple Funnel‐Plot–Based Method of Testing and Adjusting for Publication Bias in Meta‐Analysis , 2000, Biometrics.

[12]  John T. Nosek,et al.  The case for collaborative programming , 1998, CACM.

[13]  Tore Dybå,et al.  A systematic review of effect size in software engineering experiments , 2007, Inf. Softw. Technol..

[14]  M. Chi,et al.  Eliciting Self‐Explanations Improves Understanding , 1994 .

[15]  R. Lennox,et al.  Conventional wisdom on measurement: A structural equation perspective. , 1991 .

[16]  D. Altman,et al.  Measuring inconsistency in meta-analyses , 2003, BMJ : British Medical Journal.

[17]  Alex J Sutton,et al.  Performance of the trim and fill method in the presence of publication bias and between‐study heterogeneity , 2007, Statistics in medicine.

[18]  Jonathan A C Sterne,et al.  Regression Methods to Detect Publication and Other Bias in Meta-Analysis , 2006 .

[19]  Magne Jørgensen,et al.  The Role of Deliberate Artificial Design Elements in Software Engineering Experiments , 2008, IEEE Transactions on Software Engineering.

[20]  Pablo Romero,et al.  "Talking the talk": Is intermediate-level conversation the key to the pair programming success story? , 2007, Agile 2007 (AGILE 2007).

[21]  R. Tweedie,et al.  A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis , 2000 .

[22]  Mario Piattini,et al.  Evaluating performances of pair designing in industry , 2007, J. Syst. Softw..

[23]  N. Kerr,et al.  Group performance and decision making. , 2004, Annual review of psychology.

[24]  Judy A. Siguaw,et al.  Formative versus Reflective Indicators in Organizational Measure Development: A Comparison and Empirical Illustration , 2006 .

[25]  R. Rosenthal,et al.  Meta-analysis: recent developments in quantitative methods for literature reviews. , 2001, Annual review of psychology.

[26]  Laurie A. Williams,et al.  Strengthening the Case for Pair Programming , 2000, IEEE Softw..

[27]  Matthias M. Müller,et al.  A preliminary study on the impact of a pair design phase on pair programming and solo programming , 2006, Inf. Softw. Technol..

[28]  N. Pennington.  Stimulus structures and mental representations in expert comprehension of computer programs , 1987, Cognitive Psychology.

[29]  L. Hedges,et al.  Fixed- and random-effects models in meta-analysis. , 1998 .

[30]  Forrest Shull,et al.  Are Two Heads Better than One? On the Effectiveness of Pair Programming , 2007, IEEE Software.

[31]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[32]  I. Steiner.  Group process and productivity , 1972 .

[33]  Tore Dybå,et al.  Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise , 2007, IEEE Transactions on Software Engineering.

[34]  Casper Lassenius,et al.  Effects of pair programming at the development team level: an experiment , 2005, 2005 International Symposium on Empirical Software Engineering, 2005..

[35]  Marjan Hericko,et al.  Tracking Test First Pair Programming - An Experiment , 2002, XP/Agile Universe.

[36]  Barry Boehm,et al.  A Replicate Empirical Comparison between Pair Development and Software Development with Inspection , 2007, ESEM 2007.

[37]  Cheryl Burke Jarvis,et al.  A Critical Review of Construct Indicators and Measurement Model Misspecification in Marketing and Consumer Research , 2003 .

[38]  J. Levine,et al.  Progress in Small Group Research , 1990 .

[39]  Tore Dybå,et al.  A systematic review of quasi-experiments in software engineering , 2009, Inf. Softw. Technol..

[40]  Tore Dybå,et al.  A Systematic Review of Theory Use in Software Engineering Experiments , 2007, IEEE Transactions on Software Engineering.

[41]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[42]  Amela Karahasanovic,et al.  A survey of controlled experiments in software engineering , 2005, IEEE Transactions on Software Engineering.

[43]  Kate Ehrlich,et al.  Knowledge and processes in the comprehension of computer programs. , 1988 .

[44]  Jeffrey M. Voas.  Quality Time - Faster, Better, and Cheaper , 2001, IEEE Softw..

[45]  Alex J. Sutton,et al.  Evidence Concerning the Consequences of Publication and Related Biases , 2006 .

[46]  Rupert Brown,et al.  Group Processes: Dynamics Within and Between Groups , 1988 .

[47]  C. Perrow.  A FRAMEWORK FOR THE COMPARATIVE ANALYSIS OF ORGANIZATIONS , 1967 .

[48]  Lech Madeyski.  On the Effects of Pair Programming on Thoroughness and Fault-Finding Effectiveness of Unit Tests , 2007, PROFES.

[49]  Shaochun Xu,et al.  Empirical Validation of Test-Driven Pair Programming in Game Development , 2006, 5th IEEE/ACIS International Conference on Computer and Information Science and 1st IEEE/ACIS International Workshop on Component-Based Software Engineering,Software Architecture and Reuse (ICIS-COMSAR'06).

[50]  Tore Dybå,et al.  Evidence-Based Software Engineering for Practitioners , 2005, IEEE Softw..

[51]  Paul J. Feltovich,et al.  Categorization and Representation of Physics Problems by Experts and Novices , 1981, Cogn. Sci..

[52]  Laurie A. Williams,et al.  Pair Programming Illuminated , 2002 .

[53]  Tore Dybå,et al.  A systematic review of statistical power in software engineering experiments , 2006, Inf. Softw. Technol..

[54]  Thorvald Haerem,et al.  The influence of degree of expertise and objective task complexity on perceived task complexity and performance. , 2007, The Journal of applied psychology.

[55]  John E. Hunter,et al.  Fixed Effects vs. Random Effects Meta‐Analysis Models: Implications for Cumulative Research Knowledge , 2000 .