Empirical Studies on Quality in Agile Practices: A Systematic Literature Review

Nowadays one key question for most organizations is which of the agile practices should be implemented to improve product quality. This systematic literature review surveys studies published up to and including 2009 and attempts to present and evaluate the empirical findings regarding quality in agile practices. The studies were classified into three groups: test driven or test first development, pair programming, and miscellaneous agile practices and methods. The findings of most studies suggest that agile practices can improve quality if they are implemented correctly. The significant findings of this study, in conjunction with previous research, could be used as guidelines for practitioners on their own settings and situations.

[1]  Adam Wojciechowski,et al.  Experimental Evaluation of Pair Programming , 2001 .

[2]  Laurie A. Williams,et al.  Strengthening the Case for Pair Programming , 2000, IEEE Softw..

[3]  Ioannis Stamelos,et al.  Agile Software Development Quality Assurance , 2007 .

[4]  Matthias M. Müller,et al.  Do programmer pairs make different mistakes than solo programmers? , 2006, J. Syst. Softw..

[5]  Mario Piattini,et al.  Evaluating performances of pair designing in industry , 2007, J. Syst. Softw..

[6]  Laurie Williams,et al.  The costs and benefits of pair programming , 2001 .

[7]  Ioannis Stamelos,et al.  An experimental investigation of personality types impact on pair effectiveness in pair programming , 2009, Empirical Software Engineering.

[8]  G. Melnik,et al.  A cross-program investigation of students' perceptions of agile methods , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[9]  C.A. Wellington,et al.  Comparison of student experiences with plan-driven and agile methodologies , 2005, Proceedings Frontiers in Education 35th Annual Conference.

[10]  Nachiappan Nagappan,et al.  Evaluating the efficacy of test-driven development: industrial case studies , 2006, ISESE '06.

[11]  Matthias M. Müller,et al.  Two controlled experiments concerning the comparison of pair programming to peer review , 2005, J. Syst. Softw..

[12]  Charles E. McDowell,et al.  Pair programming improves student retention, confidence, and program quality , 2006, CACM.

[13]  Doris Schmedding,et al.  Pair programming in software development teams - An empirical study of its benefits , 2008, Inf. Softw. Technol..

[14]  Liang Huang,et al.  Empirical investigation towards the effectiveness of Test First programming , 2009, Inf. Softw. Technol..

[15]  Eliza Stefanova,et al.  Analyses of an agile methodology implementation , 2004 .

[16]  Witold Pedrycz,et al.  A Case Study on the Impact of Refactoring on Quality and Productivity in an Agile Team , 2008, CEE-SET.

[17]  Tore Dybå,et al.  Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise , 2007, IEEE Transactions on Software Engineering.

[18]  Marian Gheorghe,et al.  A formal experiment comparing extreme programming with traditional software construction , 2003, Proceedings of the Fourth Mexican International Conference on Computer Science, 2003. ENC 2003..

[19]  David S. Janzen,et al.  Implications of integrating test-driven development into CS1/CS2 curricula , 2009, SIGCSE '09.

[20]  Marco Torchiano,et al.  On the effectiveness of the test-first approach to programming , 2005, IEEE Transactions on Software Engineering.

[21]  Keith C. C. Chan,et al.  Pair programming productivity: Novice-novice vs. expert-expert , 2006, Int. J. Hum. Comput. Stud..

[22]  Dave Astels,et al.  Test Driven Development: A Practical Guide , 2003 .

[23]  Laurie A. Williams,et al.  On the Sustained Use of a Test-Driven Development Practice at IBM , 2007, Agile 2007 (AGILE 2007).

[24]  Lucy Dillon,et al.  Quality in qualitative evaluation: A framework for assessing research evidence. A quality framework. , 2003 .

[25]  Matthias M. Müller,et al.  Experiment about test-first programming , 2002, IEE Proc. Softw..

[26]  Boby George,et al.  A structured experiment of test-driven development , 2004, Inf. Softw. Technol..

[27]  Kent L. Beck,et al.  Test-driven Development - by example , 2002, The Addison-Wesley signature series.

[28]  P. L. Li,et al.  Estimating the Quality of Widely Used Software Products Using Software Reliability Growth Modeling: Case Study of an IBM Federated Database Project , 2007, ESEM 2007.

[29]  John T. Nosek,et al.  The case for collaborative programming , 1998, CACM.

[30]  Lars Lundberg,et al.  Results from introducing component-level test automation and Test-Driven Development , 2006, J. Syst. Softw..

[31]  Shaochun Xu,et al.  Empirical Validation of Test-Driven Pair Programming in Game Development , 2006, 5th IEEE/ACIS International Conference on Computer and Information Science and 1st IEEE/ACIS International Workshop on Component-Based Software Engineering,Software Architecture and Reuse (ICIS-COMSAR'06).

[32]  David S. Janzen,et al.  Does Test-Driven Development Really Improve Software Design Quality? , 2008, IEEE Software.

[33]  Charles E. McDowell,et al.  Program quality with pair programming in CS1 , 2004, ITiCSE '04.

[34]  G. Noblit,et al.  Meta-Ethnography: Synthesizing Qualitative Studies , 1988 .

[35]  T. Vidmar,et al.  Towards empirical evaluation of test-driven development in a university environment , 2003, The IEEE Region 8 EUROCON 2003. Computer as a Tool..

[36]  Pekka Abrahamsson,et al.  A multiple case study on the impact of pair programming on product quality , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[37]  Kent L. Beck,et al.  Extreme programming explained - embrace change , 1990 .

[38]  Barbara A. Kitchenham,et al.  A quantitative approach to monitoring software development , 1989, Softw. Eng. J..

[39]  Laurie A. Williams,et al.  Assessing test-driven development at IBM , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[40]  Lars Lundberg,et al.  Quality Impact of Introducing Component-Level Test Automation and Test-Driven Development , 2007, EuroSPI.

[41]  Joseph Chao,et al.  Critical personality traits in successful pair programming , 2006, AGILE 2006 (AGILE'06).

[42]  Atul Gupta,et al.  An Experimental Evaluation of the Effectiveness and Efficiency of the Test Driven Development , 2007, First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007).

[43]  Tim DeClue,et al.  Pair programming and pair trading: effects on learning and motivation in a CS2 course , 2003 .

[44]  Hwan-Seung Yong,et al.  Distributed agile: project management in a global environment , 2010, Empirical Software Engineering.

[45]  Randy A. Ynchausti.  Integrating Unit Testing Into A Software Development Team’s Process , 2001 .

[46]  Laurie A. Williams,et al.  Realizing quality improvement through test driven development: results and experiences of four industrial teams , 2008, Empirical Software Engineering.

[47]  Lucas Layman,et al.  Exploring extreme programming in context: an industrial case study , 2004, Agile Development Conference.

[48]  Keith C. C. Chan,et al.  When Does a Pair Outperform Two Individuals? , 2003, XP.

[49]  Paul D Jeanne Ellis Ormrod Leedy,et al.  Practical Research: Planning and Design , 1974 .

[50]  Laurie A. Williams,et al.  Test-driven development as a defect-reduction practice , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[51]  Lech Madeyski.  Is External Code Quality Correlated with Programming Experience or Feelgood Factor? , 2006, XP.

[52]  Bjørnar Tessem,et al.  Experiences in Learning XP Practices: A Qualitative Study , 2003, XP.

[53]  Andrew Begel,et al.  Pair programming: what's in it for me? , 2008, ESEM '08.

[54]  Charles E. McDowell,et al.  The effects of pair-programming on performance in an introductory programming course , 2002, SIGCSE '02.