Effects of Test-Driven Development: A Comparative Analysis of Empirical Studies

Test-driven development is a software development practice where small sections of test code are used to direct the development of program units. Writing test code prior to the production code promises several positive effects on the development process itself and on associated products and processes as well. However, there are few comparative studies on the effects of test-driven development. Thus, it is difficult to assess the potential process and product effects when applying test-driven development. In order to get an overview of the observed effects of test-driven development, an in-depth review of existing empirical studies was carried out. The results for ten different internal and external quality attributes indicate that test-driven development can reduce the amount of introduced defects and lead to more maintainable code. Parts of the implemented code may also be somewhat smaller in size and complexity. While maintenance of test-driven code can take less time, initial development may last longer. Besides the comparative analysis, this article sketches related work and gives an outlook on future research.

[1]  K. Perreault,et al.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches , 2011 .

[2]  Laurie A. Williams,et al.  Test-driven development as a defect-reduction practice , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[3]  John Huan Vu,et al.  Evaluating Test-Driven Development in an Industry-Sponsored Capstone Project , 2009, 2009 Sixth International Conference on Information Technology: New Generations.

[4]  Sharan B. Merriam,et al.  Qualitative Research: A Guide to Design and Implementation , 2009 .

[5]  R. Torraco.  Writing Integrative Literature Reviews: Guidelines and Examples , 2005 .

[6]  Liang Huang,et al.  Empirical investigation towards the effectiveness of Test First programming , 2009, Inf. Softw. Technol..

[7]  James Miller,et al.  A prototype empirical evaluation of test driven development , 2004, 10th International Symposium on Software Metrics, 2004. Proceedings..

[8]  David S. Janzen,et al.  Does Test-Driven Development Really Improve Software Design Quality? , 2008, IEEE Software.

[9]  David S. Janzen,et al.  On the Influence of Test-Driven Development on Software Design , 2006, 19th Conference on Software Engineering Education & Training (CSEET'06).

[10]  Shari Lawrence Pfleeger,et al.  Software metrics (2nd ed.): a rigorous and practical approach , 1997 .

[11]  Forrest Shull,et al.  How Effective Is Test-Driven Development? , 2011, Making Software.

[12]  P. L. Li,et al.  Estimating the Quality of Widely Used Software Products Using Software Reliability Growth Modeling: Case Study of an IBM Federated Database Project , 2007, ESEM 2007.

[13]  Pearl Brereton,et al.  Performing systematic literature reviews in software engineering , 2006, ICSE.

[14]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[15]  Laurie A. Williams,et al.  Realizing quality improvement through test driven development: results and experiences of four industrial teams , 2008, Empirical Software Engineering.

[16]  Mario Piattini,et al.  Evaluating advantages of test driven development: a controlled experiment with professionals , 2006, ISESE '06.

[17]  Vojislav B. Misic,et al.  The Effects of Test-Driven Development on External Quality and Productivity: A Meta-Analysis , 2013, IEEE Transactions on Software Engineering.

[18]  Grigori Melnik,et al.  Guest Editors' Introduction: TDD--The Art of Fearless Programming , 2007, IEEE Software.

[19]  Matthias M. Müller,et al.  The effect of experience on the test-driven development process , 2007, Empirical Software Engineering.

[20]  Jay F. Nunamaker,et al.  Comparing the Defect Reduction Benefits of Code Inspection and Test-Driven Development , 2012, IEEE Transactions on Software Engineering.

[21]  Atul Gupta,et al.  An Experimental Evaluation of the Effectiveness and Efficiency of the Test Driven Development , 2007, First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007).

[22]  Julia Eichmann,et al.  Making Software - What Really Works, and Why We Believe It , 2011, Making Software.

[23]  Mauro Pezzè,et al.  Software testing and analysis - process, principles and techniques , 2007 .

[24]  Claes Wohlin,et al.  Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering , 2006 .

[25]  David S. Janzen,et al.  A survey of evidence for test-driven development in academia , 2008, SGCS.

[26]  Lech Madeyski,et al.  The Impact of Test-Driven Development on Software Development Productivity - An Empirical Study , 2007, EuroSPI.

[27]  Laurie A. Williams,et al.  Assessing test-driven development at IBM , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[28]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[29]  Shari Lawrence Pfleeger,et al.  Software Metrics : A Rigorous and Practical Approach , 1998 .

[30]  Mojca Ciglaric,et al.  Impact of test-driven development on productivity, code and tests: A controlled experiment , 2011, Inf. Softw. Technol..

[31]  Nachiappan Nagappan,et al.  Evaluating the efficacy of test-driven development: industrial case studies , 2006, ISESE '06.

[32]  Simo Mäkinen,et al.  Driving Software Quality and Structuring Work Through Test-Driven Development , 2012 .

[33]  Kent L. Beck,et al.  Test-driven Development - by example , 2002, The Addison-Wesley signature series.

[34]  Boby George,et al.  An initial investigation of test driven development in industry , 2003, SAC '03.

[35]  Lech Madeyski,et al.  The impact of Test-First programming on branch coverage and mutation score indicator of unit tests: An experiment , 2010, Inf. Softw. Technol..

[36]  David S. Janzen,et al.  Implications of integrating test-driven development into CS1/CS2 curricula , 2009, SIGCSE '09.

[37]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[38]  Rachel Harrison,et al.  Dynamic and static views of software evolution , 2001, Proceedings IEEE International Conference on Software Maintenance. ICSM 2001.

[39]  David Batic,et al.  The effectiveness of test-driven development: an industrial case study , 2011, Software Quality Journal.