Critical Issues on Test-Driven Development

During the last decade, Test-Driven Development (TDD) has been actively discussed in the software engineering community. It has been regarded as a useful and beneficial software development practice as well in industry as in academia. After a decade of active research, there is still very little critical discussion on TDD in the literature. This paper is based on a literature review and it is focused on identifying and introducing critical viewpoints on TDD. First, the current evidence on TDD's benefits is still weak and it includes several issues. Second, the paper presents a number of other possible issues and challenges with TDD that are referred in the literature. Finally, based on the findings, a list of concrete research questions for the future research is presented.

[1]  Lei Zhang,et al.  Comparison Between Test Driven Development and Waterfall Development in a Small-Scale Project , 2006, XP.

[2]  Barry Boehm,et al.  Balancing Agility and Dis-cipline: A Guide for the Perplexed , 2003 .

[3]  Sami Kollanus,et al.  Understanding TDD in academic environment: experiences from two experiments , 2008 .

[4]  T. Vidmar,et al.  Towards empirical evaluation of test-driven development in a university environment , 2003, The IEEE Region 8 EUROCON 2003. Computer as a Tool..

[5]  Stephen H. Edwards.  Using software testing to move students from trial-and-error to reflection-in-action , 2004 .

[6]  Ville Isomöttönen,et al.  Test-driven development in education: experiences with critical viewpoints , 2008, SIGCSE 2008.

[7]  Lech Madeyski,et al.  The Impact of Test-Driven Development on Software Development Productivity - An Empirical Study , 2007, EuroSPI.

[8]  Kent L. Beck,et al.  Extreme programming explained - embrace change , 1990 .

[9]  Laurie A. Williams,et al.  Assessing test-driven development at IBM , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[10]  Hossein Saiedian,et al.  A Leveled Examination of Test-Driven Development Acceptance , 2007, 29th International Conference on Software Engineering (ICSE'07).

[11]  Arie van Deursen,et al.  Program Comprehension Risks and Opportunities in Extreme Programming , 2001, WCRE.

[12]  Alison Young,et al.  Proceedings of the 13th annual conference on Innovation and technology in computer science education , 2008 .

[13]  Svetlana G. Shasharina,et al.  Web Service Validation Enabling Test-Driven Development of Service-Oriented Applications , 2009, 2009 Congress on Services - I.

[14]  J. Grenning.  Applying test driven development to embedded software , 2007, IEEE Instrumentation & Measurement Magazine.

[15]  Lars Lundberg,et al.  Quality Impact of Introducing Component-Level Test Automation and Test-Driven Development , 2007, EuroSPI.

[16]  Mario Piattini,et al.  Evaluating advantages of test driven development: a controlled experiment with professionals , 2006, ISESE '06.

[17]  Cem Kaner,et al.  Experiences teaching a course in programmer testing , 2005, Agile Development Conference (ADC'05).

[18]  Lech Madeyski.  Preliminary Analysis of the Effects of Pair Programming and Test-Driven Development on the External Code Quality , 2005, Software Engineering: Evolution and Emerging Technologies.

[19]  Boby George,et al.  An initial investigation of test driven development in industry , 2003, SAC '03.

[20]  Lech Madeyski,et al.  The Impact of Pair Programming and Test-Driven Development on Package Dependencies in Object-Oriented Design - An Experiment , 2006, PROFES.

[21]  Laurie A. Williams,et al.  Test-driven development as a defect-reduction practice , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[22]  Pankaj Jalote,et al.  An Experimental Evaluation of the Effectiveness and Efficiency of the Test Driven Development , 2007, ESEM 2007.

[23]  Tong Li,et al.  Evaluation of Test-Driven Development: An Academic Case Study , 2009, SERA.

[24]  Sami Kollanus,et al.  Test-Driven Development - Still a Promising Approach? , 2010, 2010 Seventh International Conference on the Quality of Information and Communications Technology.

[25]  Reidar Conradi,et al.  The Impact of Test Driven Development on the Evolution of a Reusable Framework of Components – An Industrial Case Study , 2008, 2008 The Third International Conference on Software Engineering Advances.

[26]  Mario Piattini,et al.  Productivity of Test Driven Development: A Controlled Experiment with Professionals , 2006, PROFES.

[27]  Raghvinder S. Sangwan,et al.  Test-Driven Development in Large Projects , 2006, IT Professional.

[28]  Dave Astels,et al.  Test Driven Development: A Practical Guide , 2003 .

[29]  S.M. Rahman.  Applying the TBC method in introductory programming courses , 2007, 2007 37th Annual Frontiers In Education Conference - Global Engineering: Knowledge Without Borders, Opportunities Without Passports.

[30]  Görel Hedin,et al.  Teaching extreme programming to large groups of students , 2005, J. Syst. Softw..

[31]  Randy A. Ynchausti.  Integrating Unit Testing Into A Software Development Team’s Process , 2001 .

[32]  Laurie A. Williams,et al.  Realizing quality improvement through test driven development: results and experiences of four industrial teams , 2008, Empirical Software Engineering.

[33]  Arie van Deursen,et al.  Refactoring test code , 2001 .

[34]  Marco Torchiano,et al.  On the effectiveness of the test-first approach to programming , 2005, IEEE Transactions on Software Engineering.

[35]  Laurie A. Williams,et al.  On the Sustained Use of a Test-Driven Development Practice at IBM , 2007, Agile 2007 (AGILE 2007).

[36]  Victor R. Basili,et al.  Iterative and incremental developments. a brief history , 2003, Computer.

[37]  Liang Huang,et al.  Empirical investigation towards the effectiveness of Test First programming , 2009, Inf. Softw. Technol..

[38]  Tore Dybå,et al.  Strength of evidence in systematic reviews in software engineering , 2008, ESEM '08.

[39]  Onur Demirörs,et al.  Conceptual Differences Among Functional Size Measurement Methods , 2007, ESEM 2007.

[40]  Bas Vodde,et al.  Learning Test-Driven Development by Counting Lines , 2007, IEEE Software.

[41]  David S. Janzen,et al.  Does Test-Driven Development Really Improve Software Design Quality? , 2008, IEEE Software.

[42]  Pekka Abrahamsson,et al.  Improving Business Agility Through Technical Solutions: A Case Study on Test-Driven Development in Mobile Software Development , 2005, Business Agility and Information Technology Diffusion.

[43]  Nachiappan Nagappan,et al.  Evaluating the efficacy of test-driven development: industrial case studies , 2006, ISESE '06.

[44]  J. Rasmussen,et al.  Introducing XP into Greenfield Projects: lessons learned , 2003 .

[45]  Scott W. Ambler,et al.  Test-Driven Development of Relational Databases , 2007, IEEE Software.

[46]  Thomas Flohr,et al.  Lessons Learned from an XP Experiment with Students: Test-First Needs More Teachings , 2006, PROFES.

[47]  James Miller,et al.  A prototype empirical evaluation of test driven development , 2004 .

[48]  Kent L. Beck,et al.  Aim, Fire , 2001, IEEE Softw..

[49]  Lisa Crispin,et al.  Driving Software Quality: How Test-Driven Development Impacts Software Quality , 2006, IEEE Software.

[50]  Andrew Rendell.  Effective and Pragmatic Test Driven Development , 2008, Agile 2008 Conference.

[51]  Michele Marchesi,et al.  Extreme Programming and Agile Processes in Software Engineering , 2003, Lecture Notes in Computer Science.

[52]  Lech Madeyski,et al.  The impact of Test-First programming on branch coverage and mutation score indicator of unit tests: An experiment , 2010, Inf. Softw. Technol..

[53]  Pekka Abrahamsson,et al.  Does Test-Driven Development Improve the Program Code? Alarming Results from a Comparative Case Study , 2008, CEE-SET.

[54]  Keith C. C. Chan,et al.  Test Driven Development and Software Process Improvement in China , 2004, XP.

[55]  Stuart Anderson,et al.  Requirements Evolution from Process to Product Oriented Management , 2001, PROFES.

[56]  David S. Janzen,et al.  Empirical Software Engineering in Industry Short Courses , 2007, 20th Conference on Software Engineering Education & Training (CSEET'07).

[57]  John Huan Vu,et al.  Evaluating Test-Driven Development in an Industry-Sponsored Capstone Project , 2009, 2009 Sixth International Conference on Information Technology: New Generations.

[58]  Richard Baskerville,et al.  Business Agility and Information Technology Diffusion - IFIP TC8 WG 8.6 International Working Conference May 8-11, 2005, Atlanta, Georgia, USA , 2005, Business Agility and Information Technology Diffusion.

[59]  P. L. Li,et al.  Estimating the Quality of Widely Used Software Products Using Software Reliability Growth Modeling: Case Study of an IBM Federated Database Project , 2007, ESEM 2007.

[60]  David S. Janzen,et al.  Implications of integrating test-driven development into CS1/CS2 curricula , 2009, SIGCSE '09.

[61]  Matthias M. Müller,et al.  Experiment about test-first programming , 2002, IEE Proc. Softw..

[62]  P. Abrahamsson,et al.  A Comparative Case Study on the Impact of Test-Driven Development on Program Design and Test Coverage , 2007, ESEM 2007.

[63]  Boby George,et al.  A structured experiment of test-driven development , 2004, Inf. Softw. Technol..

[64]  David S. Janzen,et al.  Implications of test-driven development: a pilot study , 2003, OOPSLA '03.

[65]  Kent L. Beck,et al.  Test-driven Development - by example , 2002, The Addison-Wesley signature series.

[66]  Robert C. Martin,et al.  Iterative and Incremental development , 1999 .

[67]  Bertrand Meyer,et al.  Balancing Agility and Formalism in Software Engineering, Second IFIP TC 2 Central and East European Conference on Software Engineering Techniques, CEE-SET 2007, Poznan, Poland, October 10-12, 2007, Revised Selected Papers , 2008, CEE-SET.

[68]  David S. Janzen,et al.  On the Influence of Test-Driven Development on Software Design , 2006, 19th Conference on Software Engineering Education & Training (CSEET'06).

[69]  Tore Dybå,et al.  Empirical studies of agile software development: A systematic review , 2008, Inf. Softw. Technol..