Evaluating the efficacy of test-driven development: industrial case studies

This paper discusses software development using the Test Driven Development (TDD) methodology in two different environments (Windows and MSN divisions) at Microsoft. In both these case studies we measure the various context, product and outcome measures to compare and evaluate the efficacy of TDD. We observed a significant increase in quality of the code (greater than two times) for projects developed using TDD compared to similar projects developed in the same organization in a non-TDD fashion. The projects also took at least 15% extra upfront time for writing the tests. Additionally, the unit tests have served as auto documentation for the code when libraries/APIs had to be used as well as for code maintenance.

[1]  Jeffrey C. Carver,et al.  Issues in using students in empirical studies in software engineering education , 2003, Proceedings. 5th International Workshop on Enterprise Networking and Computing in Healthcare Industry (IEEE Cat. No.03EX717).

[2]  Victor R. Basili,et al.  Iterative and incremental developments. a brief history , 2003, Computer.

[3]  Matthias M. Müller,et al.  Experiment about test-first programming , 2002, IEE Proc. Softw..

[4]  K. Beck,et al.  Extreme Programming Explained , 2002 .

[5]  Boby George,et al.  A structured experiment of test-driven development , 2004, Inf. Softw. Technol..

[6]  Bill Curtis.  Three Problems Overcome with Behavioral Models of the Software Development Process , 1989, ICSE.

[7]  Walter F. Tichy,et al.  Case study: extreme programming in a university environment , 2001, Proceedings of the 23rd International Conference on Software Engineering. ICSE 2001.

[8]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[9]  Michael S. Deutsch,et al.  Software Quality Engineering , 1988 .

[10]  Watts S. Humphrey,et al.  Managing the software process , 1989, The SEI series in software engineering.

[11]  B. Curtis.  Three Problems Overcome With Behavioral Models Of The Software Development Process , 1989, 11th International Conference on Software Engineering.

[12]  Marco Torchiano,et al.  On the effectiveness of the test-first approach to programming , 2005, IEEE Transactions on Software Engineering.

[13]  Laurie A. Williams,et al.  Test-driven development as a defect-reduction practice , 2003, 14th International Symposium on Software Reliability Engineering, 2003. ISSRE 2003..

[14]  Bill Curtis.  Modeling the software process: three problems overcome with behavioral models of the software development process (panel session) , 1989, ICSE 1989.

[15]  Jessica Keyes,et al.  IEEE Standard Dictionary of Measures to Produce Reliable Software , 2002 .

[16]  Boby George,et al.  An initial investigation of test driven development in industry , 2003, SAC '03.

[17]  Alex M. Andrew,et al.  Software Quality Engineering , 1998 .

[18]  Kent L. Beck,et al.  Test-driven Development - by example , 2002, The Addison-Wesley signature series.

[19]  Lucas Layman,et al.  Extreme programming evaluation framework for object-oriented languages -- version 1.1 , 2003 .

[20]  Kent L. Beck,et al.  Extreme programming explained - embrace change , 1990 .