The effect of experience on the test-driven development process

We conducted a quasi-experiment to compare the characteristics of experts’ and novices’ test-driven development processes. Our novices were 11 computers science students who participated in an Extreme Programming lab course, the expert group consisted of seven professionals who had industrial experience in test-driven development. The novices as well as two of the experts worked in a laboratory environment whereas the remaining five experts worked in their office. The experts complied more to the rules of test-driven development and had shorter test-cycles than the novices. The tests written by the experts were of higher quality in terms of statement and block coverage as well. All reported results are statistically significant on the 5% level. We conclude that the results of studies which evaluate performance of test-driven development using subjects inexperienced in TDD are not easily generalisable.

[1]  D. Wolfe,et al.  Nonparametric Statistical Methods. , 1974 .

[2]  Marco Torchiano,et al.  On the effectiveness of the test-first approach to programming , 2005, IEEE Transactions on Software Engineering.

[3]  Matthias M. Müller,et al.  Experiment about test-first programming , 2002, IEE Proc. Softw..

[4]  James Miller,et al.  A prototype empirical evaluation of test driven development , 2004, 10th International Symposium on Software Metrics, 2004. Proceedings..

[5]  Kent L. Beck,et al.  Test-driven Development - by example , 2002, The Addison-Wesley signature series.

[6]  Hakan Erdogmus,et al.  Extreme Programming and Agile Methods - XP/Agile Universe 2004 , 2004, Lecture Notes in Computer Science.

[7]  Hakan Erdogmus,et al.  The Role of Process Measurement in Test-Driven Development , 2004, XP/Agile Universe.

[8]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[9]  T. Vidmar,et al.  Towards empirical evaluation of test-driven development in a university environment , 2003, The IEEE Region 8 EUROCON 2003. Computer as a Tool..

[10]  Johannes Link,et al.  Unit Testing in Java: How Tests Drive the Code , 2003 .

[11]  Johannes Link.  Softwaretests mit JUnit - Techniken der testgetriebenen Entwicklung (2. Aufl.) , 2005 .

[12]  Mario Piattini,et al.  Evaluating advantages of test driven development: a controlled experiment with professionals , 2006, ISESE '06.

[13]  Walter F. Tichy,et al.  Renaming Detection , 2004, Automated Software Engineering.

[14]  Matthias M. Müller,et al.  Extreme Programming in Curriculum: Experiences from Academia and Industry , 2004, XP.

[15]  Nachiappan Nagappan,et al.  Evaluating the efficacy of test-driven development: industrial case studies , 2006, ISESE '06.

[16]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[17]  Boby George,et al.  An initial investigation of test driven development in industry , 2003, SAC '03.

[18]  Steve Freeman,et al.  Endo-testing: unit testing with mock objects , 2001 .