An XP Experiment with Students - Setup and Problems

Designing experiments to be carried out with students as subjects in an XP setup is a difficult task: Students lack experiences with XP, there are limited resources, the experiment might not be taken seriously and other effects interfere. This paper presents an experiment using student subjects examining test-first in comparison to classical-testing. We proved several hypotheses about test coverage, number of test-cases, contacts with customer, acceptance for test-first, development speed and not required features. While designing the experiment we noticed that it is useful to include some additional XP techniques on top of test first, because of our special setup and the demands we had. Despite careful planning and conduction of the experiment we still faced a number of problems. In this paper we also discuss the problems with our experimental setup.