Using acceptance tests as a support for clarifying requirements: A series of experiments

One of the main reasons for the failure of many software projects is the late discovery of a mismatch between the customers' expectations and the pieces of functionality implemented in the delivered system. At the root of such a mismatch is often a set of poorly defined, incomplete, under-specified, and inconsistent requirements. Test driven development has recently been proposed as a way to clarify requirements during the initial elicitation phase, by means of acceptance tests that specify the desired behavior of the system. The goal of the work reported in this paper is to empirically characterize the contribution of acceptance tests to the clarification of the requirements coming from the customer. We focused on Fit tables, a way to express acceptance tests, which can be automatically translated into executable test cases. We ran two experiments with students from University of Trento and Politecnico of Torino, to assess the impact of Fit tables on the clarity of requirements. We considered whether Fit tables actually improve requirement understanding and whether this requires any additional comprehension effort. Experimental results show that Fit helps in the understanding of requirements without requiring a significant additional effort.