Are fit tables really talking?

Test-driven software development tackles the problem of operationally defining the features to be implemented by means of test cases. This approach was recently ported to the early development phase, when requirements are gathered and clarified. Among the existing proposals, Fit (Framework for Integrated Testing) supports the precise specification of requirements by means of so called Fit tables, which express relevant usage scenarios in a tabular format, easily understood also by the customer. Fit tables can be turned into executable test cases through the creation of pieces of glue code, called fixtures. In this paper, we test the claimed benefits of Fit through a series of three controlled experiments in which Fit tables and related fixtures are used to clarify a set of change requirements, in a software evolution scenario. Results indicate improved correctness achieved with no significant impact on time, however benefits of Fit vary in a substantial way depending on the developers' experience. Preliminary results on the usage of Fit in combination with pair programming revealed another relevant source of variation.

[1]  Natalia Juristo Juzgado,et al.  Basics of Software Engineering Experimentation , 2010, Springer US.

[2]  Frank Maurer,et al.  FitClipse: A Fit-Based Eclipse Plug-In for Executable Acceptance Test Driven Development , 2007, XP.

[3]  Marc J. Balcer,et al.  The category-partition method for specifying and generating fuctional tests , 1988, CACM.

[4]  Mike Chiasson,et al.  Executable acceptance tests for communicating business requirements: customer perspective , 2006, AGILE 2006 (AGILE'06).

[5]  Marco Torchiano,et al.  Talking tests: an empirical assessment of the role of fit acceptance tests in clarifying requirements , 2007, IWPSE '07.

[6]  Bertrand Meyer,et al.  On Formalism in Specifications , 1985, IEEE Software.

[7]  Peter M. Chisnall,et al.  Questionnaire Design, Interviewing and Attitude Measurement , 1993 .

[8]  Walfredo Cirne,et al.  EasyAccept: a tool to easily create, run and drive development with automated acceptance tests , 2006, AST '06.

[9]  Marco Torchiano,et al.  Electronic Communications of the EASST Volume 8 ( 2008 ) Proceedings of the Third International ERCIM Symposium on Software Evolution ( Software Evolution 2007 ) The Use of Executable FIT Tables to support Maintenance and Evolution Tasks , 2007 .

[10]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[11]  Ralph R. Young.  Effective Requirements Practices , 2001 .

[12]  D R Shearer,et al.  Acceptance testing. , 1998, Radiology management.

[13]  Karl R. P. H. Leung,et al.  Generating User Acceptance Test Plans from Test Cases , 2007, 31st Annual International Computer Software and Applications Conference (COMPSAC 2007).

[14]  Rick Mugridge,et al.  Fit for Developing Software: Framework for Integrated Tests (Robert C. Martin) , 2005 .

[15]  Grigori Melnik,et al.  Examining Usage Patterns of the FIT Acceptance Testing Framework , 2005, XP.

[16]  Grigori Melnik,et al.  Suitability of FIT User Acceptance Tests for Specifying Functional Requirements: Developer Perspective , 2004, XP/Agile Universe.

[17]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[18]  Johan Andersson,et al.  XP with Acceptance-Test Driven Development: A Rewrite Project for a Resource Optimization System , 2003, XP.

[19]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.