Besouro: A framework for exploring compliance rules in automatic TDD behavior assessment

Abstract Context The improvements promoted by Test-Driven Design (TDD) have not been confirmed by quantitative assessment studies. To a great extent, the problem lies in the lack of a rigorous definition for TDD. An emerging approach has been to measure the conformance of TDD practices with the support of automated systems that embed an operational definition, which represent the specific TDD process assumed and the validation tests used to determine its presence and quantity. The empirical construction of TDD understanding and consensus building requires the ability of comparing different definitions, evaluating them with regard to practitioners’ perception, and exploring code information for improvement of automatic assessment. Objective This paper describes Besouro, a framework targeted at the development of systems for automatic TDD behavior assessment. The main rationale of Besouro’s design is the ability to compare distinct operational definitions, evaluate them with regard to users’ perception, and explore code information for further analysis and conformance assessment improvement. Method We developed an architecture with clear separation of concerns, which enables to vary: (a) the atomic events and respective metrics to be collected from developing and testing environments; (b) the organization of atomic events in streams of actions or processes; and (c) the classification and assessment components for each set of operational definitions adopted. The architecture also includes a mechanism for on-line user assessment awareness and feedback, and integrates event-related information with the respective code in a code version system. Results We illustrate the usefulness of Besouro’s features for understanding the actions and processes underlying TDD through a prototype developed to support an experiment based on user feedback. We show how it was possible to compare variations of a same operational definition by exploring users’ feedback, and use source code to improve the automatic classification of TDD practices. Conclusion Understanding the actions and processes underlying successful TDD application is key for leveraging TDD benefits. In the absence of a rigorous definition for TDD, the proposed approach aims at building consensus from experimentation and empirical validation.