From test cases to FSMs: augmented test-driven development and property inference

This paper uses the inference of finite state machines from EUnit test suites for Erlang programs to make two contributions. First, we show that the inferred FSMs provide feedback on the adequacy of the test suite that is developed incrementally during the test-driven development of a system. This is novel because the feedback we give is independent of the implementation of the system. Secondly, we use FSM inference to develop QuickCheck properties for testing state-based systems. This has the effect of transforming a fixed set of tests into a property which can be tested using randomly generated data, substantially widening the coverage and scope of the tests.

[1]  Koen Claessen,et al.  QuickCheck: a lightweight tool for random testing of Haskell programs , 2011, SIGP.

[2]  Lech Madeyski,et al.  Test-Driven Development - An Empirical Evaluation of Agile Practice , 2009 .

[3]  Pedram Amini,et al.  Fuzzing: Brute Force Vulnerability Discovery , 2007 .

[4]  David Notkin,et al.  Mutually Enhancing Test Generation and Specification Inference , 2003, FATES.

[5]  Kent L. Beck,et al.  Test-driven Development - by example , 2002, The Addison-Wesley signature series.

[6]  Nikolai Tillmann,et al.  DySy: dynamic symbolic execution for invariant inference , 2008, ICSE.

[7]  Richard Carlsson,et al.  EUnit: a lightweight unit testing framework for Erlang , 2006, ERLANG '06.

[8]  John Hughes,et al.  Testing telecoms software with quviq QuickCheck , 2006, ERLANG '06.

[9]  Francesco Cesarini,et al.  Erlang Programming , 2009 .

[10]  Huiqing Li,et al.  Similar Code Detection and Elimination for Erlang Programs , 2010, PADL.

[11]  Michael D. Ernst,et al.  Finding the needles in the haystack: Generating legal test inputs for object-oriented programs , 2006 .

[12]  John Derrick,et al.  Iterative Refinement of Reverse-Engineered Models by Model-Based Testing , 2009, FM.

[13]  Enrique Vidal,et al.  Grammatical Inference: An Introduction Survey , 1994, ICGI.

[14]  Dave Astels,et al.  Test Driven Development: A Practical Guide , 2003 .

[15]  Neil Walkinshaw,et al.  Reverse Engineering State Machines by Interactive Grammar Inference , 2007, 14th Working Conference on Reverse Engineering (WCRE 2007).

[16]  Stephen McCamant,et al.  The Daikon system for dynamic detection of likely invariants , 2007, Sci. Comput. Program..

[17]  Carlos Pacheco,et al.  Directed random testing , 2009 .

[18]  John Hughes,et al.  QuickCheck Testing for Fun and Profit , 2007, PADL.

[19]  Alexander L. Wolf,et al.  Discovering models of software processes from event-based data , 1998, TSEM.