Gast: Generic Automated Software Testing

Software testing is a labor-intensive, and hence expensive, yet heavily used technique to control quality. In this paper we introduce GAST, a fully automatic test tool. Properties about functions and datatypes can be expressed in first order logic. GAST automatically and systematically generates appropriate test data, evaluates the property for these values, and analyzes the test results. This makes it easier and cheaper to test software components. The distinguishing property of our system is that the test data are generated in a systematic and generic way using generic programming techniques. This implies that there is no need for the user to indicate how data should be generated. Moreover, duplicated tests are avoided, and for finite domains GAST is able to prove a property by testing it for all possible values. As an important side-effect, it also encourages stating formal properties of the software.

[1]  Simon L. Peyton Jones,et al.  Report on the programming language Haskell: a non-strict, purely functional language version 1.2 , 1992, SIGP.

[2]  Thierry Jéron,et al.  Automated test generation from SDL specifications , 1999, SDL Forum.

[3]  Marinus J. Plasmeijer,et al.  Theorem Proving for functional Programmers - Sparkle: A Functional Theorem Prover , 2001 .

[4]  Ralf Hinze,et al.  Haskell 98 — A Non−strict‚ Purely Functional Language , 1999 .

[5]  David Lee,et al.  Principles and methods of testing finite state machines-a survey , 1996, Proc. IEEE.

[6]  Hendrik Brinksma,et al.  Côte de Resyste : Automated Model Based Testing , 2002 .

[7]  Marko C. J. D. van Eekelen,et al.  Theorem Proving for Functional Programmers , 2001, IFL.

[8]  Simon L. Peyton Jones,et al.  Derivable Type Classes , 2001, Haskell.

[9]  Koen Claessen,et al.  Testing monadic code with QuickCheck , 2002, ACM SIGPLAN Notices.

[10]  Ralf Hinze,et al.  Generic Haskell: Practice and Theory , 2003, Generic Programming.

[11]  Rinus Plasmeijer,et al.  Concurrent Clean Language Report - version 1.3 , 1998 .

[12]  Thierry Jéron,et al.  Using On-The-Fly Verification Techniques for the Generation of test Suites , 1996, CAV.

[13]  M.C.J.D. van Eekelen,et al.  Reasoning about explicit strictness in a lazy language using mixed lazy/strict semantics , 2002 .

[14]  Ralf Hinze,et al.  Polytypic values possess polykinded types , 2000, Sci. Comput. Program..

[15]  Colin Runciman,et al.  Inductive benchmarking for purely functional data structures , 2001, J. Funct. Program..

[16]  Koen Claessen,et al.  QuickCheck: a lightweight tool for random testing of Haskell programs , 2000, ICFP.

[17]  Michel R. V. Chaudron,et al.  Software Engineering with Formal Methods: The Development of a Storm Surge Barrier Control System Revisiting Seven Myths of Formal Methods , 2001, Formal Methods Syst. Des..

[18]  Nicolae Goga,et al.  Formal Test Automation: A Simple Experiment , 1999, IWTCS.

[19]  Loe M. G. Feijs,et al.  Conformance Testing of a Multimedia System Using PHACT , 1998, IWTCS.

[20]  Marie-Claude Gaudel,et al.  Software testing based on formal specifications: a theory and a tool , 1991, Softw. Eng. J..

[21]  M PaulHudakEt,et al.  Report on the programming language haskell: a non-strict , 1992 .

[22]  Kenneth J. Turner,et al.  Protocol-Inspired Hardware Testing , 1999, IWTCS.

[23]  Marinus J. Plasmeijer,et al.  A Generic Programming Extension for Clean , 2001, IFL.

[24]  Ralf Hinze,et al.  Generic Haskell: practice and theory , 2003 .

[25]  Jan Tretmans,et al.  Testing Transition Systems: An Annotated Bibliography , 2000, MOVEP.