Smart Testing of Functional Programs in Isabelle

We present a novel counterexample generator for the interactive theorem prover Isabelle based on a compiler that synthesizes test data generators for functional programming languages (e.g. ML, Haskell) from specifications in Isabelle. In contrast to naive type-based test data generators, the smart generators take the preconditions into account and only generate tests that fulfill the preconditions. The smart generators are constructed by a compiler that reformulates the preconditions as logic programs and analyzes them with an enriched mode inference. From this inference, the compiler can construct the desired generators in the functional programming language. Applying these test data generators reduces the number of tests significantly and enables us to find errors in specifications where naive random and exhaustive testing fail.

[1]  Tobias Nipkow.  Verifying a Hotel Key Card System , 2006, ICTAC.

[2]  Tobias Nipkow,et al.  The Isabelle Framework , 2008, TPHOLs.

[3]  Koen Claessen,et al.  QuickCheck: a lightweight tool for random testing of Haskell programs , 2011, SIGP.

[4]  Sebastian Fischer,et al.  EasyCheck - Test Data for Free , 2008, FLOPS.

[5]  Lee Naish,et al.  Adding equations to NU-Prolog , 1991, PLILP.

[6]  Colin Runciman,et al.  Smallcheck and lazy smallcheck: automatic exhaustive testing for small values , 2008, Haskell '08.

[7]  Rachid Echahed,et al.  A needed narrowing strategy , 2000, JACM.

[8]  Andy King,et al.  Mode Analysis Domains for Typed Logic Programs , 1999, LOPSTR.

[9]  Pierre Deransart,et al.  Programming Languages Implementation and Logic Programming , 1989, Lecture Notes in Computer Science.

[10]  Krzysztof R. Apt,et al.  Logic Programming , 1990, Handbook of Theoretical Computer Science, Volume B: Formal Models and Sematics.

[11]  Michael Hanus,et al.  Multi-paradigm Declarative Languages , 2007, ICLP.

[12]  Tobias Nipkow,et al.  Code Generation via Higher-Order Rewrite Systems , 2010, FLOPS.

[13]  Tobias Nipkow,et al.  Nitpick: A Counterexample Generator for Higher-Order Logic Based on a Relational Model Finder , 2010, ITP.

[14]  Herbert Kuchen,et al.  Systematic generation of glass-box test cases for functional logic programs , 2007, PPDP '07.

[15]  Tjark Weber,et al.  Bounded Model Generation for Isabelle/HOL , 2005, D/PDPAR@IJCAR.

[16]  Arnaud Gotlieb,et al.  Constraint Reasoning in FocalTest , 2010, ICSOFT.

[17]  David Aspinall,et al.  Formalising Java's Data Race Free Guarantee , 2007, TPHOLs.

[18]  Céline Rouveirol.  Flattening and Saturation: Two Representation Changes for Generalization , 2004, Machine Learning.

[19]  Tobias Nipkow,et al.  Random testing in Isabelle/HOL , 2004, Proceedings of the Second International Conference on Software Engineering and Formal Methods, 2004. SEFM 2004..

[20]  Tom Schrijvers,et al.  Functional and Logic Programming , 2012, Lecture Notes in Computer Science.

[21]  Lukas Bulwahn,et al.  Smart test data generators via logic programming , 2011, ICLP.

[22]  Peter J. Stuckey,et al.  Constraint-based mode analysis of mercury , 2002, PPDP '02.

[23]  Manuel V. Hermenegildo,et al.  Energy Consumption Analysis of Programs Based on XMOS ISA-Level Models , 2013, LOPSTR.

[24]  Alan Bundy,et al.  Constructing Induction Rules for Deductive Synthesis Proofs , 2006, CLASE.

[25]  Daniel Jackson,et al.  Software Abstractions - Logic, Language, and Analysis , 2006 .

[26]  Fredrik Lindblad.  Property Directed Generation of First-Order Test Data , 2007, Trends in Functional Programming.

[27]  Kamel Barkaoui,et al.  Theoretical Aspects of Computing - ICTAC 2006, Third International Colloquium, Tunis, Tunisia, November 20-24, 2006, Proceedings , 2006, ICTAC.

[28]  Stefan Berghofer,et al.  Turning Inductive into Equational Specifications , 2009, TPHOLs.