Cheap and Small Counterexamples

Minimal counterexamples are desirable, but expensive to compute. We propose four algorithms for computing small counterexamples that approximate the shortest case. Three of these use a new algorithm for automata-theoretic linear-time model checking, based on an early algorithm by Dijkstra for detecting strongly connected components. All four of the approximation algorithms rely on transitions shuffling; we show that the default transition ordering can behave badly. The algorithms are compared to a standard shortest counterexample algorithm on a large public data set.

[1]  Patrice Godefroid,et al.  Partial-Order Methods for the Verification of Concurrent Systems , 1996, Lecture Notes in Computer Science.

[2]  Chao Wang,et al.  Whodunit? Causal Analysis for Counterexamples , 2006, ATVA.

[3]  Gordon Fraser,et al.  Relating counterexamples to test cases in CTL model checking specifications , 2007, A-MOST '07.

[4]  Pierre Wolper,et al.  Memory-efficient algorithms for the verification of temporal properties , 1990, Formal Methods Syst. Des..

[5]  Edsger W. Dijkstra,et al.  Selected Writings on Computing: A personal Perspective , 1982, Texts and Monographs in Computer Science.

[6]  Gordon Fraser,et al.  Testing with model checkers: a survey , 2009 .

[7]  Stephan Merz,et al.  Model Checking , 2000 .

[8]  Robert K. Brayton,et al.  Partial-Order Reduction in Symbolic State-Space Exploration , 2001, Formal Methods Syst. Des..

[9]  Paul Ammann,et al.  Using model checking to generate tests from specifications , 1998, Proceedings Second International Conference on Formal Engineering Methods (Cat.No.98EX241).

[10]  Frank Reffel,et al.  Error Detection with Directed Symbolic Model Checking , 1999, World Congress on Formal Methods.

[11]  Edsger W. Dijkstra,et al.  A Discipline of Programming , 1976 .

[12]  Heikki Tauriainen,et al.  Nested emptiness search for generalized Buchi automata , 2004, Proceedings. Fourth International Conference on Application of Concurrency to System Design, 2004. ACSD 2004..

[13]  Mihalis Yannakakis,et al.  On nested depth first search , 1996, The Spin Verification System.

[14]  Stefan Edelkamp,et al.  Partial Order Reduction in Directed Model Checking , 2002, SPIN.

[15]  Rob Gerth,et al.  Model checking if your life depends on it: a view from Intel's trenches , 2001, SPIN '01.

[16]  Jaco Geldenhuys,et al.  More efficient on-the-fly LTL verification with Tarjan's algorithm , 2005, Theor. Comput. Sci..

[17]  Antti Valmari,et al.  A stubborn attack on state explosion , 1990, Formal Methods Syst. Des..

[18]  Jaco Geldenhuys,et al.  Larger Automata and Less Work for LTL Model Checking , 2006, SPIN.

[19]  Thomas A. Henzinger,et al.  Generating tests from counterexamples , 2004, Proceedings. 26th International Conference on Software Engineering.

[20]  Alex Groce,et al.  Making the Most of BMC Counterexamples , 2005, BMC@CAV.

[21]  Alex Groce,et al.  What Went Wrong: Explaining Counterexamples , 2003, SPIN.

[22]  Mayur Naik,et al.  From symptom to cause: localizing errors in counterexample traces , 2003, POPL '03.

[23]  Armin Biere,et al.  Shortest Counterexamples for Symbolic Model Checking of LTL with Past , 2005 .

[24]  Jean-Michel Couvreur,et al.  On-the-Fly Verification of Linear Temporal Logic , 1999, World Congress on Formal Methods.

[25]  Harold N. Gabow,et al.  Path-based depth-first search for strong and biconnected components , 2000, Inf. Process. Lett..

[26]  Sriram K. Rajamani,et al.  The SLAM project: debugging system software via static analysis , 2002, POPL '02.

[27]  Stefan Edelkamp,et al.  Directed explicit model checking with HSF-SPIN , 2001, SPIN '01.

[28]  Luciano Baresi,et al.  An Introduction to Software Testing , 2006, FoVMT.

[29]  Daniel Kroening,et al.  Model checking concurrent linux device drivers , 2007, ASE.

[30]  Kurt Mehlhorn,et al.  Algorithms for dense graphs and networks on the random access computer , 2005, Algorithmica.

[31]  Robert E. Tarjan,et al.  Depth-First Search and Linear Graph Algorithms , 1972, SIAM J. Comput..

[32]  Doron A. Peled,et al.  All from One, One for All: on Model Checking Using Representatives , 1993, CAV.

[33]  Henri Hansen,et al.  Minimal Counterexamples in O(n log n) Memory and O(n^2) Time , 2006, Sixth International Conference on Application of Concurrency to System Design (ACSD'06).

[34]  Bernd Becker,et al.  Computation of minimal counterexamples by using black box techniques and symbolic methods , 2007, ICCAD 2007.

[35]  Paul Gastin,et al.  Minimization of Counterexamples in SPIN , 2004, SPIN.

[36]  Gordon Fraser,et al.  Nondeterministic Testing with Linear Model-Checker Counterexamples , 2007 .

[37]  Paul Gastin,et al.  Minimal Counterexample Generation for SPIN , 2007, SPIN.

[38]  Paul Gastin,et al.  Fast LTL to Büchi Automata Translation , 2001, CAV.

[39]  Antonia Bertolino,et al.  Software Testing Research: Achievements, Challenges, Dreams , 2007, Future of Software Engineering (FOSE '07).

[40]  Denis Poitrenaud,et al.  On-the-Fly Emptiness Checks for Generalized Büchi Automata , 2005, SPIN.

[41]  Angelo Gargantini,et al.  Using model checking to generate tests from requirements specifications , 1999, ESEC/FSE-7.

[42]  Edsger W. Dijkstra.  Finding the Maximum Strong Components in a Directed Graph , 1982 .

[43]  Ying Qin,et al.  A fast counterexample minimization approach with refutation analysis and incremental SAT , 2005, ASP-DAC.

[44]  John Penix,et al.  Verification of time partitioning in the DEOS scheduler kernel , 2000, Proceedings of the 2000 International Conference on Software Engineering. ICSE 2000 the New Millennium.

[45]  P. Gastin,et al.  Minimal counter-example generation for SPIN , 2007 .

[46]  Dragan Bosnacki,et al.  Improving Spin's Partial-Order Reduction for Breadth-First Search , 2005, SPIN.

[47]  Edmund M. Clarke,et al.  Efficient generation of counterexamples and witnesses in symbolic model checking , 1995, DAC '95.

[48]  Javier Esparza,et al.  A Note on On-the-Fly Verification Algorithms , 2005, TACAS.