Executable Counterexamples in Software Model Checking

Counterexamples—execution traces of the system that illustrate how an error state can be reached from the initial state—are essential for understanding verification failures. They are one of the most salient features of Model Checkers, which distinguish them from Abstract Interpretation and other Static Analysis techniques by providing a user with information on how to debug their system and/or the specification. While in Hardware and Protocol verification, the counterexamples can be replayed in the system, in Software Model Checking (SMC) counterexamples take the form of a textual or semi-structured report. This is problematic since it complicates the debugging process by preventing developers from using existing processes and tools such as debuggers, fault localization, and fault minimization.

[1]  Thomas A. Henzinger,et al.  Generating tests from counterexamples , 2004, Proceedings. 26th International Conference on Software Engineering.

[2]  Yannis Smaragdakis,et al.  JCrasher: an automatic robustness tester for Java , 2004, Softw. Pract. Exp..

[3]  Robert W. Bowdidge,et al.  Why don't software developers use static analysis tools to find bugs? , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[4]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[5]  Patrice Godefroid,et al.  Micro execution , 2014, ICSE.

[6]  Peter Müller,et al.  An Experimental Evaluation of Deliberate Unsoundness in a Static Program Analyzer , 2015, VMCAI.

[7]  Jochen Hoenicke,et al.  Ultimate Automizer with SMTInterpol - (Competition Contribution) , 2013, TACAS.

[8]  C. Csallner,et al.  Check 'n' crash: combining static checking and testing , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[9]  W. Marsden.  I and J , 2012 .

[10]  Sarfraz Khurshid,et al.  Test input generation with java PathFinder , 2004, ISSTA '04.

[11]  Peter Müller,et al.  Guiding Dynamic Symbolic Execution toward Unverified Program Executions , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE).

[12]  Mark Lillibridge,et al.  Extended static checking for Java , 2002, PLDI '02.

[13]  Nikolai Tillmann,et al.  Pex-White Box Test Generation for .NET , 2008, TAP.

[14]  Dirk Beyer,et al.  Tests from Witnesses - Execution-Based Validation of Verification Results , 2018, TAP@STAF.

[15]  Jorge A. Navas,et al.  A Context-Sensitive Memory Model for Verification of C/C++ Programs , 2017, SAS.

[16]  Jorge A. Navas,et al.  The SeaHorn Verification Framework , 2015, CAV.

[17]  Thomas A. Henzinger,et al.  SYNERGY: a new algorithm for property checking , 2006, SIGSOFT '06/FSE-14.

[18]  Lucas C. Cordeiro,et al.  Understanding Programming Bugs in ANSI-C Software Using Bounded Model Checking Counter-Examples , 2012, IFM.

[19]  Dirk Beyer,et al.  CPAchecker: A Tool for Configurable Software Verification , 2009, CAV.

[20]  Michael Hicks,et al.  Directed Symbolic Execution , 2011, SAS.

[21]  Patrice Godefroid,et al.  VeriSoft: A Tool for the Automatic Analysis of Concurrent Reactive Software , 1997, CAV.

[22]  Bernd Fischer,et al.  SMT-Based Bounded Model Checking for Embedded ANSI-C Software , 2012, IEEE Transactions on Software Engineering.

[23]  Dirk Beyer,et al.  Software Verification with Validation of Results - (Report on SV-COMP 2017) , 2017, TACAS.

[24]  Patrice Godefroid,et al.  Automated Whitebox Fuzz Testing , 2008, NDSS.

[25]  Peter Müller,et al.  Using Debuggers to Understand Failed Verification Attempts , 2011 .

[26]  Sriram K. Rajamani,et al.  Compositional may-must program analysis: unleashing the power of alternation , 2010, POPL '10.

[27]  Koushik Sen,et al.  CUTE and jCUTE: Concolic Unit Testing and Explicit Path Model-Checking Tools , 2006, CAV.

[28]  Dirk Beyer,et al.  Software Verification: Testing vs. Model Checking - A Comparative Evaluation of the State of the Art , 2017, Haifa Verification Conference.

[29]  Patrice Godefroid,et al.  Proving Memory Safety of the ANI Windows Image Parser Using Compositional Exhaustive Testing , 2015, VMCAI.

[30]  Robert J. Simmons,et al.  Proofs from Tests , 2008, IEEE Transactions on Software Engineering.

[31]  Dawson R. Engler,et al.  EXE: automatically generating inputs of death , 2006, CCS '06.

[32]  Koushik Sen.  DART: Directed Automated Random Testing , 2009, Haifa Verification Conference.