Proofs from Tests

We present an algorithm DASH to check if a program P satisfies a safety property φ. The unique feature of this algorithm is that it uses only test generation operations, and it refines and maintains a sound program abstraction as a consequence of failed test generation operations. Thus, each iteration of the algorithm is inexpensive, and can be implemented without any global may-alias information. In particular, we introduce a new refinement operator WPα that uses only the alias information obtained by symbolically executing a test to refine abstractions in a sound manner. We present a full exposition of the DASH algorithm and its theoretical properties. We have implemented DASH in a tool called YOGI that plugs into Microsoft's Static Driver Verifier framework. We have used this framework to run YOGI on 69 Windows Vista drivers with 85 properties and find that YOGI scales much better than SLAM, the current engine driving Microsoft's Static Driver Verifier.

[1]  Sriram K. Rajamani,et al.  Compositional may-must program analysis: unleashing the power of alternation , 2010, POPL '10.

[2]  Sriram K. Rajamani,et al.  SLIC: A Specification Language for Interface Checking (of C) , 2002 .

[3]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[4]  Alex Groce,et al.  Counterexample Guided Abstraction Refinement Via Program Execution , 2004, ICFEM.

[5]  Kedar S. Namjoshi,et al.  Syntactic Program Transformations for Automatic Abstraction , 2000, CAV.

[6]  Bor-Yuh Evan Chang,et al.  Boogie: A Modular Reusable Verifier for Object-Oriented Programs , 2005, FMCO.

[7]  Dawson R. Engler,et al.  EXE: automatically generating inputs of death , 2006, CCS '06.

[8]  Thomas A. Henzinger,et al.  SYNERGY: a new algorithm for property checking , 2006, SIGSOFT '06/FSE-14.

[9]  Dinghao Wu,et al.  KISS: keep it simple and sequential , 2004, PLDI '04.

[10]  Koushik Sen,et al.  DART: directed automated random testing , 2005, PLDI '05.

[11]  Thomas A. Henzinger,et al.  Lazy abstraction , 2002, POPL '02.

[12]  Andreas Podelski,et al.  Termination proofs for systems code , 2006, PLDI '06.

[13]  Patrice Godefroid,et al.  Compositional dynamic test generation , 2007, POPL '07.

[14]  Edsger W. Dijkstra,et al.  The humble programmer , 1972, CACM.

[15]  Thomas Ball,et al.  Testing, abstraction, theorem proving: better together! , 2006, ISSTA '06.

[16]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[17]  Robert J. Simmons,et al.  Proofs from Tests , 2008, IEEE Transactions on Software Engineering.

[18]  Todd Millstein,et al.  Automatic predicate abstraction of C programs , 2001, PLDI '01.

[19]  George C. Necula,et al.  CIL: Intermediate Language and Tools for Analysis and Transformation of C Programs , 2002, CC.

[20]  Doron A. Peled,et al.  Model checking, testing and verification working together , 2005, Formal Aspects of Computing.

[21]  Nils Klarlund,et al.  Software Model Checking: Searching for Computations in the Abstract or the Concrete , 2005, IFM.

[22]  Rupak Majumdar,et al.  LATEST : Lazy Dynamic Test Input Generation , 2007 .

[23]  Sriram K. Rajamani,et al.  Automatically validating temporal safety properties of interfaces , 2001, SPIN '01.

[24]  Edsger W. Dijkstra,et al.  A Discipline of Programming , 1976 .

[25]  Edmund M. Clarke,et al.  Counterexample-Guided Abstraction Refinement , 2000, CAV.

[26]  K. Rustan M. Leino,et al.  Extended static checking , 1998, PROCOMET.