SpaceSearch: a library for building and verifying solver-aided tools

Many verification tools build on automated solvers. These tools reduce problems in a specific application domain (e.g., compiler optimization validation) to queries that can be discharged with a highly optimized solver. But the correctness of the reductions themselves is rarely verified in practice, limiting the confidence that the solver's output establishes the desired domain-level property. This paper presents SpaceSearch, a new library for developing solver-aided tools within a proof assistant. A user builds their solver-aided tool in Coq against the SpaceSearch interface, and the user then verifies that the results provided by the interface are sufficient to establish the tool's desired high-level properties. Once verified, the tool can be extracted to an implementation in a solver-aided language (e.g., Rosette), where SpaceSearch provides an efficient instantiation of the SpaceSearch interface with calls to an underlying SMT solver. This combines the strong correctness guarantees of developing a tool in a proof assistant with the high performance of modern SMT solvers. This paper also introduces new optimizations for such verified solver-aided tools, including parallelization and incrementalization. We evaluate SpaceSearch by building and verifying two solver-aided tools. The first, SaltShaker, checks that RockSalt's x86 semantics for a given instruction agrees with STOKE's x86 semantics. SaltShaker identified 7 bugs in RockSalt and 1 bug in STOKE. After these systems were patched by their developers, SaltShaker verified the semantics' agreement on 15,255 instruction instantiations in under 2h. The second tool, BGProof, is a verified version of an existing Border Gateway Protocol (BGP) router configuration checker. Like the existing checker, BGProof scales to checking industrial configurations spanning over 240 KLOC, identifying 19 configuration inconsistencies with no false positives. However, the correctness of BGProof has been formally proven, and we found 2 bugs in the unverified implementation. These results demonstrate that SpaceSearch is a practical approach to developing efficient, verified solver-aided tools.

[1]  Sorin Lerner,et al.  Proving optimizations correct using parameterized program equivalence , 2009, PLDI '09.

[2]  Emina Torlak,et al.  A lightweight symbolic virtual machine for solver-aided host languages , 2014, PLDI.

[3]  Kevin Clancy,et al.  versat: A Verified Modern SAT Solver , 2012, VMCAI.

[4]  Bennet S. Yee,et al.  Native Client: A Sandbox for Portable, Untrusted x86 Native Code , 2009, 2009 30th IEEE Symposium on Security and Privacy.

[5]  Alexander Aiken,et al.  Stochastic superoptimization , 2012, ASPLOS '13.

[6]  Sorin Lerner,et al.  Bringing extensibility to verified compilers , 2010, PLDI '10.

[7]  Emina Torlak,et al.  Growing solver-aided languages with rosette , 2013, Onward!.

[8]  Nicolas Christin,et al.  Push-Button Verification of File Systems via Crash Refinement , 2016, USENIX Annual Technical Conference.

[9]  Shuvendu K. Lahiri,et al.  Static and Precise Detection of Concurrency Errors in Systems Code Using SMT Solvers , 2009, CAV.

[10]  Ramesh Govindan,et al.  A General Approach to Network Configuration Analysis , 2015, NSDI.

[11]  Joseph Tassarotti,et al.  RockSalt: better, faster, stronger SFI for the x86 , 2012, PLDI.

[12]  Stephen N. Zilles,et al.  Programming with abstract data types , 1974, SIGPLAN Symposium on Very High Level Languages.

[13]  Guodong Li,et al.  Scalable SMT-based verification of GPU kernel functions , 2010, FSE '10.

[14]  Michael D. Ernst,et al.  Scalable verification of border gateway protocol configurations with an SMT solver , 2016, OOPSLA.

[15]  Michael Schapira,et al.  VeriCon: towards verifying controller programs in software-defined networks , 2014, PLDI.

[16]  Koen Claessen,et al.  QuickCheck: a lightweight tool for random testing of Haskell programs , 2000, ICFP.

[17]  Armando Solar-Lezama,et al.  Adaptive Concretization for Parallel Program Synthesis , 2015, CAV.

[18]  Emina Torlak,et al.  Automated reasoning for web page layout , 2016, OOPSLA.

[19]  K. Rustan M. Leino,et al.  Dafny: An Automatic Program Verifier for Functional Correctness , 2010, LPAR.

[20]  Filip Maric,et al.  Formal verification of a modern SAT solver by shallow embedding into Isabelle/HOL , 2010, Theor. Comput. Sci..

[21]  Rishabh Singh,et al.  Modular Synthesis of Sketches Using Models , 2014, VMCAI.

[22]  John Regehr,et al.  Provably correct peephole optimizations with alive , 2015, PLDI.

[23]  Sylvain Conchon,et al.  Improving Coq Propositional Reasoning Using a Lazy CNF Conversion Scheme , 2009, FroCoS.

[24]  Chao Wang,et al.  Generating Data Race Witnesses by an SMT-Based Analysis , 2011, NASA Formal Methods.

[25]  K. Rustan M. Leino,et al.  This is Boogie 2 , 2016 .

[26]  Andrew W. Appel.  Verified Software Toolchain - (Invited Talk) , 2011, ESOP.

[27]  Rastislav Bodík,et al.  Chlorophyll : Synthesis-Aided Compiler for Low-Power Spatial Architectures by Phitchaya Mangpo Phothilimthana , 2015 .

[28]  Benjamin Grégoire,et al.  A Modular Integration of SAT/SMT Solvers to Coq through Proof Witnesses , 2011, CPP.

[29]  Peter A. Dinda,et al.  Distributed Places , 2013, Trends in Functional Programming.

[30]  Alexander Aiken,et al.  Stratified synthesis: automatically learning the x86-64 instruction set , 2016, PLDI.

[31]  Nirav Dave,et al.  Smten with satisfiability-based search , 2014, OOPSLA 2014.

[32]  Emina Torlak,et al.  MemSAT: checking axiomatic specifications of memory models , 2010, PLDI '10.