Higher order symbolic execution for contract verification and refutation

We present a new approach to automated reasoning about higher-order programs by endowing symbolic execution with a notion of higher-order, symbolic values. Our approach is sound and relatively complete with respect to a first-order solver for base type values. Therefore, it can form the basis of automated verification and bug-finding tools for higher-order programs. To validate our approach, we use it to develop and evaluate a system for verifying and refuting behavioral software contracts of components in a functional language, which we call soft contract verification. In doing so, we discover a mutually beneficial relation between behavioral contracts and higher-order symbolic execution. Our system uses higher-order symbolic execution, leveraging contracts as a source of symbolic values including unknown behavioral values, and employs an updatable heap of contract invariants to reason about flow-sensitive facts. Whenever a contract is refuted, it reports a concrete counterexample reproducing the error, which may involve solving for an unknown function. The approach is able to analyze first-class contracts, recursive data structures, unknown functions, and control-flow-sensitive refinements of values, which are all idiomatic in dynamic languages. It makes effective use of an off-the-shelf solver to decide problems without heavy encodings. The approach is competitive with a wide range of existing tools---including type systems, flow analyzers, and model checkers---on their own benchmarks. We have built a tool which analyzes programs written in Racket, and report on its effectiveness in verifying and refuting contracts.

[1]  Matthias Felleisen,et al.  Componential set-based analysis , 1997, TOPL.

[2]  Matthew Might,et al.  Systematic abstraction of abstract machines , 2011, Journal of Functional Programming.

[3]  Thomas H. Austin,et al.  Virtual values for language extension , 2011, OOPSLA '11.

[4]  Naoki Kobayashi.  Model-checking higher-order functions , 2009, PPDP '09.

[5]  Matthew Might,et al.  Abstracting abstract machines , 2010, ICFP '10.

[6]  Cormac Flanagan,et al.  Temporal higher-order contracts , 2011, ICFP.

[7]  Robert Cartwright,et al.  A practical soft type system for scheme , 1997, TOPL.

[8]  Simon L. Peyton Jones,et al.  HALO: haskell to logic through denotational semantics , 2013, POPL.

[9]  Sam Tobin-Hochstadt,et al.  Logical types for untyped languages , 2010, ICFP '10.

[10]  Cormac Flanagan,et al.  Hybrid type checking , 2006, POPL '06.

[11]  Ranjit Jhala,et al.  Refinement types for Haskell , 2014, ICFP.

[12]  Matthias Felleisen,et al.  Catching bugs in the web of program invariants , 1996, PLDI '96.

[13]  Robert Cartwright,et al.  Soft typing , 2004, SIGP.

[14]  Robert Bruce Findler,et al.  Random testing for higher-order, stateful programs , 2010, OOPSLA.

[15]  Olin Shivers,et al.  Control flow analysis in scheme , 1988, PLDI '88.

[16]  Frank Pfenning,et al.  Refinement types for ML , 1991, PLDI '91.

[17]  Koushik Sen,et al.  DART: directed automated random testing , 2005, PLDI '05.

[18]  Dawson R. Engler,et al.  EXE: automatically generating inputs of death , 2006, CCS '06.

[19]  Matthias Felleisen,et al.  Correct blame for contracts: no more scapegoating , 2011, POPL '11.

[20]  Sam Tobin-Hochstadt,et al.  Typing the Numeric Tower , 2012, PADL.

[21]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[22]  Matthias Felleisen,et al.  Contracts for higher-order functions , 2002, ICFP '02.

[23]  Tachio Terauchi.  Dependent types from counterexamples , 2010, POPL '10.

[24]  Naoki Kobayashi.  Types and higher-order recursion schemes for verification of higher-order programs , 2009, POPL '09.

[25]  Sam Tobin-Hochstadt,et al.  Higher-order symbolic execution via contracts , 2012, OOPSLA '12.

[26]  Manuel Fähndrich,et al.  Static Contract Checking with Abstract Interpretation , 2010, FoVeOOS.

[27]  Koen Claessen,et al.  QuickCheck: a lightweight tool for random testing of Haskell programs , 2000, ICFP.

[28]  Suresh Jagannathan,et al.  Compositional and Lightweight Dependent Type Inference for ML , 2013, VMCAI.

[29]  Ranjit Jhala,et al.  Type Targeted Testing , 2014, ESOP.

[30]  Naoki Kobayashi,et al.  Higher-order multi-parameter tree transducers and recursion schemes for program verification , 2010, POPL '10.

[31]  Ranjit Jhala,et al.  Abstract Refinement Types , 2013, ESOP.

[32]  Fritz Henglein,et al.  Dynamic Typing: Syntax and Proof Theory , 1994, Sci. Comput. Program..

[33]  Naoki Kobayashi,et al.  Predicate abstraction and CEGAR for higher-order model checking , 2011, PLDI '11.

[34]  Alexander Aiken,et al.  Scalable error detection using boolean satisfiability , 2005, POPL '05.

[35]  Sam Tobin-Hochstadt,et al.  Chaperones and impersonators: run-time support for reasonable interposition , 2012, OOPSLA '12.

[36]  Antonio Vallecillo,et al.  Objects, Models, Components, Patterns , 2011, Lecture Notes in Computer Science.

[37]  Bertrand Meyer,et al.  Eiffel: The Language , 1991 .

[38]  David Van Horn,et al.  Abstracting abstract control , 2013, 1305.3163.

[39]  Alexander Aiken,et al.  Soft typing with conditional types , 1994, POPL '94.

[40]  Sam Tobin-Hochstadt,et al.  Soft contract verification , 2014, ICFP.

[41]  David Van Horn,et al.  Relatively complete counterexamples for higher-order programs , 2015, PLDI.

[42]  Atsushi Igarashi,et al.  Model-Checking Higher-Order Programs with Recursive Types , 2013, ESOP.

[43]  Ravi Chugh,et al.  Nested refinements: a logic for duck typing , 2012, POPL '12.

[44]  Matthias Felleisen,et al.  Program verification through soft typing , 1996, CSUR.

[45]  Naoki Kobayashi,et al.  Untyped Recursion Schemes and Infinite Intersection Types , 2010, FoSSaCS.

[46]  Ranjit Jhala,et al.  Dsolve: Safety Verification via Liquid Types , 2010, CAV.

[47]  C.-H. Luke Ong,et al.  On Model-Checking Trees Generated by Higher-Order Recursion Schemes , 2006, 21st Annual IEEE Symposium on Logic in Computer Science (LICS'06).

[48]  Atsushi Igarashi,et al.  Programming Languages and Systems , 2016, Lecture Notes in Computer Science.

[49]  Reinhold Plösch.  Design by Contract for Python , 1997, APSEC.

[50]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[51]  Alexander Aiken,et al.  Flow-sensitive type qualifiers , 2002, PLDI '02.

[52]  C.-H. Luke Ong,et al.  A Type System Equivalent to the Modal Mu-Calculus Model Checking of Higher-Order Recursion Schemes , 2009, 2009 24th Annual IEEE Symposium on Logic In Computer Science.

[53]  Sam Tobin-Hochstadt,et al.  Languages as libraries , 2011, PLDI '11.

[54]  Dana N. Xu.  Hybrid contract checking via symbolic simplification , 2012, PEPM '12.

[55]  Patrick Maxim Rondon,et al.  Liquid types , 2008, PLDI '08.

[56]  Na Xu.  Static contract checking for Haskell , 2009, POPL '09.

[57]  Ravi Chugh,et al.  Dependent types for JavaScript , 2012, OOPSLA '12.

[58]  Peter Thiemann,et al.  Contract-Driven Testing of JavaScript Code , 2010, TOOLS.

[59]  Matthias Felleisen,et al.  Modular set-based analysis from contracts , 2006, POPL '06.

[60]  Junfeng Yang,et al.  Using model checking to find serious file system errors , 2004, TOCS.