Deductive Software Verification: From Pen-and-Paper Proofs to Industrial Tools

Deductive software verification aims at formally verifying that all possible behaviors of a given program satisfy formally defined, possibly complex properties, where the verification process is based on logical inference. We follow the trajectory of the field from its inception in the late 1960s via its current state to its promises for the future, from pen-and-paper proofs for programs written in small, idealized languages to highly automated proofs of complex library or system code written in mainstream languages. We take stock of the state-of-art and give a list of the most important challenges for the further development of the field of deductive software verification.

[1]  Reiner Hähnle,et al.  KeY-ABS: A Deductive Verification Tool for the Concurrent Modelling Language ABS , 2015, CADE.

[2]  Geoff Sutcliffe The TPTP Problem Library and Associated Infrastructure , 2009, Journal of Automated Reasoning.

[3]  Laura Kovács,et al.  Symbolic Computation and Automated Reasoning for Program Analysis , 2016, IFM.

[4]  David R. Cok,et al.  OpenJML: Software verification for Java 7 using JML, OpenJDK, and Eclipse , 2014, F-IDE.

[5]  François Bobot,et al.  Why3: Shepherd Your Herd of Provers , 2011 .

[6]  Cesare Tinelli,et al.  An Automatable Formal Semantics for IEEE-754 Floating-Point Arithmetic , 2015, 2015 IEEE 22nd Symposium on Computer Arithmetic.

[7]  K. Rustan M. Leino,et al.  The Spec# Programming System: An Overview , 2004, CASSIS.

[8]  Stephen Brookes A semantics for concurrent separation logic , 2007, Theor. Comput. Sci..

[9]  Bart Jacobs,et al.  A Formalisation of Java's Exception Mechanism , 2001, ESOP.

[10]  Maritta Heisel,et al.  An Interactive Verification System Based on Dynamic Logic , 1986, CADE.

[11]  Viktor Vafeiadis,et al.  GPS: navigating weak memory with ghosts, protocols, and separation , 2014, OOPSLA.

[12]  Jean-Baptiste Jeannin,et al.  Formal verification of ACAS X, an industrial airborne collision avoidance system , 2015, 2015 International Conference on Embedded Software (EMSOFT).

[13]  Wojciech Mostowski Dynamic Frames Based Verification Method for Concurrent Java Programs , 2015, VSTTE.

[14]  Marieke Huisman,et al.  Permission-Based Separation Logic for Multithreaded Java Programs , 2014, Log. Methods Comput. Sci..

[15]  Peter W. O'Hearn,et al.  Concurrent separation logic , 2016, SIGL.

[16]  Peter Müller,et al.  An Experimental Evaluation of Deliberate Unsoundness in a Static Program Analyzer , 2015, VMCAI.

[17]  Bart Jacobs,et al.  Java Program Verification via a Hoare Logic with Abrupt Termination , 2000, FASE.

[18]  Dominique Méry,et al.  Proceedings 1st Workshop on Formal Integrated Development Environment: Preface , 2014 .

[19]  Stephen H. Edwards,et al.  Model variables: cleanly supporting abstraction in design by contract , 2005, Softw. Pract. Exp..

[20]  Francesco Logozzo Practical Verification for the Working Programmer with CodeContracts and Abstract Interpretation - (Invited Talk) , 2011, VMCAI.

[21]  Yannick Moy,et al.  SPARK 2014 and GNATprove , 2014, International Journal on Software Tools for Technology Transfer.

[22]  Viktor Vafeiadis Automatically Proving Linearizability , 2010, CAV.

[23]  Bertrand Meyer,et al.  Object-Oriented Software Construction, 2nd Edition , 1997 .

[24]  Gordon D. Plotkin,et al.  A structural approach to operational semantics , 2004, J. Log. Algebraic Methods Program..

[25]  Peter W. O'Hearn,et al.  Resources, concurrency, and local reasoning , 2007 .

[26]  Carlo A. Furia,et al.  Why Just Boogie? - Translating Between Intermediate Verification Languages , 2016, IFM.

[27]  C. A. R. Hoare,et al.  Proof of correctness of data representations , 1972, Acta Informatica.

[28]  Jeannette M. Wing,et al.  A behavioral notion of subtyping , 1994, TOPL.

[29]  Dirk Beyer,et al.  Symbolic Execution with CEGAR , 2016, ISoLA.

[30]  Bart Jacobs,et al.  One Step Towards Automatic Inference of Formal Specifications Using Automated VeriFast , 2016, FMICS-AVoCS.

[31]  Peter H. Schmitt,et al.  Integrierter Deduktiver Software-Entwurf , 1998, Künstliche Intell..

[32]  Viktor Vafeiadis,et al.  A Marriage of Rely/Guarantee and Separation Logic , 2007, CONCUR.

[33]  Jean-Raymond Abrial,et al.  The B-book - assigning programs to meanings , 1996 .

[34]  Gidon Ernst,et al.  KIV: overview and VerifyThis competition , 2014, International Journal on Software Tools for Technology Transfer.

[35]  Frank S. de Boer,et al.  OpenJDK's Java.utils.Collection.sort() Is Broken: The Good, the Bad and the Worst Case , 2015, CAV.

[36]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[37]  Michael Leuschel,et al.  Under consideration for publication in Formal Aspects of Computing Automated Property Verification for Large Scale B Models with ProB , 2010 .

[38]  K. Rustan M. Leino,et al.  An Extended Static Checker for Modular-3 , 1998, CC.

[39]  Reiner Hähnle,et al.  Symbolic Fault Injection , 2007, VERIFY.

[40]  Cristiano Calcagno,et al.  Infer: An Automatic Program Verifier for Memory Safety of C Programs , 2011, NASA Formal Methods.

[41]  Gordon J. Pace,et al.  StaRVOOrS: A Tool for Combined Static and Runtime Verification of Java , 2015, RV.

[42]  Maritta Heisel,et al.  Program Verification by Symbolic Execution and Induction , 1987, GWAI.

[43]  Mattias Ulbrich Dynamic Logic for an Intermediate Language: Verification, Interaction and Refinement , 2013 .

[44]  Bertrand Meyer,et al.  Applying 'design by contract' , 1992, Computer.

[45]  Reiner Hähnle,et al.  Symbolic Execution Debugger (SED) , 2014, RV.

[46]  Peter Müller,et al.  Formal Translation of Bytecode into BoogiePL , 2007, Electron. Notes Theor. Comput. Sci..

[47]  Reiner Hähnle,et al.  Proof Repositories for Compositional Verification of Evolving Software Systems - Managing Change When Proving Software Correct , 2016, LNCS Trans. Found. Mastering Chang..

[48]  Marieke Huisman,et al.  Specification and verification of GPGPU programs , 2013, Sci. Comput. Program..

[49]  Nadia Polikarpova,et al.  A Fully Verified Container Library , 2015, FM.

[50]  Marieke Huisman,et al.  VerCors: A Layered Approach to Practical Verification of Concurrent Software , 2016, 2016 24th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP).

[51]  Yannick Moy,et al.  Static versus Dynamic Verification in Why3, Frama-C and SPARK 2014 , 2016, ISoLA.

[52]  Bernhard Beckert,et al.  Deductive Software Verification – The KeY Book , 2016, Lecture Notes in Computer Science.

[53]  Albert L. Baker,et al.  Preliminary design of JML: a behavioral interface specification language for java , 2006, SOEN.

[54]  David C. Luckham,et al.  An Overview of Anna, a Specification Language for Ada , 1985, IEEE Software.

[55]  Mike Jackson,et al.  Interactive Proof Critics , 1999, Formal Aspects of Computing.

[56]  Marieke Huisman,et al.  Formal Specification with the Java Modeling Language , 2016, Deductive Software Verification.

[57]  Marieke Huisman,et al.  Resource Protection Using Atomics - Patterns and Verification , 2014, APLAS.

[58]  Frank Piessens,et al.  VeriFast: A Powerful, Sound, Predictable, Fast Verifier for C and Java , 2011, NASA Formal Methods.

[59]  Andrei Voronkov,et al.  Invariant Generation in Vampire , 2011, TACAS.

[60]  Elvira Albert,et al.  The ABS tool suite: modelling, executing and analysing distributed adaptable object-oriented systems , 2012, International Journal on Software Tools for Technology Transfer.

[61]  David R. Cok,et al.  ESC/Java2: Uniting ESC/Java and JML , 2004, CASSIS.

[62]  Cliff B. Jones,et al.  Tentative steps toward a development method for interfering programs , 1983, TOPL.

[63]  Bor-Yuh Evan Chang,et al.  Boogie: A Modular Reusable Verifier for Object-Oriented Programs , 2005, FMCO.

[64]  Reiner Hähnle,et al.  Uniform Modeling of Railway Operations , 2016, FTSCS.

[65]  Nikolai Kosmatov,et al.  Frama-C: A software analysis perspective , 2015, Formal Aspects of Computing.

[66]  Peter Müller,et al.  Comparing Verification Condition Generation with Symbolic Execution: An Experience Report , 2012, VSTTE.

[67]  Dirk Beyer,et al.  Correctness witnesses: exchanging verification results between verifiers , 2016, SIGSOFT FSE.

[68]  John Tang Boyland,et al.  Checking Interference with Fractional Permissions , 2003, SAS.

[69]  Bernhard Beckert,et al.  A Usability Evaluation of Interactive Theorem Provers Using Focus Groups , 2014, SEFM Workshops.

[70]  Vaughan R. Pratt,et al.  A Proof-Checker for Dynamic Logic , 1977, IJCAI.

[71]  Jan Smans,et al.  Verification of Concurrent Programs with Chalice , 2009, FOSAD.

[72]  Reiner Hähnle,et al.  The interactive verification debugger: Effective understanding of interactive proof attempts , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[73]  Claude Marché,et al.  The Why/Krakatoa/Caduceus Platform for Deductive Program Verification , 2007, CAV.

[74]  K. Rustan M. Leino,et al.  Fine-Grained Caching of Verification Results , 2015, CAV.

[75]  Bernhard Steffen,et al.  The physics of software tools: SWOT analysis and vision , 2017, International Journal on Software Tools for Technology Transfer.

[76]  Jürgen Giesl,et al.  Proving Termination of Programs Automatically with AProVE , 2014, IJCAR.

[77]  Peter W. O'Hearn,et al.  Permission accounting in separation logic , 2005, POPL '05.

[78]  Michael Norrish C formalised in HOL , 1998 .

[79]  Bernhard Beckert,et al.  Regression verification for Java using a secure information flow calculus , 2015, FTfJP@ECOOP.

[80]  Marieke Huisman,et al.  How Do Developers Use APIs? A Case Study in Concurrency , 2013, 2013 18th International Conference on Engineering of Complex Computer Systems.

[81]  Gidon Ernst,et al.  Development of a Verified Flash File System , 2014, ABZ.

[82]  C. A. R. Hoare,et al.  An axiomatic basis for computer programming , 1969, CACM.

[83]  Daniel Kroening,et al.  Making Software Verification Tools Really Work , 2011, ATVA.

[84]  Stephen McCamant,et al.  The Daikon system for dynamic detection of likely invariants , 2007, Sci. Comput. Program..

[85]  Stephen H. Edwards,et al.  Model variables: cleanly supporting abstraction in design by contract: Research Articles , 2005 .

[86]  Reiner Hähnle,et al.  An empirical evaluation of two user interfaces of an interactive program verifier , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[87]  Enric Rodríguez-Carbonell,et al.  Automatic generation of polynomial invariants of bounded degree using abstract interpretation , 2007, Sci. Comput. Program..

[88]  Dermot Cochran,et al.  The KOA Remote Voting System: A Summary of Work to Date , 2006, TGC.

[89]  Reiner Hähnle,et al.  ABS: A Core Language for Abstract Behavioral Specification , 2010, FMCO.

[90]  Marieke Huisman,et al.  Verification of Concurrent Systems with VerCors , 2014, SFM.

[91]  Nathan Fulton,et al.  KeYmaera X: An Axiomatic Tactical Theorem Prover for Hybrid Systems , 2015, CADE.

[92]  Bernd Finkbeiner,et al.  Verifying Temporal Properties of Reactive Systems: A STeP Tutorial , 2000, Formal Methods Syst. Des..

[93]  Clark W. Barrett,et al.  The SMT-LIB Standard Version 2.0 , 2010 .

[94]  Evan Ackerman,et al.  Hail, robo-taxi! [Top Tech 2017] , 2017, IEEE Spectrum.

[95]  Yde Venema,et al.  Dynamic Logic by David Harel, Dexter Kozen and Jerzy Tiuryn. The MIT Press, Cambridge, Massachusetts. Hardback: ISBN 0–262–08289–6, $50, xv + 459 pages , 2002, Theory and Practice of Logic Programming.

[96]  Rod M. Burstall,et al.  Program Proving as Hand Simulation with a Little Induction , 1974, IFIP Congress.

[97]  Bernhard Beckert,et al.  Lessons Learned From Microkernel Verification -- Specification is the New Bottleneck , 2012, SSV.

[98]  Viktor Vafeiadis,et al.  Relaxed separation logic: a program logic for C11 concurrency , 2013, OOPSLA.

[99]  Bernhard Beckert,et al.  Software Verification with Integrated Data Type Refinement for Integer Arithmetic , 2004, IFM.

[100]  Wojciech Mostowski,et al.  Fully Verified Java Card API Reference Implementation , 2007, VERIFY.

[101]  Marieke Huisman,et al.  The VerCors Tool for Verification of Concurrent Programs , 2014, FM.

[102]  Peter V. Homeier,et al.  A Mechanically Verified Verification Condition Generator , 1995, Comput. J..

[103]  David von Oheimb Hoare logic for Java in Isabelle/HOL , 2001, Concurr. Comput. Pract. Exp..

[104]  Susan Owicki,et al.  An axiomatic proof technique for parallel programs I , 1976, Acta Informatica.

[105]  Peter Deussen,et al.  The Verification System Tatzelwurm , 1995, KORSO Book.

[106]  Reiner Hähnle,et al.  Exploit Generation for Information Flow Leaks in Object-Oriented Programs , 2015, SEC.

[107]  Mark A. Hillebrand,et al.  VCC: A Practical System for Verifying Concurrent C , 2009, TPHOLs.

[108]  Robert W. Floyd,et al.  Assigning Meanings to Programs , 1993 .

[109]  K. Rustan M. Leino,et al.  Dafny: An Automatic Program Verifier for Functional Correctness , 2010, LPAR.

[110]  Gabriele Paganelli,et al.  Verifying (In-)Stability in Floating-Point Programs by Increasing Precision, Using SMT Solving , 2013, 2013 15th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing.

[111]  Peter Müller,et al.  Viper: A Verification Infrastructure for Permission-Based Reasoning , 2016, VMCAI.

[112]  Claude Marché,et al.  The KRAKATOA tool for certificationof JAVA/JAVACARD programs annotated in JML , 2004, J. Log. Algebraic Methods Program..

[113]  Marieke Huisman,et al.  History-Based Verification of Functional Behaviour of Concurrent Programs , 2015, SEFM.

[114]  K. Rustan M. Leino,et al.  The Dafny Integrated Development Environment , 2014, F-IDE.

[115]  Reiner Hähnle,et al.  An Interactive Verification Tool Meets an IDE , 2014, IFM.

[116]  Nikolai Kosmatov,et al.  Your Proof Fails? Testing Helps to Find the Reason , 2015, TAP@STAF.

[117]  Frank Piessens,et al.  Expressive modular fine-grained concurrency specification , 2011, POPL '11.

[118]  Ioannis T. Kassios The dynamic frames theory , 2010, Formal Aspects of Computing.

[119]  Reiner Hähnle,et al.  A General Lattice Model for Merging Symbolic Execution Branches , 2016, ICFEM.

[120]  Ken Robinson The B Method and the B Toolkit , 1997, AMAST.