Position paper: the science of deep specification

We introduce our efforts within the project ‘The science of deep specification’ to work out the key formal underpinnings of industrial-scale formal specifications of software and hardware components, anticipating a world where large verified systems are routinely built out of smaller verified components that are also used by many other projects. We identify an important class of specification that has already been used in a few experiments that connect strong component-correctness theorems across the work of different teams. To help popularize the unique advantages of that style, we dub it deep specification, and we say that it encompasses specifications that are rich, two-sided, formal and live (terms that we define in the article). Our core team is developing a proof-of-concept system (based on the Coq proof assistant) whose specification and verification work is divided across largely decoupled subteams at our four institutions, encompassing hardware microarchitecture, compilers, operating systems and applications, along with cross-cutting principles and tools for effective specification. We also aim to catalyse interest in the approach, not just by basic researchers but also by users in industry. This article is part of the themed issue ‘Verified trustworthy software systems’.

[1]  Lijun Zhang,et al.  Tools and Algorithms for the Construction and Analysis of Systems , 2019, Lecture Notes in Computer Science.

[2]  Toby Murray,et al.  Provably trustworthy systems , 2017, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[3]  John Launchbury,et al.  The HACMS program: using formal methods to eliminate exploitable bugs , 2017, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[4]  Adam Chlipala,et al.  Kami: a platform for high-level parametric hardware specification and its modular verification , 2017, Proc. ACM Program. Lang..

[5]  Cesare Tinelli,et al.  SMTCoq: A Plug-In for Integrating SMT Solvers into Coq , 2017, CAV.

[6]  Stefan Savage,et al.  How They Did It: An Analysis of Emission Defeat Devices in Modern Automobiles , 2017, 2017 IEEE Symposium on Security and Privacy (SP).

[7]  Other Contributors Are Indicated Where They Contribute The Free Software Foundation , 2017 .

[8]  Nikhil Swamy,et al.  Everest: Towards a Verified, Drop-in Replacement of HTTPS , 2017, SNAPL.

[9]  Zhong Shao,et al.  Toward Compositional Verification of Interruptible OS Kernels and Device Drivers , 2016, Journal of Automated Reasoning.

[10]  Adam Chlipala,et al.  The End of History? Using a Proof Assistant to Replace Language Design with Library Design , 2017, SNAPL.

[11]  Nicolas Christin,et al.  Push-Button Verification of File Systems via Crash Refinement , 2016, USENIX Annual Technical Conference.

[12]  Zhong Shao,et al.  CertiKOS: An Extensible Architecture for Building Certified Concurrent OS Kernels , 2016, OSDI.

[13]  W. Paul,et al.  System Architecture: An Ordinary Engineering Discipline , 2016 .

[14]  Stephen Kell,et al.  The missing link: explaining ELF static linking, semantically , 2016, OOPSLA.

[15]  Zhong Shao,et al.  End-to-end verification of information-flow security for C and assembly programs , 2016, PLDI.

[16]  Sidney Amani,et al.  Cogent: Verifying High-Assurance File System Implementations , 2016, ASPLOS.

[17]  Adam Chlipala,et al.  Using Crash Hoare logic for certifying the FSCQ file system , 2015, USENIX Annual Technical Conference.

[18]  Benjamin C. Pierce,et al.  A verified information-flow architecture , 2014, J. Comput. Secur..

[19]  Matthew Z. Weaver,et al.  CertiCoq : A verified compiler for Coq , 2016 .

[20]  Andrew W. Appel,et al.  Verified Correctness and Security of OpenSSL HMAC , 2015, USENIX Security Symposium.

[21]  Xi Wang,et al.  Verdi: a framework for implementing and formally verifying distributed systems , 2015, PLDI.

[22]  Andrew W. Appel,et al.  Verification of a Cryptographic Primitive: SHA-256 , 2015, TOPL.

[23]  C. Newcombe,et al.  How Amazon web services uses formal methods , 2015, Commun. ACM.

[24]  Adam Chlipala From Network Interface to Multithreaded Web Applications: A Case Study in Modular Program Verification , 2015, POPL.

[25]  Andrew W. Appel,et al.  Compositional CompCert , 2015, POPL.

[26]  Yu Guo,et al.  Deep Specifications and Certified Abstraction Layers , 2015, POPL.

[27]  J. Gregory Morrisett,et al.  The Foundational Cryptography Framework , 2014, POST.

[28]  Andrej Bauer,et al.  Programming with algebraic effects and handlers , 2012, J. Log. Algebraic Methods Program..

[29]  Tobias Nipkow,et al.  Concrete Semantics: With Isabelle/HOL , 2014 .

[30]  Vern Paxson,et al.  The Matter of Heartbleed , 2014, Internet Measurement Conference.

[31]  Danfeng Zhang,et al.  Ironclad Apps: End-to-End Security via Automated Full-System Verification , 2014, OSDI.

[32]  Andrew W. Appel,et al.  Program Logics for Certified Compilers , 2014 .

[33]  Arthur Charguéraud,et al.  A trusted mechanised JavaScript specification , 2014, POPL.

[34]  Ramana Kumar,et al.  CakeML: a verified implementation of ML , 2014, POPL.

[35]  Xinyu Feng,et al.  Characterizing Progress Properties of Concurrent Objects via Contextual Refinements , 2013, CONCUR.

[36]  Arjun Guha,et al.  Machine-verified network controllers , 2013, PLDI.

[37]  Santosh Nagarakatte,et al.  Formal verification of SSA-based optimizations for LLVM , 2013, PLDI.

[38]  Magnus O. Myreen,et al.  Translation validation for a verified OS kernel , 2013, PLDI.

[39]  Wolfgang J. Paul,et al.  Theory of Multi Core Hypervisor Verification , 2013, SOFSEM.

[40]  Achim D. Brucker,et al.  On theorem prover-based testing , 2012, Formal Aspects of Computing.

[41]  Lukas Bulwahn,et al.  The New Quickcheck for Isabelle - Random, Exhaustive and Symbolic Testing under One Roof , 2012, CPP.

[42]  Steve Zdancewic,et al.  Mechanized Verification of Computing Dominators for Formalizing Compilers , 2012, CPP.

[43]  Sorin Lerner,et al.  Establishing Browser Security Guarantees through Formal Shim Verification , 2012, USENIX Security Symposium.

[44]  Lukas Bulwahn,et al.  Smart Testing of Functional Programs in Isabelle , 2012, LPAR.

[45]  Milo M. K. Martin,et al.  Formalizing the LLVM intermediate representation for verified program transformations , 2012, POPL '12.

[46]  Peter H. Feiler,et al.  Model-Based Engineering with AADL , 2012 .

[47]  Jade Alglave,et al.  Understanding POWER multiprocessors , 2011, PLDI '11.

[48]  Xuejun Yang,et al.  Finding and understanding bugs in C compilers , 2011, PLDI '11.

[49]  Panagiotis Manolios,et al.  Integrating Testing and Interactive Theorem Proving , 2011, ACL2.

[50]  K. Claessen,et al.  QuickCheck: a lightweight tool for random testing of Haskell programs , 2000, SIGP.

[51]  Manuel M. T. Chakravarty,et al.  An llVM backend for GHC , 2010, Haskell.

[52]  Mark A. Hillebrand,et al.  Automated Verification of a Small Hypervisor , 2010, VSTTE.

[53]  Francesco Zappa Nardelli,et al.  x86-TSO , 2010, Commun. ACM.

[54]  Benjamin C. Pierce,et al.  Featherweight Firefox: Formalizing the Core of a Web Browser , 2010, WebApps.

[55]  Chris Hawblitzel,et al.  Safe to the last instruction: automated verification of a type-safe operating system , 2010, PLDI '10.

[56]  K. Rustan M. Leino,et al.  Dafny: An Automatic Program Verifier for Functional Correctness , 2010, LPAR.

[57]  J. Gregory Morrisett,et al.  Toward a verified relational database management system , 2010, POPL '10.

[58]  Thomas Santen,et al.  Verifying the Microsoft Hyper-V Hypervisor with VCC , 2009, FM.

[59]  Michael Norrish,et al.  seL4: formal verification of an OS kernel , 2009, SOSP '09.

[60]  Herman Geuvers,et al.  Social processes, program verification and all that , 2009, Mathematical Structures in Computer Science.

[61]  Mark A. Hillebrand,et al.  VCC: A Practical System for Verifying Concurrent C , 2009, TPHOLs.

[62]  Xavier Leroy,et al.  Formal verification of a realistic compiler , 2009, CACM.

[63]  Carl Eastlund DoubleCheck your theorems , 2009, ACL2 '09.

[64]  Peter W. O'Hearn,et al.  Abstraction for concurrent objects , 2009, Theor. Comput. Sci..

[65]  Gordon D. Plotkin,et al.  Handlers of Algebraic Effects , 2009, ESOP.

[66]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[67]  Fredrik Lindblad Property Directed Generation of First-Order Test Data , 2007, Trends in Functional Programming.

[68]  Daniel Jackson,et al.  Software Abstractions - Logic, Language, and Analysis , 2006 .

[69]  Fredrik Lindblad,et al.  A Tool for Automated Theorem Proving in Agda , 2004, TYPES.

[70]  Vikram S. Adve,et al.  LLVM: a compilation framework for lifelong program analysis & transformation , 2004, International Symposium on Code Generation and Optimization, 2004. CGO 2004..

[71]  Andrew W. Appel,et al.  A Trustworthy Proof Checker , 2004, Journal of Automated Reasoning.

[72]  Christoph Kreitz,et al.  Building reliable, high-performance networks with the Nuprl proof development system , 2004, Journal of Functional Programming.

[73]  Peter Dybjer,et al.  Combining Testing and Proving in Dependent Type Theory , 2003, TPHOLs.

[74]  J. S. Moore,et al.  A Grand Challenge Proposal for Formal Methods: A Verified Stack , 2002, 10th Anniversary Colloquium of UNU/IIST.

[75]  David B. MacQueen,et al.  The Definition of Standard ML (Revised) , 1997 .

[76]  Donald Sannella,et al.  The Definition of Extended ML: A Gentle Introduction , 1997, Theor. Comput. Sci..

[77]  Robert Pollack,et al.  How to Believe a Machine-Checked Proof , 1997 .

[78]  Savi Maharaj,et al.  Studying the ML Module System in HOL , 1995, Comput. J..

[79]  Benjamin Werner,et al.  Une Théorie des Constructions Inductives , 1994 .

[80]  Tom Melham,et al.  Higher Order Logic Theorem Proving and Its Applications , 1994, Lecture Notes in Computer Science.

[81]  Don Syme,et al.  Reasoning with the Formal Definition of Standard ML in HOL , 1993, HUG.

[82]  Elsa L. Gunter,et al.  Hol-ml , 1993, HUG.

[83]  Stefan Kahrs,et al.  Mistakes and Ambiguities in the definition of Standard ML , 1993 .

[84]  Robin Milner,et al.  Definition of standard ML , 1990 .

[85]  Robin Milner,et al.  Commentary on standard ML , 1990 .

[86]  J. Michael Spivey,et al.  The Z notation - a reference manual , 1992, Prentice Hall International Series in Computer Science.

[87]  Thierry Coquand,et al.  The Calculus of Constructions , 1988, Inf. Comput..

[88]  Richard J. Lipton,et al.  Social processes and proofs of theorems and programs , 1977, POPL.