Scaling symbolic evaluation for automated verification of systems code with Serval

This paper presents Serval, a framework for developing automated verifiers for systems software. Serval provides an extensible infrastructure for creating verifiers by lifting interpreters under symbolic evaluation, and a systematic approach to identifying and repairing verification performance bottlenecks using symbolic profiling and optimizations. Using Serval, we build automated verifiers for the RISC-V, x86--32, LLVM, and BPF instruction sets. We report our experience of retrofitting CertiKOS and Komodo, two systems previously verified using Coq and Dafny, respectively, for automated verification using Serval, and discuss trade-offs of different verification methodologies. In addition, we apply Serval to the Keystone security monitor and the BPF compilers in the Linux kernel, and uncover 18 new bugs through verification, all confirmed and fixed by developers.

[1]  Michael Norrish,et al.  seL4: formal verification of an OS kernel , 2009, SOSP '09.

[2]  Christoforos E. Kozyrakis,et al.  Usenix Association 10th Usenix Symposium on Operating Systems Design and Implementation (osdi '12) 335 Dune: Safe User-level Access to Privileged Cpu Features , 2022 .

[3]  Emina Torlak,et al.  Growing solver-aided languages with rosette , 2013, Onward!.

[4]  Srinath T. V. Setty,et al.  IronFleet: proving practical distributed systems correct , 2015, SOSP.

[5]  Gerwin Klein,et al.  Operating system verification—An overview , 2009 .

[6]  Dawson R. Engler,et al.  EXE: automatically generating inputs of death , 2006, CCS '06.

[7]  Srinath T. V. Setty,et al.  Vale: Verifying High-Performance Cryptographic Assembly Code , 2017, USENIX Security Symposium.

[8]  James C. King,et al.  Symbolic execution and program testing , 1976, CACM.

[9]  Vikram S. Adve,et al.  LLVM: a compilation framework for lifelong program analysis & transformation , 2004, International Symposium on Code Generation and Optimization, 2004. CGO 2004..

[10]  Armin Biere,et al.  Symbolic Model Checking without BDDs , 1999, TACAS.

[11]  K. Rustan M. Leino,et al.  Extended static checking , 1998, PROCOMET.

[12]  Nicolas Christin,et al.  Push-Button Verification of File Systems via Crash Refinement , 2016, USENIX Annual Technical Conference.

[13]  Carlos V. Rozas,et al.  Intel® Software Guard Extensions (Intel® SGX) Support for Dynamic Memory Management Inside an Enclave , 2016, HASP 2016.

[14]  Adam Chlipala.  From Network Interface to Multithreaded Web Applications: A Case Study in Modular Program Verification , 2015, POPL.

[15]  Andrew Ferraiuolo,et al.  Komodo: Using verification to disentangle secure-enclave hardware from software , 2017, SOSP.

[16]  Krste Asanovic,et al.  The RISC-V Instruction Set Manual Volume 2: Privileged Architecture Version 1.7 , 2015 .

[17]  Adrian Perrig,et al.  SecVisor: a tiny hypervisor to provide lifetime kernel code integrity for commodity OSes , 2007, SOSP.

[18]  Christian Jacobi,et al.  Putting it all together – Formal verification of the VAMP , 2006, International Journal on Software Tools for Technology Transfer.

[19]  Leslie Lamport.  Computation and State Machines , 2008 .

[20]  Peng Li,et al.  Downgrading policies and relaxed noninterference , 2005, POPL '05.

[21]  Sam Tobin-Hochstadt,et al.  A programmable programming language , 2018, Commun. ACM.

[22]  Xi Wang,et al.  Investigating Safety of a Radiotherapy Machine Using System Models with Pluggable Checkers , 2016, CAV.

[23]  Emina Torlak,et al.  Finding code that explodes under symbolic evaluation , 2018, Proc. ACM Program. Lang..

[24]  Zhong Shao,et al.  End-to-end verification of stack-space bounds for C programs , 2014, PLDI.

[25]  Chris Hawblitzel,et al.  Safe to the last instruction: automated verification of a type-safe operating system , 2011, CACM.

[26]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[27]  Brian Huffman,et al.  Continuous Formal Verification of Amazon s2n , 2018, CAV.

[28]  Danfeng Zhang,et al.  Ironclad Apps: End-to-End Security via Automated Full-System Verification , 2014, OSDI.

[29]  Leonardo Mendonça de Moura,et al.  Solving non-linear arithmetic , 2012, ACCA.

[30]  Michael Hamburg,et al.  Spectre Attacks: Exploiting Speculative Execution , 2018, 2019 IEEE Symposium on Security and Privacy (SP).

[31]  Emina Torlak,et al.  Nickel: A Framework for Design and Verification of Information Flow Control Systems , 2018, OSDI.

[32]  Robert S. Boyer,et al.  The Boyer-Moore theorem prover and its interactive enhancement , 1995 .

[33]  Nikolaj Bjørner,et al.  Bugs, Moles and Skeletons: Symbolic Reasoning for Software Development , 2010, IJCAR.

[34]  Dan Tsafrir,et al.  Virtual CPU validation , 2015, SOSP.

[35]  Gernot Heiser,et al.  Comprehensive formal verification of an OS microkernel , 2014, TOCS.

[36]  Srinivas Devadas,et al.  Sanctum: Minimal Hardware Extensions for Strong Software Isolation , 2016, USENIX Security Symposium.

[37]  Lawrence Charles Paulson,et al.  Isabelle/HOL: A Proof Assistant for Higher-Order Logic , 2002 .

[38]  Wolfgang J. Paul,et al.  Pervasive Verification of an OS Microkernel - Inline Assembly, Memory Consumption, Concurrent Devices , 2010, VSTTE.

[39]  Alastair David Reid.  Who guards the guards? formal validation of the Arm v8-m architecture specification , 2017, Proc. ACM Program. Lang..

[40]  Patrice Godefroid,et al.  Proving Memory Safety of the ANI Windows Image Parser Using Compositional Exhaustive Testing , 2015, VMCAI.

[41]  Peter Sestoft,et al.  Partial evaluation and automatic program generation , 1993, Prentice Hall international series in computer science.

[42]  Dawn Xiaodong Song,et al.  Keystone: A Framework for Architecting TEEs , 2019, ArXiv.

[43]  Xi Wang,et al.  Jitk: A Trustworthy In-Kernel Interpreter Infrastructure , 2014, OSDI.

[44]  Lori A. Clarke,et al.  A System to Generate Test Data and Symbolically Execute Programs , 1976, IEEE Transactions on Software Engineering.

[45]  Yu Guo,et al.  Deep Specifications and Certified Abstraction Layers , 2015, POPL.

[46]  Robert M. Norton,et al.  ISA semantics for ARMv8-a, RISC-v, and CHERI-MIPS , 2019, Proc. ACM Program. Lang..

[47]  Armando Solar-Lezama,et al.  Towards optimization-safe systems: analyzing the impact of undefined behavior , 2013, SOSP.

[48]  Xavier Leroy,et al.  The CompCert Memory Model, Version 2 , 2012 .

[49]  Yunsup Lee,et al.  The RISC-V Instruction Set Manual , 2014 .

[50]  John Rushby,et al.  Noninterference, Transitivity, and Channel-Control Security Policies 1 , 2005 .

[51]  Roberto Baldoni,et al.  A Survey of Symbolic Execution Techniques , 2016, ACM Comput. Surv..

[52]  K. Rustan M. Leino,et al.  Dafny: An Automatic Program Verifier for Functional Correctness , 2010, LPAR.

[53]  Adam Chlipala,et al.  Verifying a high-performance crash-safe file system using a tree specification , 2017, SOSP.

[54]  Patrice Godefroid,et al.  SAGE: Whitebox Fuzzing for Security Testing , 2012, ACM Queue.

[55]  Zhong Shao,et al.  CertiKOS: An Extensible Architecture for Building Certified Concurrent OS Kernels , 2016, OSDI.

[56]  Xavier Leroy,et al.  Formal verification of a realistic compiler , 2009, CACM.

[57]  Sidney Amani,et al.  Refinement through restraint: bringing down the cost of verification , 2016, ICFP 2016.

[58]  Magnus O. Myreen,et al.  Translation validation for a verified OS kernel , 2013, PLDI.

[59]  Xi Wang,et al.  Hyperkernel: Push-Button Verification of an OS Kernel , 2017, SOSP.

[60]  Thomas F. Wenisch,et al.  Foreshadow: Extracting the Keys to the Intel SGX Kingdom with Transient Out-of-Order Execution , 2018, USENIX Security Symposium.

[61]  Cristian Cadar,et al.  Targeted program transformations for symbolic execution , 2015, ESEC/SIGSOFT FSE.

[62]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[63]  Thomas W. Reps,et al.  Partial evaluation of machine code , 2015, OOPSLA.

[64]  Xavier Leroy,et al.  Mechanized Semantics for the Clight Subset of the C Language , 2009, Journal of Automated Reasoning.

[65]  Alexander Aiken,et al.  Secure Information Flow as a Safety Problem , 2005, SAS.

[66]  George Candea,et al.  -OVERIFY: Optimizing Programs for Fast Verification , 2013, HotOS.

[67]  Michael Hamburg,et al.  Meltdown: Reading Kernel Memory from User Space , 2018, USENIX Security Symposium.

[68]  Samuel T. King,et al.  Verifying security invariants in ExpressOS , 2013, ASPLOS '13.

[69]  Bor-Yuh Evan Chang,et al.  Boogie: A Modular Reusable Verifier for Object-Oriented Programs , 2005, FMCO.

[70]  William R. Bevier,et al.  Kit: A Study in Operating System Verification , 1989, IEEE Trans. Software Eng..

[71]  Sidney Amani,et al.  Cogent: Verifying High-Assurance File System Implementations , 2016, ASPLOS.

[72]  Zhong Shao,et al.  End-to-end verification of information-flow security for C and assembly programs , 2016, PLDI.

[73]  Emina Torlak,et al.  A lightweight symbolic virtual machine for solver-aided host languages , 2014, PLDI.

[74]  Adam Chlipala,et al.  Using Crash Hoare logic for certifying the FSCQ file system , 2015, USENIX Annual Technical Conference.

[75]  Michael D. Ernst,et al.  SpaceSearch: a library for building and verifying solver-aided tools , 2017, Proc. ACM Program. Lang..

[76]  Katerina J. Argyraki,et al.  Software dataplane verification , 2014, NSDI.

[77]  Koushik Sen,et al.  Symbolic execution for software testing: three decades later , 2013, CACM.

[78]  Timothy Bourke,et al.  seL4: From General Purpose to a Proof of Information Flow Enforcement , 2013, 2013 IEEE Symposium on Security and Privacy.

[79]  J. Meseguer,et al.  Security Policies and Security Models , 1982, 1982 IEEE Symposium on Security and Privacy.

[80]  George Candea,et al.  Efficient state merging in symbolic execution , 2012, Software Engineering.

[81]  George Candea,et al.  S2E: a platform for in-vivo multi-path analysis of software systems , 2011, ASPLOS XVI.

[82]  Daniel Jackson,et al.  Lightweight Formal Methods , 2001, FME.

[83]  Haibo Chen,et al.  CloudVisor: retrofitting protection of virtual machines in multi-tenant cloud with nested virtualization , 2011, SOSP.