CertiKOS: An Extensible Architecture for Building Certified Concurrent OS Kernels

Complete formal verification of a non-trivial concurrent OS kernel is widely considered a grand challenge. We present a novel compositional approach for building certified concurrent OS kernels. Concurrency allows interleaved execution of kernel/user modules across different layers of abstraction. Each such layer can have a different set of observable events. We insist on formally specifying these layers and their observable events, and then verifying each kernel module at its proper abstraction level. To support certified linking with other CPUs or threads, we prove a strong contextual refinement property for every kernel function, which states that the implementation of each such function will behave like its specification under any kernel/user context with any valid interleaving. We have successfully developed a practical concurrent OS kernel and verified its (contextual) functional correctness in Coq. Our certified kernel is written in 6500 lines of C and x86 assembly and runs on stock x86 multicore machines. To our knowledge, this is the first proof of functional correctness of a complete, general-purpose concurrent OS kernel with fine-grained locking.

[1]  Srinath T. V. Setty,et al.  IronFleet: proving practical distributed systems correct , 2015, SOSP.

[2]  Zhong Shao.  Certified software , 2010, Commun. ACM.

[3]  Natarajan Shankar,et al.  Design and verification of multi-rate distributed systems , 2015, 2015 ACM/IEEE International Conference on Formal Methods and Models for Codesign (MEMOCODE).

[4]  Mark A. Hillebrand,et al.  VCC: A Practical System for Verifying Concurrent C , 2009, TPHOLs.

[5]  Zhong Shao,et al.  Toward Compositional Verification of Interruptible OS Kernels and Device Drivers , 2017, Journal of Automated Reasoning.

[6]  Lawrence Robinson,et al.  A Provably Secure Operating System. , 1975 .

[7]  Xinyu Feng,et al.  A Practical Verification Framework for Preemptive OS Kernels , 2016, CAV.

[8]  Maurice Herlihy,et al.  The art of multiprocessor programming , 2020, PODC '06.

[9]  David Mosberger,et al.  httperf—a tool for measuring web server performance , 1998, PERV.

[10]  Gernot Heiser,et al.  For a Microkernel, a Big Lock Is Fine , 2015, APSys.

[11]  Thomas Anderson,et al.  Operating Systems: Principles and Practice , 2012 .

[12]  N. S. Barnett,et al.  Private communication , 1969 .

[13]  Dawson R. Engler,et al.  Exokernel: an operating system architecture for application-level resource management , 1995, SOSP.

[14]  Sidney Amani,et al.  Cogent: Verifying High-Assurance File System Implementations , 2016, ASPLOS.

[15]  Adam Chlipala,et al.  Using Crash Hoare logic for certifying the FSCQ file system , 2015, USENIX Annual Technical Conference.

[16]  Zhong Shao,et al.  End-to-end verification of information-flow security for C and assembly programs , 2016, PLDI.

[17]  Mark A. Hillebrand,et al.  Automated Verification of a Small Hypervisor , 2010, VSTTE.

[18]  Jochen Liedtke,et al.  On micro-kernel construction , 1995, SOSP.

[19]  Edsger W. Dijkstra,et al.  The structure of the “THE”-multiprogramming system , 1968, CACM.

[20]  Sam Tobin-Hochstadt,et al.  Run your research: on the effectiveness of lightweight mechanization , 2012, POPL '12.

[21]  Xinyu Feng,et al.  A program logic for concurrent objects under fair scheduling , 2016, POPL.

[22]  Danfeng Zhang,et al.  Ironclad Apps: End-to-End Security via Automated Full-System Verification , 2014, OSDI.

[23]  Jon Crowcroft,et al.  Unikernels: library operating systems for the cloud , 2013, ASPLOS '13.

[24]  Maurice Herlihy,et al.  Linearizability: a correctness condition for concurrent objects , 1990, TOPL.

[25]  Thomas Santen,et al.  Verifying the Microsoft Hyper-V Hypervisor with VCC , 2009, FM.

[26]  Eddie Kohler,et al.  Making information flow explicit in HiStar , 2006, OSDI '06.

[27]  M. Frans Kaashoek.  Parallel computing and the OS , 2015, SOSP History Day.

[28]  Edsger W. Dijkstra,et al.  Notes on structured programming , 1970 .

[29]  Nancy A. Lynch,et al.  Forward and Backward Simulations: I. Untimed Systems , 1995, Inf. Comput..

[30]  Chris Hawblitzel,et al.  Safe to the last instruction: automated verification of a type-safe operating system , 2010, PLDI '10.

[31]  William R. Bevier,et al.  Kit: A Study in Operating System Verification , 1989, IEEE Trans. Software Eng..

[32]  Richard J. Lipton,et al.  Social processes and proofs of theorems and programs , 1977, POPL.

[33]  James R. Larus,et al.  Singularity: rethinking the software stack , 2007, OPSR.

[34]  Xinyu Feng,et al.  Characterizing Progress Properties of Concurrent Objects via Contextual Refinements , 2013, CONCUR.

[35]  Richard J. Lipton,et al.  Social processes and proofs of theorems and programs , 1979, CACM.

[36]  Zhong Shao,et al.  End-to-end verification of stack-space bounds for C programs , 2014, PLDI.

[37]  Gernot Heiser,et al.  Comprehensive formal verification of an OS microkernel , 2014, TOCS.

[38]  Michael Norrish,et al.  seL4: formal verification of an OS kernel , 2009, SOSP '09.

[39]  Rusty Russell,et al.  virtio: towards a de-facto standard for virtual I/O devices , 2008, OPSR.

[40]  Christoforos E. Kozyrakis,et al.  Usenix Association 10th Usenix Symposium on Operating Systems Design and Implementation (osdi '12) 335 Dune: Safe User-level Access to Privileged Cpu Features , 2022 .

[41]  Michael von Tessin.  The clustered multikernel: an approach to formal verification of multiprocessor operating-system kernels , 2013 .

[42]  Xi Wang,et al.  Verdi: a framework for implementing and formally verifying distributed systems , 2015, PLDI.

[43]  Amer Diwan,et al.  The DaCapo benchmarks: java benchmarking development and analysis , 2006, OOPSLA '06.

[44]  Chris Hawblitzel,et al.  Safe to the last instruction: automated verification of a type-safe operating system , 2011, CACM.

[45]  N. Lynch,et al.  Forward and backward simulations , 1993 .

[46]  Yu Guo,et al.  Deep Specifications and Certified Abstraction Layers , 2015, POPL.

[47]  Michael L. Scott,et al.  Algorithms for scalable synchronization on shared-memory multiprocessors , 1991, TOCS.

[48]  Xavier Leroy,et al.  Formal verification of a realistic compiler , 2009, CACM.

[49]  Yu Guo,et al.  Certifying Low-Level Programs with Hardware Interrupts and Preemptive Threads , 2009, Journal of Automated Reasoning.

[50]  Peter W. O'Hearn,et al.  Abstraction for concurrent objects , 2009, Theor. Comput. Sci..

[51]  Xinyu Feng,et al.  A rely-guarantee-based simulation for verifying concurrent program transformations , 2012, POPL '12.

[52]  Butler W. Lampson,et al.  Experience with processes and monitors in Mesa , 1980, CACM.

[53]  Sagar Chaki,et al.  überSpark: Enforcing Verifiable Object Abstractions for Automated Compositional Security Analysis of a Hypervisor , 2016, USENIX Security Symposium.