VSync: push-button verification and optimization for synchronization primitives on weak memory models

Implementing highly efficient and correct synchronization primitives on modern Weak Memory Model (WMM) architectures, such as ARM and RISC-V, is very difficult even for human experts. We introduce VSync, a framework to assist in optimizing and verifying synchronization primitives on WMM architectures. VSync automatically detects missing and overly-constrained barriers, while ensuring essential safety and liveness properties. VSync relies on two novel techniques: 1) Adaptive Linear Relaxation (ALR), which utilizes barrier monotonicity and speculation to quickly find a correct maximally-relaxed barrier combination; and 2) Await Model Checking (AMC), which for the first time makes it possible to check termination of await loops on WMMs. We use VSync to automatically optimize and verify state-of-the-art synchronization primitives from systems like seL4, CertiKOS, musl libc, DPDK, Concurrency Kit, and Linux, as well as from the literature. In doing so, we found three correctness bugs on deployed systems due to missing barriers and several performance bugs due to overly-constrained barriers. Synchronization primitives optimized by VSync have similar performance to industrial libraries optimized by experts.

[1]  Konstantinos Sagonas,et al.  Stateless model checking of the Linux kernel's hierarchical read-copy-update (tree RCU) , 2017, SPIN.

[2]  Peter Sewell,et al.  Cerberus-BMC: A Principled Reference Semantics and Exploration Tool for Concurrent and Sequential C , 2019, CAV.

[3]  Francesco Zappa Nardelli,et al.  86-TSO : A Rigorous and Usable Programmer ’ s Model for x 86 Multiprocessors , 2010 .

[4]  Parosh Aziz Abdulla,et al.  Optimal stateless model checking under the release-acquire semantics , 2018, Proc. ACM Program. Lang..

[5]  Viktor Vafeiadis,et al.  HMC: Model Checking for Hardware Memory Models , 2020, ASPLOS.

[6]  Michael Norrish,et al.  seL4: formal verification of an OS kernel , 2009, SOSP '09.

[7]  Jeehoon Kang,et al.  A promising semantics for relaxed-memory concurrency , 2017, POPL.

[8]  Taesoo Kim,et al.  Scalable and practical locking with shuffling , 2019, SOSP.

[9]  Parosh Aziz Abdulla,et al.  Stateless Model Checking for TSO and PSO , 2015, TACAS.

[10]  Daniel Kroening,et al.  Don’t Sit on the Fence , 2013, ACM Trans. Program. Lang. Syst..

[11]  Parosh Aziz Abdulla,et al.  Stateless Model Checking for POWER , 2016, CAV.

[12]  Emina Torlak,et al.  Scaling symbolic evaluation for automated verification of systems code with Serval , 2019, SOSP.

[13]  Patrick Lam,et al.  SATCheck: SAT-directed stateless model checking for SC and TSO , 2015, OOPSLA.

[14]  Wang Yi,et al.  Uppaal in a nutshell , 1997, International Journal on Software Tools for Technology Transfer.

[15]  Norbert Schirmer,et al.  A Better Reduction Theorem for Store Buffers , 2009, ArXiv.

[16]  Joseph Tassarotti,et al.  Verifying concurrent, crash-safe systems with Perennial , 2019, SOSP.

[17]  Mark A. Hillebrand,et al.  Formal Verification of a Reader-Writer Lock Implementation in C , 2009, SSV.

[18]  Mark Moir,et al.  Adaptive integration of hardware and software lock elision techniques , 2014, SPAA.

[19]  Brian Demsky,et al.  A Practical Approach for Model Checking C/C++11 Code , 2016, TOPL.

[20]  Virendra J. Marathe,et al.  Lock cohorting: a general technique for designing NUMA locks , 2012, PPoPP '12.

[21]  Gernot Heiser,et al.  For a Microkernel, a Big Lock Is Fine , 2015, APSys.

[22]  Sérgio Vale Aguiar Campos,et al.  Symbolic Model Checking , 1993, CAV.

[23]  Ulrich Drepper,et al.  Futexes Are Tricky , 2004 .

[24]  Bernhard Beckert,et al.  Deductive Verification of System Software in the Verisoft XT Project , 2010, KI - Künstliche Intelligenz.

[25]  Alan Huang.  Maximally Stateless Model Checking for Concurrent Bugs under Relaxed Memory Models , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C).

[26]  Binyu Zang,et al.  No barrier in the road: a comprehensive study and optimization of ARM barriers , 2020, PPoPP.

[27]  Eran Yahav,et al.  Automatic inference of memory fences , 2010, Formal Methods in Computer Aided Design.

[28]  Viktor Vafeiadis,et al.  Validating optimizations of concurrent C/C++ programs , 2016, 2016 IEEE/ACM International Symposium on Code Generation and Optimization (CGO).

[29]  Nir Shavit,et al.  A Hierarchical CLH Queue Lock , 2006, Euro-Par.

[30]  W. Paul,et al.  System Architecture: An Ordinary Engineering Discipline , 2016 .

[31]  Thomas A. Henzinger,et al.  The software model checker B last : Applications to software engineering , 2007 .

[32]  Jaejin Lee,et al.  Automatic fence insertion for shared memory multiprocessing , 2003, ICS '03.

[33]  Viktor Vafeiadis,et al.  Verifying Fence Elimination Optimisations , 2011, SAS.

[34]  Alastair F. Donaldson,et al.  Software Model Checking , 2014, Computing Handbook, 3rd ed..

[35]  Viktor Vafeiadis.  Program Verification Under Weak Memory Consistency Using Separation Logic , 2017, CAV.

[36]  LahavOri,et al.  Effective stateless model checking for C/C++ concurrency , 2017 .

[37]  Rajeev Alur,et al.  An Axiomatic Memory Model for POWER Multiprocessors , 2012, CAV.

[38]  D. M. Hutton,et al.  The Art of Multiprocessor Programming , 2008 .

[39]  Parosh Aziz Abdulla,et al.  Context-Bounded Analysis for POWER , 2017, TACAS.

[40]  Stephen N. Freund,et al.  Exploiting purity for atomicity , 2004, IEEE Transactions on Software Engineering.

[41]  Viktor Vafeiadis,et al.  Common Compiler Optimisations are Invalid in the C11 Memory Model and what we can do about it , 2015, POPL.

[42]  Tuan-Phong Ngo,et al.  Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology 1745 Model Checking of Software Systems under Weak Memory Models , 2018 .

[43]  Xi Wang,et al.  Hyperkernel: Push-Button Verification of an OS Kernel , 2017, SOSP.

[44]  Jeehoon Kang,et al.  Repairing sequential consistency in C/C++11 , 2017, PLDI.

[45]  David Dice,et al.  TWA - Ticket Locks Augmented with a Waiting Array , 2019, Euro-Par.

[46]  Yunsup Lee,et al.  The RISC-V Instruction Set Manual , 2014 .

[47]  Leslie Lamport,et al.  Model Checking TLA+ Specifications , 1999, CHARME.

[48]  Jacob R. Lorch,et al.  Armada: low-effort verification of high-performance concurrent programs , 2020, PLDI.

[49]  Zhong Shao,et al.  CertiKOS: An Extensible Architecture for Building Certified Concurrent OS Kernels , 2016, OSDI.

[50]  Roland Meyer,et al.  BMC for Weak Memory Models: Relation Analysis for Compact SMT Encodings , 2019, CAV.

[51]  Ori Lahav,et al.  Effective stateless model checking for C/C++ concurrency , 2017, Proc. ACM Program. Lang..

[52]  Patrice Godefroid,et al.  Dynamic partial-order reduction for model checking software , 2005, POPL '05.

[53]  Zhong Shao,et al.  Building certified concurrent OS kernels , 2019, Commun. ACM.

[54]  Viktor Vafeiadis,et al.  Model checking for weakly consistent libraries , 2019, PLDI.

[55]  Martin Peschke,et al.  Design and Validation of Computer Protocols , 2003 .

[56]  Viktor Vafeiadis,et al.  Effective lock handling in stateless model checking , 2019, Proc. ACM Program. Lang..

[57]  Shaked Flur,et al.  Simplifying ARM concurrency: multicopy-atomic axiomatic and operational models for ARMv8 , 2017, Proc. ACM Program. Lang..

[58]  Jonas Oberhauser.  A Simpler Reduction Theorem for x86-TSO , 2015, VSTTE.

[59]  Bengt Jonsson.  State-space exploration for concurrent algorithms under weak memory orderings: (preliminary version) , 2009, CARN.

[60]  Michael L. Scott,et al.  Algorithms for scalable synchronization on shared-memory multiprocessors , 1991, TOCS.

[61]  Ali Sezgin,et al.  Modelling the ARMv8 architecture, operationally: concurrency and ISA , 2016, POPL.

[62]  Anton Podkopaev,et al.  Bridging the gap between programming languages and hardware weak memory models , 2018, Proc. ACM Program. Lang..

[63]  Sebastian Burckhardt,et al.  Memory model sensitive analysis of concurrent data types , 2007 .

[64]  Roland Meyer,et al.  Robustness against Relaxed Memory Models , 2014, Software Engineering.

[65]  Parosh Aziz Abdulla,et al.  Verification of programs under the release-acquire semantics , 2019, PLDI.

[66]  M. Hill,et al.  Weak ordering-a new definition , 1990, [1990] Proceedings. The 17th Annual International Symposium on Computer Architecture.

[67]  Zhong Shao,et al.  Safety and Liveness of MCS Lock - Layer by Layer , 2017, APLAS.

[68]  Ronghui Gu,et al.  Using concurrent relational logic with helpers for verifying the AtomFS file system , 2019, SOSP.