Efficient system-enforced deterministic parallelism

Deterministic execution offers many benefits for debugging, fault tolerance, and security. Current methods of executing parallel programs deterministically, however, often incur high costs, allow misbehaved software to defeat repeatability, and transform time-dependent races into input-or path-dependent races without eliminating them. We introduce a new parallel programming model addressing these issues, and use Determinator, a proof-of-concept OS, to demonstrate the model's practicality. Determinator's microkernel application programming interface (API) provides only "shared-nothing" address spaces and deterministic interprocess communication primitives to make execution of all unprivileged code---well-behaved or not---precisely repeatable. Atop this microkernel, Determinator's user-level runtime offers a private workspace model for both thread-level and process-level parallel programming. This model avoids the introduction of read/write data races, and converts write/write races into reliably detected conflicts. Coarse-grained parallel benchmarks perform and scale comparably to non-deterministic systems, both on multicore PCs and across nodes in a distributed cluster.

[1]  Luis Ceze,et al.  Deterministic Process Groups in dOS , 2010, OSDI.

[2]  Ramakrishna Gummadi,et al.  Determinating timing channels in compute clouds , 2010, CCSW '10.

[3]  Peter M. Chen,et al.  Execution replay of multiprocessor virtual machines , 2008, VEE '08.

[4]  Stephen A. Edwards,et al.  SHIM: a deterministic model for heterogeneous embedded systems , 2006, IEEE Trans. Very Large Scale Integr. Syst..

[5]  Stephen N. Freund,et al.  SingleTrack: A Dynamic Determinism Checker for Multithreaded Programs , 2009, ESOP.

[6]  T. Chiueh,et al.  Integrating segmentation and paging protection for safe, efficient and transparent software extensions , 2000, OPSR.

[7]  Larry D. Wittie,et al.  BUGNET: A Debugging system for parallel programming environments , 1982, ICDCS.

[8]  Dawson R. Engler,et al.  RacerX: effective, static detection of race conditions and deadlocks , 2003, SOSP '03.

[9]  Anoop Gupta,et al.  The SPLASH-2 programs: characterization and methodological considerations , 1995, ISCA.

[10]  Bryan Ford,et al.  Deterministic Consistency: A Programming Model for Shared Memory Parallelism , 2009, 0912.0926.

[11]  Marek Olszewski,et al.  Kendo: efficient deterministic multithreading in software , 2009, ASPLOS.

[12]  Jochen Liedtke,et al.  On micro-kernel construction , 1995, SOSP.

[13]  Anoop Gupta,et al.  Memory consistency and event ordering in scalable shared-memory multiprocessors , 1990, ISCA '90.

[14]  Ramakrishna Gummadi,et al.  Determinating Timing Channels in Statistically Multiplexed Clouds , 2010, ArXiv.

[15]  Tal Garfinkel,et al.  Compatibility Is Not Transparency: VMM Detection Myths and Realities , 2007, HotOS.

[16]  Miguel Oom Temudo de Castro,et al.  Practical Byzantine fault tolerance , 1999, OSDI '99.

[17]  Tzi-cker Chiueh,et al.  Integrating segmentation and paging protection for safe, efficient and transparent software extensions , 1999, SOSP.

[18]  Yang Zhang,et al.  Corey: An Operating System for Many Cores , 2008, OSDI.

[19]  Fred B. Schneider,et al.  Implementing fault-tolerant services using the state machine approach: a tutorial , 1990, CSUR.

[20]  Sarita V. Adve,et al.  Parallel programming must be deterministic by default , 2009 .

[21]  Calton Pu,et al.  TOCTTOU vulnerabilities in UNIX-style file systems: an anatomical study , 2005, FAST'05.

[22]  Andreas Haeberlen,et al.  Accountable Virtual Machines , 2010, OSDI.

[23]  Bryan Ford,et al.  Workspace Consistency : A Programming Model for Shared Memory Parallelism , 2011 .

[24]  Donald E. Porter,et al.  Operating System Transactions , 2009, SOSP '09.

[25]  Dawson R. Engler,et al.  Exokernel: an operating system architecture for application-level resource management , 1995, SOSP.

[26]  Alley Stoughton,et al.  Detection of Mutual Inconsistency in Distributed Systems , 1983, IEEE Transactions on Software Engineering.

[27]  Gilles Kahn,et al.  The Semantics of a Simple Language for Parallel Programming , 1974, IFIP Congress.

[28]  Brandon Lucia,et al.  DMP: Deterministic Shared-Memory Multiprocessing , 2010, IEEE Micro.

[29]  Edward A. Lee The problem with threads , 2006, Computer.

[30]  Emery D. Berger,et al.  Grace: safe multithreaded programming for C/C++ , 2009, OOPSLA '09.

[31]  Junfeng Yang,et al.  Stable Deterministic Multithreading through Schedule Memoization , 2010, OSDI.

[32]  Alan L. Cox,et al.  TreadMarks: shared memory computing on networks of workstations , 1996 .

[33]  GrayJim,et al.  A critique of ANSI SQL isolation levels , 1995 .

[34]  Monica Beltrametti,et al.  The control mechanism for the Myrias parallel computer system , 1988, CARN.

[35]  Larry D. Wittie The Bugnet distributed debugging system , 1986, EW 2.

[36]  Nir Shavit,et al.  Software transactional memory , 1995, PODC '95.

[37]  Yuanyuan Zhou,et al.  Learning from mistakes: a comprehensive study on real world concurrency bug characteristics , 2008, ASPLOS.

[38]  Stephen A. Edwards,et al.  Programming Shared Memory Multiprocessors with Deterministic Message-Passing Concurrency: Compiling SHIM to Pthreads , 2008, 2008 Design, Automation and Test in Europe.

[39]  Koushik Sen,et al.  Asserting and checking determinism for multithreaded programs , 2009, ESEC/FSE '09.

[40]  Mike Hibler,et al.  Microkernels meet recursive virtual machines , 1996, OSDI '96.

[41]  Stuart I. Feldman,et al.  IGOR: a system for program debugging via reversible execution , 1988, PADD '88.

[42]  Armin Biere,et al.  High‐level data races , 2003, Softw. Test. Verification Reliab..

[43]  Thomas Ball,et al.  Finding and Reproducing Heisenbugs in Concurrent Programs , 2008, OSDI.

[44]  Samuel T. King,et al.  Debugging Operating Systems with Time-Traveling Virtual Machines (Awarded General Track Best Paper Award!) , 2005, USENIX Annual Technical Conference, General Track.

[45]  Maurice Herlihy,et al.  Transactional Memory: Architectural Support For Lock-free Data Structures , 1993, Proceedings of the 20th Annual International Symposium on Computer Architecture.

[46]  George C. Necula,et al.  Capriccio: scalable threads for internet services , 2003, SOSP '03.

[47]  Fred B. Schneider,et al.  Hypervisor-based fault tolerance , 1996, TOCS.

[48]  Alan L. Cox,et al.  Lazy release consistency for software distributed shared memory , 1992, ISCA '92.

[49]  BeltramettiMonica,et al.  The control mechanism for the Myrias parallel computer system , 1988 .

[50]  Jim Gray,et al.  A critique of ANSI SQL isolation levels , 1995, SIGMOD '95.

[51]  Jong-Deok Choi,et al.  Deterministic replay of Java multithreaded applications , 1998, SPDT '98.

[52]  Mark A. Linton,et al.  Supporting reverse execution for parallel programs , 1988, PADD '88.

[53]  Jeffrey Overbey,et al.  A type and effect system for deterministic parallel Java , 2009, OOPSLA '09.

[54]  Bruce Walker,et al.  The LOCUS distributed operating system , 1983, SOSP '83.

[55]  Fabrice Bellard,et al.  QEMU, a Fast and Portable Dynamic Translator , 2005, USENIX ATC, FREENIX Track.

[56]  Stephen A. Edwards,et al.  Scheduling-independent threads and exceptions in SHIM , 2006, EMSOFT '06.

[57]  Samuel T. King,et al.  ReVirt: enabling intrusion analysis through virtual-machine logging and replay , 2002, OPSR.

[58]  Brian N. Bershad,et al.  Extensibility safety and performance in the SPIN operating system , 1995, SOSP.

[59]  Andreas Haeberlen,et al.  Practical accountability for distributed systems , 2007 .

[60]  Thomas J. LeBlanc,et al.  Debugging Parallel Programs with Instant Replay , 1987, IEEE Transactions on Computers.

[61]  Leslie Lamport,et al.  How to Make a Multiprocessor Computer That Correctly Executes Multiprocess Programs , 2016, IEEE Transactions on Computers.

[62]  David A. Wagner,et al.  A Secure Environment for Untrusted Helper Applications , 1996, USENIX Security Symposium.

[63]  Marvin Theimer,et al.  Managing update conflicts in Bayou, a weakly connected replicated storage system , 1995, SOSP.

[64]  Samuel T. King,et al.  Detecting past and present intrusions through vulnerability-specific predicates , 2005, SOSP '05.

[65]  Willy Zwaenepoel,et al.  Implementation and performance of Munin , 1991, SOSP '91.

[66]  Andreas Haeberlen,et al.  PeerReview: practical accountability for distributed systems , 2007, SOSP.

[67]  Dan Grossman,et al.  CoreDet: a compiler and runtime system for deterministic multithreaded execution , 2010, ASPLOS XV.

[68]  Robert H. Halstead,et al.  MULTILISP: a language for concurrent symbolic computation , 1985, TOPL.

[69]  Anoop Gupta,et al.  Memory consistency and event ordering in scalable shared-memory multiprocessors , 1990, [1990] Proceedings. The 17th Annual International Symposium on Computer Architecture.

[70]  Kai Li,et al.  The PARSEC benchmark suite: Characterization and architectural implications , 2008, 2008 International Conference on Parallel Architectures and Compilation Techniques (PACT).