Deterministic Consistency: A Programming Model for Shared Memory Parallelism

The difficulty of developing reliable parallel software is generating interest in deterministic environments, where a given program and input can yield only one possible result. Languages or type systems can enforce determinism in new code, and runtime systems can impose synthetic schedules on legacy parallel code. To parallelize existing serial code, however, we would like a programming model that is naturally deterministic without language restrictions or artificial scheduling. We propose deterministic consistency, a parallel programming model as easy to understand as the “parallel assignment” construct in sequential languages such as Perl and JavaScript, where concurrent threads always read their inputs before writing shared outputs. DC supports common data- and task-parallel synchronization abstractions such as fork/join and barriers, as well as non-hierarchical structures such as producer/consumer pipelines and futures. A preliminary prototype suggests that softwareonly implementations of DC can run applications written for popular parallel environments such as OpenMP with low (< 10%) overhead for some applications.

[1]  Charles Antony Richard Hoare Towards a theory of parallel programming , 2002 .

[2]  Andreas Haeberlen,et al.  PeerReview: practical accountability for distributed systems , 2007, SOSP.

[3]  Stuart I. Feldman,et al.  IGOR: a system for program debugging via reversible execution , 1988, PADD '88.

[4]  Armin Biere,et al.  High‐level data races , 2003, Softw. Test. Verification Reliab..

[5]  Thomas Ball,et al.  Finding and Reproducing Heisenbugs in Concurrent Programs , 2008, OSDI.

[6]  Dan Grossman,et al.  CoreDet: a compiler and runtime system for deterministic multithreaded execution , 2010, ASPLOS XV.

[7]  Vincent Lefèvre,et al.  Worst Cases of a Periodic Function for Large Arguments , 2007, 18th IEEE Symposium on Computer Arithmetic (ARITH '07).

[8]  Sarita V. Adve,et al.  Parallel programming must be deterministic by default , 2009 .

[9]  Tomás Lang,et al.  Bounds on runs of zeros and ones for algebraic functions , 2001, Proceedings 15th IEEE Symposium on Computer Arithmetic. ARITH-15 2001.

[10]  Nir Shavit,et al.  Software transactional memory , 1995, PODC '95.

[11]  Gilles Kahn,et al.  The Semantics of a Simple Language for Parallel Programming , 1974, IFIP Congress.

[12]  Edward A. Lee The problem with threads , 2006, Computer.

[13]  Emery D. Berger,et al.  Grace: safe multithreaded programming for C/C++ , 2009, OOPSLA 2009.

[14]  Earl E. Swartzlander,et al.  Hardware Designs for Exactly Rounded Elemantary Functions , 1994, IEEE Trans. Computers.

[15]  Robert H. Halstead,et al.  MULTILISP: a language for concurrent symbolic computation , 1985, TOPL.

[16]  Anoop Gupta,et al.  Memory consistency and event ordering in scalable shared-memory multiprocessors , 1990, [1990] Proceedings. The 17th Annual International Symposium on Computer Architecture.

[17]  Kai Li,et al.  The PARSEC benchmark suite: Characterization and architectural implications , 2008, 2008 International Conference on Parallel Architectures and Compilation Techniques (PACT).

[18]  Fred B. Schneider,et al.  Implementing fault-tolerant services using the state machine approach: a tutorial , 1990, CSUR.

[19]  Justin O. Graver,et al.  Multilisp: a Language for Concurrent Symbolic Computation Acm Transactions on Programming Languages and Systems , 1989 .

[20]  Stephen A. Edwards,et al.  SHIM: a deterministic model for heterogeneous embedded systems , 2005, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[21]  Alan L. Cox,et al.  Lazy release consistency for software distributed shared memory , 1992, ISCA '92.

[22]  Stephen A. Edwards,et al.  SHIM: a deterministic model for heterogeneous embedded systems , 2006, IEEE Trans. Very Large Scale Integr. Syst..

[23]  Dawson R. Engler,et al.  RacerX: effective, static detection of race conditions and deadlocks , 2003, SOSP '03.

[24]  Samuel T. King,et al.  Detecting past and present intrusions through vulnerability-specific predicates , 2005, SOSP '05.

[25]  Willy Zwaenepoel,et al.  Implementation and performance of Munin , 1991, SOSP '91.

[26]  Stephen A. Edwards,et al.  Scheduling-independent threads and exceptions in SHIM , 2006, EMSOFT '06.

[27]  Samuel T. King,et al.  ReVirt: enabling intrusion analysis through virtual-machine logging and replay , 2002, OPSR.

[28]  Bryan Ford,et al.  Vx32: Lightweight User-level Sandboxing on the x86 , 2008, USENIX Annual Technical Conference.

[29]  Larry D. Wittie,et al.  BUGNET: A Debugging system for parallel programming environments , 1982, ICDCS.

[30]  Brandon Lucia,et al.  DMP: deterministic shared memory multiprocessing , 2009, IEEE Micro.

[31]  Maurice Herlihy,et al.  Transactional Memory: Architectural Support For Lock-free Data Structures , 1993, Proceedings of the 20th Annual International Symposium on Computer Architecture.

[32]  Stephen N. Freund,et al.  SingleTrack: A Dynamic Determinism Checker for Multithreaded Programs , 2009, ESOP.

[33]  Miguel Oom Temudo de Castro,et al.  Practical Byzantine fault tolerance , 1999, OSDI '99.

[34]  Yuanyuan Zhou,et al.  Learning from mistakes: a comprehensive study on real world concurrency bug characteristics , 2008, ASPLOS.

[35]  Stephen A. Edwards,et al.  Programming Shared Memory Multiprocessors with Deterministic Message-Passing Concurrency: Compiling SHIM to Pthreads , 2008, 2008 Design, Automation and Test in Europe.

[36]  Anoop Gupta,et al.  Memory consistency and event ordering in scalable shared-memory multiprocessors , 1990, ISCA '90.

[37]  Robert N. M. Watson,et al.  Exploiting Concurrency Vulnerabilities in System Call Wrappers , 2007, WOOT.

[38]  Jeffrey Overbey,et al.  A type and effect system for deterministic parallel Java , 2009, OOPSLA 2009.

[39]  Thomas J. LeBlanc,et al.  Debugging Parallel Programs with Instant Replay , 1987, IEEE Transactions on Computers.

[40]  Leslie Lamport,et al.  How to Make a Multiprocessor Computer That Correctly Executes Multiprocess Programs , 2016, IEEE Transactions on Computers.

[41]  Alan L. Cox,et al.  TreadMarks: shared memory computing on networks of workstations , 1996 .

[42]  Stephen McCamant,et al.  Evaluating SFI for a CISC Architecture , 2006, USENIX Security Symposium.