Mastering Concurrent Computing Through Sequential Thinking: A Half-century Evolution

Concurrency, the art of doing many things at the same time is slowly becoming a science. It is very difficult to master, yet it arises all over modern computing systems, both when the communication medium is shared memory and when it is by message passing. Concurrent programming is hard because it requires to cope with many possible, unpredictable behaviors of communicating processes interacting with each other. Right from the start in the 1960s, the main way of dealing with concurrency has been by reduction to sequential reasoning. We trace this history, and illustrate it through several examples, from early ideas based on mutual exclusion, passing through consensus and concurrent objects, until today ledgers and blockchains. We conclude with a discussion on the limits that this approach encounters, related to fault-tolerance, performance, and inherently concurrent problems.

[1]  Jennifer L. Welch,et al.  Relaxed Data Types as Consistency Conditions , 2018, Algorithms.

[2]  Christian Cachin,et al.  State Machine Replication with Byzantine Faults , 2010, Replication.

[3]  Maurice Herlihy,et al.  Obstruction-free synchronization: double-ended queues as an example , 2003, 23rd International Conference on Distributed Computing Systems, 2003. Proceedings..

[4]  Maurice Herlihy,et al.  Wait-free synchronization , 1991, TOPL.

[5]  Gary L. Peterson,et al.  Concurrent Reading While Writing , 1983, TOPL.

[6]  Gadi Taubenfeld.  Synchronization Algorithms and Concurrent Programming , 2006 .

[7]  Maurice Herlihy,et al.  Linearizability: a correctness condition for concurrent objects , 1990, TOPL.

[8]  Sam Toueg,et al.  A Modular Approach to Fault-Tolerant Broadcasts and Related Problems , 1994 .

[9]  Satoshi Nakamoto.  Bitcoin : A Peer-to-Peer Electronic Cash System , 2009 .

[10]  Michel Raynal,et al.  Unifying Concurrent Objects and Distributed Tasks , 2018, J. ACM.

[11]  Michael O. Rabin,et al.  Randomized byzantine generals , 1983, 24th Annual Symposium on Foundations of Computer Science (sfcs 1983).

[12]  Michel Raynal,et al.  Distributed Universal Constructions: a Guided Tour , 2017, Bull. EATCS.

[13]  Sam Toueg,et al.  The weakest failure detector for solving consensus , 1992, PODC '92.

[14]  Hagit Attiya,et al.  Distributed Computing: Fundamentals, Simulations and Advanced Topics , 1998 .

[15]  Michel Raynal,et al.  The Distributed Computing Column , 2013 .

[16]  Leslie Lamport,et al.  How to Make a Multiprocessor Computer That Correctly Executes Multiprocess Programs , 2016, IEEE Transactions on Computers.

[17]  Gary L. Peterson,et al.  Myths About the Mutual Exclusion Problem , 1981, Inf. Process. Lett..

[18]  Marcos K. Aguilera,et al.  How to implement any concurrent data structure for modern servers , 2017, OPSR.

[19]  N. G. de Bruijn,et al.  Additional comments on a problem in concurrent programming control , 1967, Commun. ACM.

[20]  Danny Dolev,et al.  On the minimal synchronism needed for distributed consensus , 1983, 24th Annual Symposium on Foundations of Computer Science (sfcs 1983).

[21]  Seif Haridi,et al.  Distributed Algorithms , 1992, Lecture Notes in Computer Science.

[22]  Michel Raynal,et al.  Fault-Tolerant Message-Passing Distributed Systems , 2018, Springer International Publishing.

[23]  Ajay D. Kshemkalyani,et al.  Distributed Computing: Principles, Algorithms, and Systems , 2008 .

[24]  Eli Gafni,et al.  Recursion in Distributed Computing , 2010, SSS.

[25]  Michel Raynal,et al.  Distributed Universality , 2016, Algorithmica.

[26]  Leslie Lamport,et al.  A fast mutual exclusion algorithm , 1987, TOCS.

[27]  Bowen Alpern,et al.  Defining Liveness , 1984, Inf. Process. Lett..

[28]  Nancy A. Lynch,et al.  Perspectives on the CAP Theorem , 2012, Computer.

[29]  Marcos K. Aguilera,et al.  Consistency-based service level agreements for cloud storage , 2013, SOSP.

[30]  Maurice Herlihy,et al.  The art of multiprocessor programming , 2020, PODC '06.

[31]  Nir Shavit,et al.  Transactional Locking II , 2006, DISC.

[32]  Bharat K. Bhargava,et al.  Concurrency Control in Database Systems , 2019, IEEE Trans. Knowl. Data Eng..

[33]  Fred B. Schneider,et al.  Implementing fault-tolerant services using the state machine approach: a tutorial , 1990, CSUR.

[34]  Edsger W. Dijkstra,et al.  Solution of a problem in concurrent programming control , 1965, CACM.

[35]  Maurice Herlihy,et al.  Transactional Memory: Architectural Support For Lock-free Data Structures , 1993, Proceedings of the 20th Annual International Symposium on Computer Architecture.

[36]  Leslie Lamport,et al.  The Byzantine Generals Problem , 1982, TOPL.

[37]  Jiannong Cao,et al.  A Pleasant Stroll Through the Land of Distributed Machines, Computation, and Universality , 2018, MCU.

[38]  Leslie Lamport,et al.  Time, clocks, and the ordering of events in a distributed system , 1978, CACM.

[39]  Michael Ben-Or,et al.  Another advantage of free choice (Extended Abstract): Completely asynchronous agreement protocols , 1983, PODC '83.

[40]  Leslie Lamport,et al.  The part-time parliament , 1998, TOCS.

[41]  David Lorge Parnas,et al.  Concurrent control with “readers” and “writers” , 1971, CACM.

[42]  Nir Shavit,et al.  Software transactional memory , 1995, PODC '95.

[43]  Leslie Lamport,et al.  A new solution of Dijkstra's concurrent programming problem , 1974, Commun. ACM.

[44]  Michel Raynal,et al.  Power and limits of distributed computing shared memory models , 2013, Theor. Comput. Sci..

[45]  Nancy A. Lynch,et al.  Impossibility of distributed consensus with one faulty process , 1985, JACM.

[46]  Sam Toueg,et al.  Unreliable failure detectors for reliable distributed systems , 1996, JACM.

[47]  Nancy A. Lynch,et al.  Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services , 2002, SIGA.

[48]  Christos H. Papadimitriou,et al.  The serializability of concurrent database updates , 1979, JACM.

[49]  Leslie Lamport,et al.  On Interprocess Communication-Part I: Basic Formalism, Part II: Algorithms , 2016 .

[50]  Edsger W. Dijkstra,et al.  Hierarchical ordering of sequential processes , 1971, Acta Informatica.

[51]  Nancy A. Lynch,et al.  Consensus in the presence of partial synchrony , 1988, JACM.

[52]  Panagiota Fatourou,et al.  Highly-Efficient Wait-Free Synchronization , 2013, Theory of Computing Systems.

[53]  Leslie Lamport,et al.  Concurrent reading and writing , 1977, Commun. ACM.

[54]  Lukas Furst.  Concurrent Programming Algorithms Principles And Foundations , 2016 .

[55]  Edsger W. Dijkstra,et al.  Cooperating sequential processes , 2002 .

[56]  Ralph C. Merkle,et al.  A Digital Signature Based on a Conventional Encryption Function , 1987, CRYPTO.

[57]  Leslie Lamport,et al.  Reaching Agreement in the Presence of Faults , 1980, JACM.

[58]  Michel Raynal,et al.  Algorithms for mutual exclusion , 1986 .

[59]  Nir Shavit.  Data structures in the multicore age , 2011, CACM.

[60]  Michel Raynal,et al.  Distributed Algorithms for Message-Passing Systems , 2013, Springer Berlin Heidelberg.

[61]  Ana Sokolova,et al.  Local Linearizability for Concurrent Container-Type Data Structures , 2016, CONCUR.