Consistency without borders

Distributed consistency is a perennial research topic; in recent years it has become an urgent practical matter as well. The research literature has focused on enforcing various flavors of consistency at the I/O layer, such as linearizability of read/write registers. For practitioners, strong I/O consistency is often impractical at scale, while looser forms of I/O consistency are difficult to map to application-level concerns. Instead, it is common for developers to take matters of distributed consistency into their own hands, leading to application-specific solutions that are tricky to write, test and maintain. In this paper, we agitate for the technical community to shift its attention to approaches that lie between the extremes of I/O-level and application-level consistency. We ground our discussion in early work in the area, including our own experiences building programmer tools and languages that help developers guarantee distributed consistency at the application level. Much remains to be done, and we highlight some of the challenges that we feel deserve more attention.

[1]  David Maier,et al.  Testing implications of data dependencies , 1979, SIGMOD '79.

[2]  Christos H. Papadimitriou,et al.  The serializability of concurrent database updates , 1979, JACM.

[3]  Philip A. Bernstein,et al.  Concurrency Control in Distributed Database Systems , 1986, CSUR.

[4]  K. Mani Chandy,et al.  A distributed algorithm for detecting resource deadlocks in distributed systems , 1982, PODC '82.

[5]  Hector Garcia-Molina,et al.  Using semantic knowledge for transaction processing in a distributed database , 1983, TODS.

[6]  Alfred Z. Spector,et al.  Synchronizing shared abstract types , 1984, TOCS.

[7]  Leslie Lamport,et al.  Distributed snapshots: determining global states of distributed systems , 1985, TOCS.

[8]  Hector Garcia-Molina,et al.  Consistency in a partitioned network: a survey , 1985, CSUR.

[9]  Maurice Herlihy Optimistic concurrency control for abstract data types , 1986, PODC '86.

[10]  Patrick E. O'Neil,et al.  The Escrow transactional method , 1986, TODS.

[11]  B. R. Badrinath,et al.  Semantics-based concurrency control: Beyond commutativity , 1987, 1987 IEEE Third International Conference on Data Engineering.

[12]  Kenneth P. Birman,et al.  Exploiting virtual synchrony in distributed systems , 1987, SOSP '87.

[13]  Michael Stonebraker,et al.  Semantics based transaction management techniques for replicated data , 1988, SIGMOD '88.

[14]  William E. Weihl,et al.  Commutativity-based concurrency control for abstract data types , 1988, [1988] Proceedings of the Twenty-First Annual Hawaii International Conference on System Sciences. Volume II: Software track.

[15]  Yehoshua Sagiv,et al.  Inference of monotonicity constraints in Datalog programs , 1989, PODS '89.

[16]  M. Tamer Özsu,et al.  Using semantic knowledge of transactions to increase concurrency , 1989, TODS.

[17]  Maurice Herlihy,et al.  Linearizability: a correctness condition for concurrent objects , 1990, TOPL.

[18]  Fred B. Schneider,et al.  Implementing fault-tolerant services using the state machine approach: a tutorial , 1990, CSUR.

[19]  David R. Cheriton,et al.  Understanding the limitations of causally and totally ordered communication , 1994, SOSP '93.

[20]  Inderpal Singh Mumick,et al.  The Stanford Data Warehousing Project , 1995 .

[21]  Lorenz Huelsbergen,et al.  Very concurrent mark-&-sweep garbage collection without fine-grain synchronization , 1998, ISMM '98.

[22]  Graem A. Ringwood,et al.  Garbage collecting the Internet: a survey of distributed garbage collection , 1998, CSUR.

[23]  Nancy A. Lynch,et al.  Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services , 2002, SIGA.

[24]  David Maier,et al.  Exploiting Punctuation Semantics in Continuous Data Streams , 2003, IEEE Trans. Knowl. Data Eng..

[25]  A. Schiper,et al.  Total order broadcast and multicast algorithms: Taxonomy and survey , 2003, CSUR.

[26]  Hector Garcia-Molina,et al.  The demarcation protocol: A technique for maintaining constraints in distributed database systems , 1994, The VLDB Journal.

[27]  André Schiper,et al.  Strong stable properties in distributed systems , 1994, Distributed Computing.

[28]  Alan Fekete,et al.  Allocating isolation levels to transactions , 2005, PODS '05.

[29]  Werner Vogels,et al.  Dynamo: amazon's highly available key-value store , 2007, SOSP.

[30]  Hans-Arno Jacobsen,et al.  PNUTS: Yahoo!'s hosted data serving platform , 2008, Proc. VLDB Endow..

[31]  Werner Vogels,et al.  Building reliable distributed systems at a worldwide scale demands trade-offs between consistency and availability. , 2022 .

[32]  James Cheney,et al.  Provenance in Databases: Why, How, and Where , 2009, Found. Trends Databases.

[33]  Robbert van Renesse,et al.  Toward a cloud computing research agenda , 2009, SIGA.

[34]  Pat Helland,et al.  Building on Quicksand , 2009, CIDR.

[35]  Amit A. Levy,et al.  Comet: An active distributed key-value store , 2010, OSDI.

[36]  Joseph M. Hellerstein,et al.  Boom analytics: exploring data-centric, declarative programming for the cloud , 2010, EuroSys '10.

[37]  David Maier,et al.  Dedalus: Datalog in Time and Space , 2010, Datalog.

[38]  José Martinez,et al.  A framework for designing concurrent and recoverable abstract data types based on commutativity , 2010, ArXiv.

[39]  Joseph M. Hellerstein,et al.  The declarative imperative: experiences and conjectures in distributed logic , 2010, SGMD.

[40]  Joseph M. Hellerstein,et al.  Consistency Analysis in Bloom: a CALM and Collected Approach , 2011, CIDR.

[41]  Michael J. Freedman,et al.  Don't settle for eventual: scalable causal consistency for wide-area storage with COPS , 2011, SOSP.

[42]  Joonwon Lee,et al.  Parallel Distrib , 2022 .

[43]  Marc Shapiro,et al.  Conflict-Free Replicated Data Types , 2011, SSS.

[44]  Marcos K. Aguilera,et al.  Transactional storage for geo-replicated systems , 2011, SOSP.

[45]  David Maier,et al.  Logic and lattices for distributed programming , 2012, SoCC '12.

[46]  Christopher Frost,et al.  Spanner: Google's Globally-Distributed Database , 2012, OSDI.

[47]  Ali Ghodsi,et al.  The potential dangers of causal consistency and an explicit solution , 2012, SoCC '12.

[48]  David Maier,et al.  Confluence Analysis for Distributed Programs: A Model-Theoretic Approach , 2012, Datalog.

[49]  Jonathan Leibiusky,et al.  Getting Started with Storm - Continuous Streaming Computation with Twitter's Cluster Technology , 2012 .

[50]  Jonathan Leibiusky,et al.  Getting Started with Storm , 2012 .

[51]  Ian Rae,et al.  F1: A Distributed SQL Database That Scales , 2013, Proc. VLDB Endow..

[52]  Ali Ghodsi,et al.  Highly Available Transactions: Virtues and Limitations , 2013, Proc. VLDB Endow..

[53]  Pat Helland,et al.  Engagements: Building Eventually ACiD Business Transactions , 2013, CIDR.

[54]  Frank Neven,et al.  Relational transducers for declarative networking , 2010, JACM.

[55]  Michael J. Freedman,et al.  Stronger Semantics for Low-Latency Geo-Replicated Storage , 2013, NSDI.

[56]  Hui Ding,et al.  TAO: Facebook's Distributed Data Store for the Social Graph , 2013, USENIX Annual Technical Conference.

[57]  Marc Shapiro,et al.  Non-monotonic Snapshot Isolation: Scalable and Strong Consistency for Geo-replicated Transactional Systems , 2013, 2013 IEEE 32nd International Symposium on Reliable Distributed Systems.

[58]  David Maier,et al.  Blazes: Coordination analysis for distributed programs , 2013, 2014 IEEE 30th International Conference on Data Engineering.