Geo-Replication: Fast If Possible, Consistent If Necessary

Geo-replicated storage systems are at the core of current Internet services. Unfortunately, there exists a fundamental tension between consistency and performance for offering scalable geo-replication. Weakening consistency semantics leads to less coordination and consequently a good user experience, but it may introduce anomalies such as state divergence and invariant violation. In contrast, maintaining stronger consistency precludes anomalies but requires more coordination. This paper discusses two main contributions to address this tension. First, RedBlue Consistency enables blue operations to be fast (and weakly consistent) while the remaining red operations are strongly consistent (and slow). We identify sufficient conditions for determining when operations can be blue or must be red. Second, Explicit Consistency further increases the space of operations that can be fast by restricting the concurrent execution of only the operations that can break application-defined invariants. We further show how to allow operations to complete locally in the common case, by relying on a reservation system that moves coordination off the critical path of operation execution.

[1]  Rodrigo Rodrigues,et al.  Extending Eventually Consistent Cloud Databases for Enforcing Numeric Invariants , 2015, 2015 IEEE 34th Symposium on Reliable Distributed Systems (SRDS).

[2]  João Leitão,et al.  Automating the Choice of Consistency Levels in Replicated Systems , 2014, USENIX Annual Technical Conference.

[3]  Patrick E. O'Neil,et al.  The Escrow transactional method , 1986, TODS.

[4]  Petr Kuznetsov,et al.  Zeno: Eventually Consistent Byzantine-Fault Tolerance , 2009, NSDI.

[5]  Marvin Theimer,et al.  Managing update conflicts in Bayou, a weakly connected replicated storage system , 1995, SOSP.

[6]  José Legatheaux Martins,et al.  Reservations for Conflict Avoidance in a Mobile Database System , 2003, MobiSys '03.

[7]  Marcos K. Aguilera,et al.  Transaction chains: achieving serializability with low latency in geo-distributed storage systems , 2013, SOSP.

[8]  Tim Kraska,et al.  MDCC: multi-data center consistency , 2012, EuroSys '13.

[9]  Werner Vogels,et al.  Dynamo: amazon's highly available key-value store , 2007, SOSP.

[10]  Michael J. Freedman,et al.  Stronger Semantics for Low-Latency Geo-Replicated Storage , 2013, NSDI.

[11]  Swaminathan Sivasubramanian,et al.  Amazon dynamoDB: a seamlessly scalable non-relational database service , 2012, SIGMOD Conference.

[12]  Sameh Elnikety,et al.  Orbe: scalable causal consistency using dependency matrices and physical clocks , 2013, SoCC.

[13]  Srinath T. V. Setty,et al.  Depot: Cloud Storage with Minimal Trust , 2010, TOCS.

[14]  Andrew C. Myers,et al.  Warranties for Faster Strong Consistency , 2014, NSDI.

[15]  Marcos K. Aguilera,et al.  Consistency-based service level agreements for cloud storage , 2013, SOSP.

[16]  Michael J. Freedman,et al.  Don't settle for eventual: scalable causal consistency for wide-area storage with COPS , 2011, SOSP.

[17]  Johannes Gehrke,et al.  The Homeostasis Protocol: Avoiding Transaction Coordination Through Program Analysis , 2014, SIGMOD Conference.

[18]  Hans-Arno Jacobsen,et al.  PNUTS: Yahoo!'s hosted data serving platform , 2008, Proc. VLDB Endow..

[19]  Hector Garcia-Molina,et al.  Using semantic knowledge for transaction processing in a distributed database , 1983, TODS.

[20]  Ali Ghodsi,et al.  Scalable atomic visibility with RAMP transactions , 2014, SIGMOD Conference.

[21]  Prashant Malik,et al.  Cassandra: a decentralized structured storage system , 2010, OPSR.

[22]  Panos K. Chrysanthis,et al.  Supporting semantics-based transaction processing in mobile database applications , 1995, Proceedings. 14th Symposium on Reliable Distributed Systems.

[23]  Hector Garcia-Molina,et al.  The demarcation protocol: A technique for maintaining constraints in distributed database systems , 1994, The VLDB Journal.

[24]  Liuba Shrira,et al.  Providing high availability using lazy replication , 1992, TOCS.

[25]  Marcos K. Aguilera,et al.  Transactional storage for geo-replicated systems , 2011, SOSP.

[26]  Cheng Li,et al.  Making geo-replicated systems fast as possible, consistent when necessary , 2012, OSDI 2012.

[27]  João Leitão,et al.  ChainReaction: a causal+ consistent datastore based on chain replication , 2013, EuroSys '13.

[28]  Divyakant Agrawal,et al.  Low-Latency Multi-Datacenter Databases using Replicated Commit , 2013, Proc. VLDB Endow..

[29]  Hongseok Yang,et al.  'Cause I'm strong enough: Reasoning about consistency choices in distributed systems , 2016, POPL.

[30]  Sérgio Duarte,et al.  Write Fast, Read in the Past: Causal Consistency for Client-Side Applications , 2015, Middleware.

[31]  Marc Shapiro,et al.  Conflict-Free Replicated Data Types , 2011, SSS.

[32]  Liuba Shrira,et al.  Exo-Leasing: Escrow Synchronization for Mobile Clients of Commodity Storage Servers , 2008, Middleware.

[33]  Ali Ghodsi,et al.  Bolt-on causal consistency , 2013, SIGMOD '13.

[34]  Sérgio Duarte,et al.  Putting consistency back into eventual consistency , 2015, EuroSys.

[35]  Christopher Frost,et al.  Spanner: Google's Globally-Distributed Database , 2012, OSDI.