Bolt-On Global Consistency for the Cloud

Web services that enable users in multiple regions to collaborate can increase availability and decrease latency by replicating data across data centers. If such a service spreads its data across multiple cloud providers---for the associated performance, cost, and reliability benefits---it cannot rely on cloud providers to keep the data globally consistent. Therefore, in this paper, we present an alternate approach to realizing global consistency in the cloud, which relies on cloud providers to only offer a strongly consistent storage service within each data center. A client library then accesses replicas stored in different data stores in a manner that preserves global consistency. To do so, our key contribution lies in rethinking the Paxos replication protocol to account for the limited interface offered by cloud storage. Compared to approaches not tailored for use in the cloud, our system CRIC can either halve median write latency or lower cost by up to 60%.

[1]  Jialin Li,et al.  Designing Distributed Systems Using Approximate Synchrony in Data Center Networks , 2015, NSDI.

[2]  Ali Ghodsi,et al.  Bolt-on causal consistency , 2013, SIGMOD '13.

[3]  I. Stoica,et al.  FairCloud: sharing the network in cloud computing , 2011, CCRV.

[4]  A. Rowstron,et al.  Towards predictable datacenter networks , 2011, SIGCOMM.

[5]  Yang Wang,et al.  Gnothi: Separating Data and Metadata for Efficient and Available Storage Replication , 2012, USENIX Annual Technical Conference.

[6]  Adam Silberstein,et al.  Benchmarking cloud serving systems with YCSB , 2010, SoCC '10.

[7]  Song Jiang,et al.  Workload analysis of a large-scale key-value store , 2012, SIGMETRICS '12.

[8]  Leslie Lamport,et al.  Fast Paxos , 2006, Distributed Computing.

[9]  Tanakorn Leesatapornwongsa,et al.  What Bugs Live in the Cloud? A Study of 3000+ Issues in Cloud Systems , 2014, SoCC.

[10]  Tony Tung,et al.  Scaling Memcache at Facebook , 2013, NSDI.

[11]  David G. Andersen,et al.  There is more consensus in Egalitarian parliaments , 2013, SOSP.

[12]  Marcos K. Aguilera,et al.  Transaction chains: achieving serializability with low latency in geo-distributed storage systems , 2013, SOSP.

[13]  Tim Kraska,et al.  MDCC: multi-data center consistency , 2012, EuroSys '13.

[14]  Christopher Frost,et al.  Spanner: Google's Globally-Distributed Database , 2012, OSDI.

[15]  Robbert van Renesse,et al.  Consus: Taming the Paxi , 2016, ArXiv.

[16]  Hui Ding,et al.  TAO: Facebook's Distributed Data Store for the Social Graph , 2013, USENIX Annual Technical Conference.

[17]  Anja Feldmann,et al.  C3: Cutting Tail Latency in Cloud Data Stores via Adaptive Replica Selection , 2015, NSDI.

[18]  Yawei Li,et al.  Megastore: Providing Scalable, Highly Available Storage for Interactive Services , 2011, CIDR.

[19]  Haryadi S. Gunawi,et al.  Why Does the Cloud Stop Computing?: Lessons from Hundreds of Service Outages , 2016, SoCC.

[20]  Ethan Katz-Bassett,et al.  SPANStore: cost-effective geo-replicated storage spanning multiple cloud services , 2013, SOSP.

[21]  Hitesh Ballani,et al.  End-to-end Performance Isolation Through Virtual Datacenters , 2014, OSDI.

[22]  Keith Marzullo,et al.  Classic Paxos vs. fast Paxos: caveat emptor , 2007 .

[23]  Michael J. Freedman,et al.  Don't settle for eventual: scalable causal consistency for wide-area storage with COPS , 2011, SOSP.

[24]  Ethan Katz-Bassett,et al.  CSPAN: cost-effective geo-replicated storage spanning multiple cloud services , 2013, SIGCOMM.

[25]  Marcos K. Aguilera,et al.  Consistency-based service level agreements for cloud storage , 2013, SOSP.

[26]  Dinan Gunawardena,et al.  Chatty Tenants and the Cloud Network Sharing Problem , 2013, NSDI.

[27]  Hakim Weatherspoon,et al.  The Xen-Blanket: virtualize once, run everywhere , 2012, EuroSys '12.

[28]  Cheng Huang,et al.  Giza: Erasure Coding Objects across Global Data Centers , 2017, USENIX Annual Technical Conference.

[29]  Cheng Li,et al.  Making geo-replicated systems fast as possible, consistent when necessary , 2012, OSDI 2012.

[30]  Arvind Krishnamurthy,et al.  Building consistent transactions with inconsistent replication , 2015, SOSP.

[31]  Leslie Lamport,et al.  Disk Paxos , 2003, Distributed Computing.

[32]  Srinath T. V. Setty,et al.  Realizing the Fault-Tolerance Promise of Cloud Storage Using Locks with Intent , 2016, OSDI.

[33]  Liang Zhu,et al.  The Need for End-to-End Evaluation of Cloud Availability , 2014, PAM.

[34]  Leslie Lamport,et al.  Paxos Made Simple , 2001 .

[35]  Robbert van Renesse,et al.  Supercloud: Opportunities and Challenges , 2015, OPSR.

[36]  M. Slee,et al.  Thrift : Scalable Cross-Language Services Implementation , 2022 .

[37]  Seungyeop Han,et al.  MetaSync: File Synchronization Across Multiple Untrusted Storage Services , 2015, USENIX Annual Technical Conference.

[38]  Michael J. Freedman,et al.  Stronger Semantics for Low-Latency Geo-Replicated Storage , 2013, NSDI.

[39]  Antony I. T. Rowstron,et al.  IOFlow: a software-defined storage architecture , 2013, SOSP.

[40]  D. Zats,et al.  DeTail: reducing the flow completion time tail in datacenter networks , 2012, CCRV.

[41]  Divyakant Agrawal,et al.  Minimizing Commit Latency of Transactions in Geo-Replicated Data Stores , 2015, SIGMOD Conference.

[42]  Sanjeev Kumar,et al.  Existential consistency: measuring and understanding consistency at Facebook , 2015, SOSP.

[43]  Ion Stoica,et al.  Failure as a Service (FaaS): A Cloud Service for Large- Scale, Online Failure Drills , 2011 .

[44]  Robert Griesemer,et al.  Paxos made live: an engineering perspective , 2007, PODC '07.

[45]  Jialin Li,et al.  Just Say NO to Paxos Overhead: Replacing Consensus with Network Ordering , 2016, OSDI.

[46]  Antony I. T. Rowstron,et al.  Better never than late: meeting deadlines in datacenter networks , 2011, SIGCOMM.

[47]  Brighten Godfrey,et al.  Finishing flows quickly with preemptive scheduling , 2012, CCRV.