DynaMast: Adaptive Dynamic Mastering for Replicated Systems

Single-master replicated database systems strive to be scalable by offloading reads to replica nodes. However, single-master systems suffer from the performance bottleneck of all updates executing at a single site. Multi-master replicated systems distribute updates among sites but incur costly coordination for multi-site transactions. We present DynaMast, a lazily replicated, multi-master database system that guarantees one-site transaction execution while effectively distributing both reads and updates among multiple sites. DynaMast benefits from these advantages by dynamically transferring the mastership of data, or remastering, among sites using a lightweight metadata-based protocol. DynaMast leverages remastering to adaptively place master copies to balance load and minimize future remastering. Using benchmark workloads, we demonstrate that DynaMast delivers superior performance over existing replicated database system architectures.

[1]  Masaru Kitsuregawa,et al.  Speculative locking protocols to improve performance for distributed database systems , 2004, IEEE Transactions on Knowledge and Data Engineering.

[2]  Jim Gray,et al.  A critique of ANSI SQL isolation levels , 1995, SIGMOD '95.

[3]  Jialin Li,et al.  Just Say NO to Paxos Overhead: Replacing Consensus with Network Ordering , 2016, OSDI.

[4]  Srinivas Devadas,et al.  Sundial: Harmonizing Concurrency Control and Caching in a Distributed OLTP Database Management System , 2018, Proc. VLDB Endow..

[5]  Carlo Curino,et al.  OLTP-Bench: An Extensible Testbed for Benchmarking Relational Databases , 2013, Proc. VLDB Endow..

[6]  Jeffrey F. Naughton,et al.  Middle-tier database caching for e-business , 2002, SIGMOD '02.

[7]  Divyakant Agrawal,et al.  G-Store: a scalable data store for transactional multi key access in the cloud , 2010, SoCC '10.

[8]  Samuel Madden,et al.  MacroBase: Prioritizing Attention in Fast Data , 2016, SIGMOD Conference.

[9]  Carsten Binnig,et al.  The End of a Myth: Distributed Transaction Can Scale , 2016, Proc. VLDB Endow..

[10]  Carlo Curino,et al.  Skew-aware automatic database partitioning in shared-nothing, parallel OLTP systems , 2012, SIGMOD Conference.

[11]  Jayant R. Haritsa,et al.  Revisiting commit processing in distributed database systems , 1997, SIGMOD '97.

[12]  Jay Kreps,et al.  Kafka : a Distributed Messaging System for Log Processing , 2011 .

[13]  Michael Stonebraker,et al.  Clay: Fine-Grained Adaptive Partitioning for General Database Schemas , 2016, Proc. VLDB Endow..

[14]  Sameh Elnikety,et al.  One-copy serializability with snapshot isolation under the hood , 2011, 2011 IEEE 27th International Conference on Data Engineering.

[15]  Jinyang Li,et al.  Consolidating Concurrency Control and Consensus for Commits under Conflicts , 2016, OSDI.

[16]  Samuel Madden,et al.  A robust partitioning scheme for ad-hoc query workloads , 2017, SoCC.

[17]  Abraham Silberschatz,et al.  Database System Concepts , 1980 .

[18]  Michael Stonebraker,et al.  E-Store: Fine-Grained Elastic Partitioning for Distributed Transaction Processing , 2014, Proc. VLDB Endow..

[19]  Cheng Li,et al.  Making geo-replicated systems fast as possible, consistent when necessary , 2012, OSDI 2012.

[20]  Arvind Krishnamurthy,et al.  Building consistent transactions with inconsistent replication , 2015, SOSP.

[21]  Divyakant Agrawal,et al.  Squall: Fine-Grained Live Reconfiguration for Partitioned Main Memory Databases , 2015, SIGMOD Conference.

[22]  Anurag Gupta,et al.  Amazon Aurora: Design Considerations for High Throughput Cloud-Native Relational Databases , 2017, SIGMOD Conference.

[23]  Kun Ren SLOG : Serializable , Low-latency , Geo-replicated Transactions , 2019 .

[24]  Jignesh M. Patel,et al.  High-Performance Concurrency Control Mechanisms for Main-Memory Databases , 2011, Proc. VLDB Endow..

[25]  M. Slee,et al.  Thrift : Scalable Cross-Language Services Implementation , 2022 .

[26]  Jonathan Goldstein,et al.  MTCache: Mid-Tier Database Caching for SQL Server , 2004, IEEE Data Eng. Bull..

[27]  Ippokratis Pandis,et al.  Query Fresh: Log Shipping on Steroids , 2017, Proc. VLDB Endow..

[28]  Yi Lu,et al.  STAR : Scaling Transactions through Asymmetrical Replication , 2018 .

[29]  Sameh Elnikety,et al.  Strongly consistent replication for a bargain , 2010, 2010 IEEE 26th International Conference on Data Engineering (ICDE 2010).

[30]  Daniel J. Abadi,et al.  Calvin: fast distributed transactions for partitioned database systems , 2012, SIGMOD Conference.

[31]  Marcos K. Aguilera,et al.  Transactional storage for geo-replicated systems , 2011, SOSP.

[32]  Douglas B. Terry,et al.  A Self-Configurable Geo-Replicated Cloud Storage System , 2014, OSDI.

[33]  Divyakant Agrawal,et al.  MaaT: Effective and scalable coordination of distributed transactions in the cloud , 2014, Proc. VLDB Endow..

[34]  Craig Freedman,et al.  Hekaton: SQL server's memory-optimized OLTP engine , 2013, SIGMOD '13.

[35]  Miguel Castro,et al.  No compromises: distributed transactions with consistency, availability, and performance , 2015, SOSP.

[36]  Hui Ding,et al.  TAO: Facebook's Distributed Data Store for the Social Graph , 2013, USENIX Annual Technical Conference.

[37]  Daniel J. Abadi,et al.  Low overhead concurrency control for partitioned main memory databases , 2010, SIGMOD Conference.

[38]  Donald Kossmann,et al.  On the Design and Scalability of Distributed Shared-Data Databases , 2015, SIGMOD Conference.

[39]  Michael Stonebraker,et al.  An Evaluation of Distributed Concurrency Control , 2017, Proc. VLDB Endow..

[40]  Bettina Kemme,et al.  Postgres-R(SI): combining replica control with concurrency control based on snapshot isolation , 2005, 21st International Conference on Data Engineering (ICDE'05).

[41]  Ashraf Aboulnaga,et al.  Accordion: Elastic Scalability for Database Systems Supporting Distributed Transactions , 2014, Proc. VLDB Endow..

[42]  Norman May,et al.  Distributed snapshot isolation: global transactions pay globally, local transactions pay locally , 2014, The VLDB Journal.

[43]  Michael Stonebraker,et al.  The End of an Architectural Era (It's Time for a Complete Rewrite) , 2007, VLDB.

[44]  Juan Loaiza,et al.  Distributed Architecture of Oracle Database In-memory , 2015, Proc. VLDB Endow..

[45]  Hans-Arno Jacobsen,et al.  PNUTS: Yahoo!'s hosted data serving platform , 2008, Proc. VLDB Endow..

[46]  M. Tamer Özsu,et al.  ConfluxDB: Multi-Master Replication for Partitioned Snapshot Isolation Databases , 2014, Proc. VLDB Endow..

[47]  Alfons Kemper,et al.  HyPer: A hybrid OLTP&OLAP main memory database system based on virtual memory snapshots , 2011, 2011 IEEE 27th International Conference on Data Engineering.

[48]  Kenneth Salem,et al.  Lazy database replication with snapshot isolation , 2006, VLDB.

[49]  Gang Chen,et al.  Towards a Non-2PC Transaction Management in Distributed Database Systems , 2016, SIGMOD Conference.

[50]  Philip A. Bernstein,et al.  Hyder - A Transactional Record Manager for Shared Flash , 2011, CIDR.

[51]  Carlo Curino,et al.  Schism , 2010, Proc. VLDB Endow..

[52]  Tilmann Rabl,et al.  Query Centric Partitioning and Allocation for Partially Replicated Database Systems , 2017, SIGMOD Conference.

[53]  Barbara Liskov,et al.  Granola: Low-Overhead Distributed Transaction Coordination , 2012, USENIX Annual Technical Conference.

[54]  Yi Lu,et al.  STAR: Scaling Transactions through Asymmetric Replication , 2018, Proc. VLDB Endow..