QueCC: A Queue-oriented, Control-free Concurrency Architecture

We investigate a coordination-free approach to transaction processing on emerging multi-sockets, many-core, shared-memory architecture to harness its unprecedented available parallelism. We propose a queue-oriented, control-free concurrency architecture, referred to as QueCC, that exhibits minimal contention among concurrent threads by eliminating the overhead of concurrency control from the critical path of the transaction. QueCC operates on batches of transactions in two deterministic phases of priority-based planning followed by control-free execution. We extensively evaluate our transaction execution architecture and compare its performance against seven state-of-the-art concurrency control protocols designed for in-memory stores. We demonstrate that QueCC can significantly outperform state-of-the-art concurrency control protocols under high-contention by up to 6.3x. Moreover, our results show that QueCC can process nearly 40 million YCSB transactional operations per second while maintaining serializability guarantees with write-intensive workloads. Remarkably, QueCC out-performs H-Store by up to two orders of magnitude.

[1]  Ricardo Jiménez-Peris,et al.  Deterministic scheduling for transactional multithreaded replicas , 2000, Proceedings 19th IEEE Symposium on Reliable Distributed Systems SRDS-2000.

[2]  Gottfried Vossen,et al.  Transactional Information Systems: Theory, Algorithms, and the Practice of Concurrency Control and Recovery , 2002 .

[3]  Hyeontaek Lim,et al.  Cicada: Dependably Fast Multi-Core In-Memory Transactions , 2017, SIGMOD Conference.

[4]  Srinivas Devadas,et al.  TicToc: Time Traveling Optimistic Concurrency Control , 2016, SIGMOD Conference.

[5]  Mohammad Sadoghi,et al.  Blockchain Transaction Processing , 2021, Encyclopedia of Big Data Technologies.

[6]  Sébastien Monnet,et al.  Gargamel: Boosting DBMS Performance by Parallelising Write Transactions , 2012, 2012 IEEE 18th International Conference on Parallel and Distributed Systems.

[7]  Ippokratis Pandis,et al.  Data-oriented transaction execution , 2010, Proc. VLDB Endow..

[8]  Hideaki Kimura,et al.  FOEDUS: OLTP Engine for a Thousand Cores and NVRAM , 2015, SIGMOD Conference.

[9]  Arthur T. Whitney,et al.  High volume transaction processing without currency control, two phase commit, SQLor C++ , 1997 .

[10]  Michael Stonebraker,et al.  Implementation techniques for main memory database systems , 1984, SIGMOD '84.

[11]  Gang Chen,et al.  Exploiting Single-Threaded Model in Multi-Core In-Memory Systems , 2016, IEEE Transactions on Knowledge and Data Engineering.

[12]  Arthur J. Bernstein,et al.  Concurrency control for step-decomposed transactions , 1999, Inf. Syst..

[13]  Gustavo Alonso,et al.  Don't Be Lazy, Be Consistent: Postgres-R, A New Way to Implement Database Replication , 2000, VLDB.

[14]  Mohammad Sadoghi,et al.  Hybrid OLTP and OLAP , 2019, Encyclopedia of Big Data Technologies.

[15]  Daniel J. Abadi,et al.  Rethinking serializable multiversion concurrency control , 2014, Proc. VLDB Endow..

[16]  Daniel J. Abadi,et al.  CalvinFS: Consistent WAN Replication and Scalable Metadata Management for Distributed File Systems , 2015, FAST.

[17]  Michael Stonebraker,et al.  Staring into the Abyss: An Evaluation of Concurrency Control with One Thousand Cores , 2014, Proc. VLDB Endow..

[18]  Mustafa Canim,et al.  L-Store: A Real-time OLTP and OLAP System , 2016, EDBT.

[19]  Carlo Curino,et al.  Schism , 2010, Proc. VLDB Endow..

[20]  Michael Stonebraker,et al.  H-store: a high-performance, distributed main memory transaction processing system , 2008, Proc. VLDB Endow..

[21]  Mohammad Sadoghi,et al.  EasyCommit: A Non-blocking Two-phase Commit Protocol , 2018, EDBT.

[22]  Michael Stonebraker,et al.  An Evaluation of Distributed Concurrency Control , 2017, Proc. VLDB Endow..

[23]  Patrick Valduriez,et al.  Transaction chopping: algorithms and performance studies , 1995, TODS.

[24]  Xiaoning Ding,et al.  BCC: Reducing False Aborts in Optimistic Concurrency Control with Low Cost for In-Memory Databases , 2016, Proc. VLDB Endow..

[25]  Patrick E. O'Neil,et al.  Generalized isolation level definitions , 2000, Proceedings of 16th International Conference on Data Engineering (Cat. No.00CB37073).

[26]  Kenneth A. Ross,et al.  Reducing Database Locking Contention Through Multi-version Concurrency , 2014, Proc. VLDB Endow..

[27]  Daniel J. Abadi,et al.  Lazy evaluation of transactions in database systems , 2014, SIGMOD Conference.

[28]  Daniel J. Abadi,et al.  An Evaluation of the Advantages and Disadvantages of Deterministic Database Systems , 2014, Proc. VLDB Endow..

[29]  Daniel J. Abadi,et al.  Calvin: fast distributed transactions for partitioned database systems , 2012, SIGMOD Conference.

[30]  Ippokratis Pandis,et al.  ERMIA: Fast Memory-Optimized Database System for Heterogeneous Workloads , 2016, SIGMOD Conference.

[31]  Adam Silberstein,et al.  Benchmarking cloud serving systems with YCSB , 2010, SoCC '10.

[32]  Irving L. Traiger,et al.  The notions of consistency and predicate locks in a database system , 1976, CACM.

[33]  Alexander Thomasian,et al.  Concurrency control: methods, performance, and analysis , 1998, CSUR.

[34]  Gerhard Weikum,et al.  CHAPTER FOUR – Concurrency Control Algorithms , 2002 .

[35]  Marc H. Scholl,et al.  Transactional information systems: theory, algorithms, and the practice of concurrency control and recovery , 2001, SGMD.

[36]  J. T. Robinson,et al.  On optimistic methods for concurrency control , 1979, TODS.

[37]  Haibo Chen,et al.  Scaling Multicore Databases via Constrained Parallel Execution , 2016, SIGMOD Conference.

[38]  Eddie Kohler,et al.  Speedy transactions in multicore in-memory databases , 2013, SOSP.

[39]  Hideaki Kimura,et al.  Mostly-Optimistic Concurrency Control for Highly Contended Dynamic Workloads on a Thousand Cores , 2016, Proc. VLDB Endow..

[40]  Daniel J. Abadi,et al.  Design Principles for Scaling Multi-core OLTP Under High Contention , 2015, SIGMOD Conference.

[41]  Philip A. Bernstein,et al.  Concurrency Control in Distributed Database Systems , 1986, CSUR.

[42]  Daniel J. Abadi,et al.  High Performance Transactions via Early Write Visibility , 2017, Proc. VLDB Endow..

[43]  Jignesh M. Patel,et al.  High-Performance Concurrency Control Mechanisms for Main-Memory Databases , 2011, Proc. VLDB Endow..

[44]  Chilukuri K. Mohan Performance of Concurrency Control Mechanisms in Centralized Database Systems , 1995 .