Cicada: Dependably Fast Multi-Core In-Memory Transactions

Multi-core in-memory databases promise high-speed online transaction processing. However, the performance of individual designs suffers when the workload characteristics miss their small sweet spot of a desired contention level, read-write ratio, record size, processing rate, and so forth. Cicada is a single-node multi-core in-memory transactional database with serializability. To provide high performance under diverse workloads, Cicada reduces overhead and contention at several levels of the system by leveraging optimistic and multi-version concurrency control schemes and multiple loosely synchronized clocks while mitigating their drawbacks. On the TPC-C and YCSB benchmarks, Cicada outperforms Silo, TicToc, FOEDUS, MOCC, two-phase locking, Hekaton, and ERMIA in most scenarios, achieving up to 3X higher throughput than the next fastest design. It handles up to 2.07 M TPC-C transactions per second and 56.5 M YCSB transactions per second, and scans up to 356 M records per second on a single 28-core machine.

[1]  Tudor David,et al.  Everything you always wanted to know about synchronization but were afraid to ask , 2013, SOSP.

[2]  Alfons Kemper,et al.  Fast Serializable Multi-Version Concurrency Control for Main-Memory Database Systems , 2015, SIGMOD Conference.

[3]  Kian-Lee Tan,et al.  Scalable In-Memory Transaction Processing with HTM , 2016, USENIX Annual Technical Conference.

[4]  Eddie Kohler,et al.  Speedy transactions in multicore in-memory databases , 2013, SOSP.

[5]  Sudipta Sengupta,et al.  Multi-Version Range Concurrency Control in Deuteronomy , 2015, Proc. VLDB Endow..

[6]  Eddie Kohler,et al.  Cache craftiness for fast multicore key-value storage , 2012, EuroSys '12.

[7]  Craig Freedman,et al.  Hekaton: SQL server's memory-optimized OLTP engine , 2013, SIGMOD '13.

[8]  Christopher Frost,et al.  Spanner: Google's Globally-Distributed Database , 2012, OSDI.

[9]  J. T. Robinson,et al.  On optimistic methods for concurrency control , 1979, TODS.

[10]  Rachid Guerraoui,et al.  On the correctness of transactional memory , 2008, PPoPP.

[11]  Michael Stonebraker,et al.  Staring into the Abyss: An Evaluation of Concurrency Control with One Thousand Cores , 2014, Proc. VLDB Endow..

[12]  Nir Shavit,et al.  Software transactional memory , 1995, PODC '95.

[13]  Sudipta Sengupta,et al.  High Performance Transactions in Deuteronomy , 2015, CIDR.

[14]  Miguel Castro,et al.  No compromises: distributed transactions with consistency, availability, and performance , 2015, SOSP.

[15]  Maged M. Michael.  Safe memory reclamation for dynamic lock-free objects using atomic reads and writes , 2002, PODC '02.

[16]  Hideaki Kimura,et al.  FOEDUS: OLTP Engine for a Thousand Cores and NVRAM , 2015, SIGMOD Conference.

[17]  Keir Fraser,et al.  Practical lock-freedom , 2003 .

[18]  Andreas Reuter,et al.  Group Commit Timers and High Volume Transaction Systems , 1987, HPTS.

[19]  Eddie Kohler,et al.  Type-aware transactions for faster concurrent code , 2016, EuroSys.

[20]  Silas Boyd-Wickizer.  Optimizing communication bottlenecks in multiprocessor operating system kernels , 2014 .

[21]  Robert Gruber,et al.  Efficient optimistic concurrency control using loosely synchronized clocks , 1995, SIGMOD '95.

[22]  Michael Stonebraker,et al.  H-store: a high-performance, distributed main memory transaction processing system , 2008, Proc. VLDB Endow..

[23]  Timothy Roscoe,et al.  Arrakis , 2014, OSDI.

[24]  Alfons Kemper,et al.  HyPer: A hybrid OLTP&OLAP main memory database system based on virtual memory snapshots , 2011, 2011 IEEE 27th International Conference on Data Engineering.

[25]  Irving L. Traiger,et al.  The notions of consistency and predicate locks in a database system , 1976, CACM.

[26]  Kevin Wilkinson,et al.  Distributed Transaction Management in Jasmin , 1984, VLDB.

[27]  S. B. Yao,et al.  Efficient locking for concurrent operations on B-trees , 1981, TODS.

[28]  Michael J. Cahill.  Serializable isolation for snapshot databases , 2009, TODS.

[29]  Daniel J. Abadi,et al.  Rethinking serializable multiversion concurrency control , 2014, Proc. VLDB Endow..

[30]  Haibo Chen,et al.  Using restricted transactional memory to build a scalable in-memory database , 2014, EuroSys '14.

[31]  David Kenneth Gifford,et al.  Information storage in a decentralized computer system , 1981 .

[32]  Maurice Herlihy,et al.  Transactional Memory: Architectural Support For Lock-free Data Structures , 1993, Proceedings of the 20th Annual International Symposium on Computer Architecture.

[33]  Daniel J. Abadi,et al.  Low-Overhead Asynchronous Checkpointing in Main-Memory Database Systems , 2016, SIGMOD Conference.

[34]  Rajesh Gupta,et al.  From ARIES to MARS: transaction support for next-generation, solid-state drives , 2013, SOSP.

[35]  C.U. Orji,et al.  A performance analysis of an optimistic and a basic timestamp-ordering concurrency control algorithms for centralized database systems , 1988, Proceedings. Fourth International Conference on Data Engineering.

[36]  Adam Silberstein,et al.  Benchmarking cloud serving systems with YCSB , 2010, SoCC '10.

[37]  Barbara Liskov,et al.  Practical uses of synchronized clocks in distributed systems , 1991, PODC '91.

[38]  Dan R. K. Ports,et al.  Serializable Snapshot Isolation in PostgreSQL , 2012, Proc. VLDB Endow..

[39]  Arvind Krishnamurthy,et al.  Building consistent transactions with inconsistent replication , 2015, SOSP.

[40]  Jignesh M. Patel,et al.  High-Performance Concurrency Control Mechanisms for Main-Memory Databases , 2011, Proc. VLDB Endow..

[41]  Marvin Theimer,et al.  Flexible update propagation for weakly consistent replication , 1997, SOSP.

[42]  Srinivas Devadas,et al.  TicToc: Time Traveling Optimistic Concurrency Control , 2016, SIGMOD Conference.

[43]  Philip A. Bernstein,et al.  Concurrency Control in Distributed Database Systems , 1986, CSUR.

[44]  Christoforos E. Kozyrakis,et al.  IX: A Protected Dataplane Operating System for High Throughput and Low Latency , 2014, OSDI.

[45]  Eunyoung Jeong,et al.  mTCP: a Highly Scalable User-level TCP Stack for Multicore Systems , 2014, NSDI.

[46]  Ippokratis Pandis,et al.  ERMIA: Fast Memory-Optimized Database System for Heterogeneous Workloads , 2016, SIGMOD Conference.

[47]  Karsten Schwan,et al.  NVRAM-aware Logging in Transaction Systems , 2014, Proc. VLDB Endow..

[48]  Daniel J. Abadi,et al.  Design Principles for Scaling Multi-core OLTP Under High Contention , 2015, SIGMOD Conference.

[49]  Xiaoning Ding,et al.  BCC: Reducing False Aborts in Optimistic Concurrency Control with Low Cost for In-Memory Databases , 2016, Proc. VLDB Endow..

[50]  Michael Stonebraker,et al.  The Performance of Concurrency Control Algorithms for Database Management Systems , 1984, VLDB.

[51]  Cody Cutler,et al.  Phase Reconciliation for Contended In-Memory Transactions , 2014, OSDI.

[52]  Hideaki Kimura,et al.  Mostly-Optimistic Concurrency Control for Highly Contended Dynamic Workloads on a Thousand Cores , 2016, Proc. VLDB Endow..

[53]  Viktor Leis,et al.  Exploiting hardware transactional memory in main-memory databases , 2014, 2014 IEEE 30th International Conference on Data Engineering.

[54]  Haibo Chen,et al.  Fast In-Memory Transaction Processing Using RDMA and HTM , 2017, ACM Trans. Comput. Syst..

[55]  Eddie Kohler,et al.  Fast Databases with Fast Durability and Recovery Through Multicore Parallelism , 2014, OSDI.