Revisiting Concurrency in High-Performance NoSQL Databases

We measure the performance of five popular databases and show that single-node performance does not scale while hosting data on high-performance storage systems (e.g., Flash-based SSDs). We then analyze each system, unveiling techniques each system uses to increase concurrent performance; our taxonomy places said approaches into six different categories (thread architecture, batching, granularity, partitioning, scheduling and low-level efficiency) and thus points towards possible remedies that can scale the system. Finally, we introduce Xyza, a modified version of MongoDB that uses a wide range of classic and novel techniques to improve performance under concurrent, write-heavy workloads. Empirical analysis reveals that Xyza is 2× to 3× faster than MongoDB and scales well (up to 32 processing cores).

[1]  Goetz Graefe,et al.  Query processing techniques for solid state drives , 2009, SIGMOD Conference.

[2]  Alexander Thomasian,et al.  Concurrency control: methods, performance, and analysis , 1998, CSUR.

[3]  Eddie Kohler,et al.  Cache craftiness for fast multicore key-value storage , 2012, EuroSys '12.

[4]  Hyeontaek Lim,et al.  MICA: A Holistic Approach to Fast In-Memory Key-Value Storage , 2014, NSDI.

[5]  Philip A. Bernstein,et al.  Optimizing Optimistic Concurrency Control for Tree-Structured, Log-Structured Databases , 2015, SIGMOD Conference.

[6]  Philip A. Bernstein,et al.  Concurrency Control in Distributed Database Systems , 1986, CSUR.

[7]  Michael Stonebraker,et al.  The End of an Architectural Era (It's Time for a Complete Rewrite) , 2007, VLDB.

[8]  Eddie Kohler,et al.  Speedy transactions in multicore in-memory databases , 2013, SOSP.

[9]  Jin Li,et al.  FlashStore , 2010, Proc. VLDB Endow..

[10]  David J. DeWitt,et al.  Query processing on smart SSDs: opportunities and challenges , 2013, SIGMOD '13.

[11]  Irving L. Traiger,et al.  Granularity of Locks and Degrees of Consistency in a Shared Data Base , 1998, IFIP Working Conference on Modelling in Data Base Management Systems.

[12]  Shuai Wang,et al.  Oracle NoSQL Database , 2016 .

[13]  Erez Petrank,et al.  A lock-free B+tree , 2012, SPAA '12.

[14]  Mendel Rosenblum,et al.  Fast crash recovery in RAMCloud , 2011, SOSP.

[15]  Michael Stonebraker,et al.  An Evaluation of Distributed Concurrency Control , 2017, Proc. VLDB Endow..

[16]  Srinivas Devadas,et al.  TicToc: Time Traveling Optimistic Concurrency Control , 2016, SIGMOD Conference.

[17]  Remzi H. Arpaci-Dusseau.  Operating Systems: Three Easy Pieces , 2015, login Usenix Mag..

[18]  Vinay Devadas,et al.  To Waffinity and Beyond: A Scalable Architecture for Incremental Parallelization of File System Code , 2016, OSDI.

[19]  Amar Phanishayee,et al.  FAWN: a fast array of wimpy nodes , 2009, SOSP '09.

[20]  Jin Li,et al.  SkimpyStash: RAM space skimpy key-value store on flash-based storage , 2011, SIGMOD '11.

[21]  J. T. Robinson,et al.  On optimistic methods for concurrency control , 1979, TODS.

[22]  Nisha Talagala,et al.  NVMKV: A Scalable, Lightweight, FTL-aware Key-Value Store , 2015, USENIX Annual Technical Conference.

[23]  Tony Tung,et al.  Scaling Memcache at Facebook , 2013, NSDI.

[24]  Hao Jiang,et al.  Adaptive Concurrency Control: Despite the Looking Glass, One Concurrency Control Does Not Fit All , 2017, CIDR.

[25]  Bin Fan,et al.  SILT: a memory-efficient, high-performance key-value store , 2011, SOSP.

[26]  Andrea C. Arpaci-Dusseau,et al.  WiscKey: Separating Keys from Values in SSD-conscious Storage , 2016, FAST.

[27]  Jignesh M. Patel,et al.  High-Performance Concurrency Control Mechanisms for Main-Memory Databases , 2011, Proc. VLDB Endow..

[28]  Jim Gray,et al.  A critique of ANSI SQL isolation levels , 1995, SIGMOD '95.