Hyrise-NV: Instant Recovery for In-Memory Databases Using Non-Volatile Memory

Emerging non-volatile memory technologies NVM offer fast and byte-addressable access, allowing to rethink the durability mechanisms of in-memory databases. In this paper, we present Hyrise-NV, a database storage engine that maintains table and index structures on NVM. Our architecture updates the database state and index structures transactionally consistent on NVM using multi-version data structures, allowing to instantly recover databases independent of their size. For index structures, we present nvBTree using multi-versioning to provide failure-atomic tree updates on NVM. We evaluate Hyrise-NV both on DRAM and with hardware-based emulation of NVM using the TPC-C benchmark. Hyrise-NV recovers databases independent of their size, allowing the recovery of a table with 10 million rows in less than 100i¾źms.

[1]  Satish M. Thatte.  Persistent Memory: A Storage System for Object-Oriented Databases , 1991, On Object-Oriented Database System.

[2]  Rajesh K. Gupta,et al.  NV-Heaps: making persistent objects fast and safe with next-generation, non-volatile memories , 2011, ASPLOS XVI.

[3]  Joos-Hendrik Böse,et al.  Efficient logging for enterprise workloads on column-oriented in-memory databases , 2012, CIKM '12.

[4]  Mendel Rosenblum,et al.  Fast crash recovery in RAMCloud , 2011, SOSP.

[5]  Satish M. Thatte,et al.  Persistent Memory: A Storage Architecture for Object-Oriented Database Systems , 1986, OODBS.

[6]  Hasso Plattner,et al.  The Impact of Columnar In-Memory Databases on Enterprise Systems , 2014, Proc. VLDB Endow..

[7]  Ryan Johnson,et al.  Scalable Logging through Emerging Non-Volatile Memory , 2014, Proc. VLDB Endow..

[8]  Chao Wang,et al.  NVMalloc: Exposing an Aggregate SSD Store as a Memory Partition in Extreme-Scale Machines , 2012, 2012 IEEE 26th International Parallel and Distributed Processing Symposium.

[9]  Martin Grund,et al.  Efficient Transaction Processing for Hyrise in Mixed Workload Environments , 2014, IMDM@VLDB.

[10]  Michael Stonebraker,et al.  C-Store: A Column-oriented DBMS , 2005, VLDB.

[11]  Michael Stonebraker,et al.  Rethinking main memory OLTP recovery , 2014, 2014 IEEE 30th International Conference on Data Engineering.

[12]  Christopher Frost,et al.  Better I/O through byte-addressable, persistent memory , 2009, SOSP '09.

[13]  Michael Stonebraker,et al.  A Prolegomenon on OLTP Database Systems for Non-Volatile Memory , 2014, ADMS@VLDB.

[14]  Bernhard Höppner,et al.  An Approach for Hybrid-Memory Scaling Columnar In-Memory Databases , 2014, ADMS@VLDB.

[15]  Jian Yang,et al.  Mojim: A Reliable and Highly-Available Non-Volatile Memory System , 2015, ASPLOS.

[16]  Ismail Oukid,et al.  Instant Recovery for Main Memory Databases , 2015, CIDR.

[17]  C. Mohan,et al.  High performance database logging using storage class memory , 2011, 2011 IEEE 27th International Conference on Data Engineering.

[18]  Qin Jin,et al.  Persistent B+-Trees in Non-Volatile Main Memory , 2015, Proc. VLDB Endow..

[19]  Alfons Kemper,et al.  HyPer: A hybrid OLTP&OLAP main memory database system based on virtual memory snapshots , 2011, 2011 IEEE 27th International Conference on Data Engineering.

[20]  Hamid Pirahesh,et al.  ARIES: a transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging , 1998 .

[21]  Hasso Plattner,et al.  Fast Lookups for In-Memory Column Stores: Group-Key Indices, Lookup and Maintenance , 2012, ADMS@VLDB.

[22]  Alexander Zeier,et al.  HYRISE - A Main Memory Hybrid Storage Engine , 2010, Proc. VLDB Endow..

[23]  Pradeep Dubey,et al.  Fast Updates on Read-Optimized Databases Using Multi-Core CPUs , 2011, Proc. VLDB Endow..

[24]  Michael Stonebraker,et al.  H-store: a high-performance, distributed main memory transaction processing system , 2008, Proc. VLDB Endow..

[25]  Hasso Plattner,et al.  nvm malloc: Memory Allocation for NVRAM , 2015, ADMS@VLDB.

[26]  Sam Lightstone,et al.  DB2 with BLU Acceleration: So Much More than Just a Column Store , 2013, Proc. VLDB Endow..

[27]  Roy H. Campbell,et al.  Consistent and Durable Data Structures for Non-Volatile Byte-Addressable Memory , 2011, FAST.

[28]  Sanjay Kumar,et al.  System software for persistent memory , 2014, EuroSys '14.

[29]  Michael Wu,et al.  eNVy: a non-volatile, main memory storage system , 1994, ASPLOS VI.

[30]  Stratis Viglas,et al.  REWIND: Recovery Write-Ahead System for In-Memory Non-Volatile Data-Structures , 2015, Proc. VLDB Endow..

[31]  Orion Hodson,et al.  Whole-system persistence , 2012, ASPLOS XVII.

[32]  Suman Nath,et al.  Rethinking Database Algorithms for Phase Change Memory , 2011, CIDR.

[33]  Thomas F. Wenisch,et al.  Storage Management in the NVRAM Era , 2013, Proc. VLDB Endow..

[34]  Hasso Plattner,et al.  NVC-Hashmap: A Persistent and Concurrent Hashmap For Non-Volatile Memories , 2015, IMDM@VLDB.

[35]  Michael M. Swift,et al.  Mnemosyne: lightweight persistent memory , 2011, ASPLOS XVI.

[36]  Bingsheng He,et al.  NV-Tree: Reducing Consistency Cost for NVM-based Single Level Systems , 2015, FAST.