Integer Compression in NVRAM-centric Data Stores: Comparative Experimental Analysis to DRAM

Lightweight integer compression algorithms play an important role in in-memory database systems to tackle the growing gap between processor speed and main memory bandwidth. Thus, there is a large number of algorithms to choose from, while different algorithms are tailored to different data characteristics. As we show in this paper, with the availability of byte-addressable non-volatile random-access memory (NVRAM), a novel type of main memory with specific characteristics increases the overall complexity in this domain. In particular, we provide a detailed evaluation of state-of-the-art lightweight integer compression schemes and database operations on NVRAM and compare it with DRAM. Furthermore, we reason about possible deployments of middle- and heavyweight approaches for better adaptation to NVRAM characteristics. Finally, we investigate a combined approach where both volatile and non-volatile memories are used in a cooperative fashion that is likely to be the case for hybrid and NVRAM-centric database systems.

[1]  Jignesh M. Patel,et al.  BitWeaving: fast scans for main memory data processing , 2013, SIGMOD '13.

[2]  Wolfgang Lehner,et al.  Efficient compute node-local replication mechanisms for NVRAM-centric data structures , 2018, DaMoN.

[3]  Roy H. Campbell,et al.  Consistent and Durable Data Structures for Non-Volatile Byte-Addressable Memory , 2011, FAST.

[4]  Fang Liu,et al.  WAlloc: An efficient wear-aware allocator for non-volatile main memory , 2015, 2015 IEEE 34th International Performance Computing and Communications Conference (IPCCC).

[5]  Hideaki Kimura,et al.  FOEDUS: OLTP Engine for a Thousand Cores and NVRAM , 2015, SIGMOD Conference.

[6]  David A. Huffman,et al.  A method for the construction of minimum-redundancy codes , 1952, Proceedings of the IRE.

[7]  Stratis Viglas,et al.  Write-limited sorts and joins for persistent memory , 2014, Proc. VLDB Endow..

[8]  Xiao Liu,et al.  Basic Performance Measurements of the Intel Optane DC Persistent Memory Module , 2019, ArXiv.

[9]  Mihnea Andrei,et al.  SAP HANA Adoption of Non-Volatile Memory , 2017, Proc. VLDB Endow..

[10]  Jian Xu,et al.  NOVA: A Log-structured File System for Hybrid Volatile/Non-volatile Main Memories , 2016, FAST.

[11]  Alfons Kemper,et al.  Data Blocks: Hybrid OLTP and OLAP on Compressed Storage using both Vectorization and Compilation , 2016, SIGMOD Conference.

[12]  Marcin Zukowski,et al.  MonetDB/X100: Hyper-Pipelining Query Execution , 2005, CIDR.

[13]  Carsten Binnig,et al.  Dictionary-based order-preserving string compression for main memory column stores , 2009, SIGMOD Conference.

[14]  Ismail Oukid,et al.  Instant Recovery for Main Memory Databases , 2015, CIDR.

[15]  Ismail Oukid,et al.  SOFORT: a hybrid SCM-DRAM storage engine for fast data recovery , 2014, DaMoN '14.

[16]  Wolfgang Lehner,et al.  A Benchmark Framework for Data Compression Techniques , 2015, TPCTC.

[17]  Wolfgang Lehner,et al.  From a Comprehensive Experimental Survey to a Cost-based Selection Strategy for Lightweight Integer Compression Algorithms , 2019, ACM Trans. Database Syst..

[18]  Wolfgang Lehner,et al.  Compression-Aware In-Memory Query Processing: Vision, System Design and Beyond , 2016, ADMS/IMDM@VLDB.

[19]  Ismail Oukid,et al.  Data Structure Engineering For Byte-Addressable Non-Volatile Memory , 2017, SIGMOD Conference.

[20]  Wolfgang Lehner,et al.  Lightweight Data Compression Algorithms: An Experimental Survey (Experiments and Analyses) , 2017, EDBT.

[21]  Qin Jin,et al.  Persistent B+-Trees in Non-Volatile Main Memory , 2015, Proc. VLDB Endow..

[22]  Hans-Juergen Boehm,et al.  Makalu: fast recoverable allocation of non-volatile memory , 2016, OOPSLA.

[23]  Leonid Boytsov,et al.  Decoding billions of integers per second through vectorization , 2012, Softw. Pract. Exp..

[24]  Andy Rudoff,et al.  Persistent Memory Programming , 2017, login Usenix Mag..

[25]  Bingsheng He,et al.  NV-Tree: A Consistent and Workload-Adaptive Tree Structure for Non-Volatile Memory , 2016, IEEE Transactions on Computers.

[26]  Andrew Pavlo,et al.  Write-Behind Logging , 2016, Proc. VLDB Endow..

[27]  Ian H. Witten,et al.  Arithmetic coding for data compression , 1987, CACM.

[28]  Abraham Lempel,et al.  A universal algorithm for sequential data compression , 1977, IEEE Trans. Inf. Theory.

[29]  Suman Nath,et al.  Rethinking Database Algorithms for Phase Change Memory , 2011, CIDR.

[30]  Parthasarathy Ranganathan,et al.  Consistent, durable, and safe memory management for byte-addressable non volatile main memory , 2013, TRIOS@SOSP.

[31]  Mark A. Roth,et al.  Database compression , 1993, SGMD.

[32]  Hasso Plattner,et al.  nvm malloc: Memory Allocation for NVRAM , 2015, ADMS@VLDB.

[33]  Jonathan Goldstein,et al.  Compressing relations and indexes , 1998, Proceedings 14th International Conference on Data Engineering.

[34]  Wolfgang Lehner,et al.  MorphStore - In-Memory Query Processing based on Morphing Compressed Intermediates LIVE , 2019, SIGMOD Conference.

[35]  Yoshiyasu Doi,et al.  Managing Non-Volatile Memory in Database Systems , 2018, SIGMOD Conference.

[36]  Daniel J. Abadi,et al.  Integrating compression and execution in column-oriented database systems , 2006, SIGMOD Conference.

[37]  Ismail Oukid,et al.  FPTree: A Hybrid SCM-DRAM Persistent and Concurrent B-Tree for Storage Class Memory , 2016, SIGMOD Conference.