HyPer beyond software: Exploiting modern hardware for main-memory database systems

In this presentation, we survey the use of advanced hard-ware features for optimizing main-memory database systems in the context of our HyPer project. The access behavior of database objects from simultaneous OLTP transactions is monitored using the virtual memory management component in order to compact the database into hot and cold partitions. The cold partitions are stored in compressed data blocks. Decompression is expedited by vectorized SIMD scan processing. Utilizing many-core NUMA-organized database servers is facilitated by the morsel-driven adaptive parallelization and partitioning that guarantees data locality w.r.t. the processing core. The most recent Hardware Transactional Memory support of, e.g., Intel's Haswell processor, can be used as the basis for a lock-free concurrency control scheme for OLTP transactions. Finally, we show how heterogeneous processors of “wimpy” devices such as tablets can be utilized for high-performance and energy-efficient query processing. This is joint work with my colleague Thomas Neumann at TUM and the HyPer team (hyper-db.com).

[1]  Alfons Kemper,et al.  Heterogeneity-conscious parallel query execution: getting a better mileage while driving faster! , 2014, DaMoN '14.

[2]  Michael J. Carey.  Modeling and Evaluation of Database Concurrency Control Algorithms , 1983 .

[3]  Alfons Kemper,et al.  Compacting Transactional Data in Hybrid OLTP & OLAP Databases , 2012, Proc. VLDB Endow..

[4]  Viktor Leis,et al.  Morsel-driven parallelism: a NUMA-aware query evaluation framework for the many-core age , 2014, SIGMOD Conference.

[5]  Alfons Kemper,et al.  HyPer: A hybrid OLTP&OLAP main memory database system based on virtual memory snapshots , 2011, 2011 IEEE 27th International Conference on Data Engineering.

[6]  Thomas Neumann,et al.  Efficiently Compiling Efficient Query Plans for Modern Hardware , 2011, Proc. VLDB Endow..

[7]  Babak Falsafi,et al.  Toward Dark Silicon in Servers , 2011, IEEE Micro.

[8]  Daniel Schall,et al.  Energy-proportional query execution using a cluster of wimpy nodes , 2013, DaMoN '13.

[9]  Maurice Herlihy,et al.  Transactional Memory: Architectural Support For Lock-free Data Structures , 1993, Proceedings of the 20th Annual International Symposium on Computer Architecture.

[10]  Konrad K. Lai,et al.  The Impact of Performance Asymmetry in Emerging Multicore Architectures , 2005, ISCA 2005.

[11]  Wolfgang Lehner,et al.  SAP HANA database: data management for modern business applications , 2012, SGMD.

[12]  Ippokratis Pandis,et al.  NUMA-aware algorithms: the case of data shuffling , 2013, CIDR.

[13]  Alfons Kemper,et al.  One DBMS for all: the brawny few and the wimpy crowd , 2014, SIGMOD Conference.

[14]  Karthikeyan Sankaralingam,et al.  Dark silicon and the end of multicore scaling , 2011, 2011 38th Annual International Symposium on Computer Architecture (ISCA).

[15]  Lieven Eeckhout,et al.  Understanding fundamental design choices in single-ISA heterogeneous multicore architectures , 2013, TACO.

[16]  Mehul A. Shah,et al.  Analyzing the energy efficiency of a database server , 2010, SIGMOD Conference.

[17]  Viktor Leis,et al.  The adaptive radix tree: ARTful indexing for main-memory databases , 2013, 2013 IEEE 29th International Conference on Data Engineering (ICDE).

[18]  Viktor Leis,et al.  Exploiting hardware transactional memory in main-memory databases , 2014, 2014 IEEE 30th International Conference on Data Engineering.

[19]  Michael Stonebraker,et al.  OLTP through the looking glass, and what we found there , 2008, SIGMOD Conference.

[20]  Gustavo Alonso,et al.  Multi-Core, Main-Memory Joins: Sort vs. Hash Revisited , 2013, Proc. VLDB Endow..

[21]  Gustavo Alonso,et al.  SWissBox: An Architecture for Data Processing Appliances , 2011, CIDR.

[22]  Alfons Kemper,et al.  Massively Parallel Sort-Merge Joins in Main Memory Multi-Core Database Systems , 2012, Proc. VLDB Endow..