Speeding Up Queries in Column Stores - A Case for Compression

BI accelerator solutions like the SAP NetWeaver database engine TREX achieve high performance when processing complex analytic queries in large data warehouses. They do so with a combination of column-oriented data organization, memory-based processing, and a scalable multiserver architecture. The use of data compression techniques further reduces both memory consumption and processing time. In this paper we study query operators like scan and aggregation on compressed data structures implemented in TREX.

[1]  Kenneth A. Ross,et al.  Implementing database operations using SIMD instructions , 2002, SIGMOD '02.

[2]  Norman Winterbottom,et al.  Performing joins without decompression in a compressed database system , 2003, SGMD.

[3]  Michael Stonebraker,et al.  C-Store: A Column-oriented DBMS , 2005, VLDB.

[4]  W. Paul Cockshott,et al.  High-Performance Operations Using a Compressed Database Architecture , 1998, Comput. J..

[5]  Marcin Zukowski,et al.  Super-Scalar RAM-CPU Cache Compression , 2006, 22nd International Conference on Data Engineering (ICDE'06).

[6]  Daniel J. Abadi,et al.  Integrating compression and execution in column-oriented database systems , 2006, SIGMOD Conference.

[7]  Frederick Reiss,et al.  Constant-Time Query Processing , 2008, 2008 IEEE 24th International Conference on Data Engineering.

[8]  Alexander Zeier,et al.  SIMD-Scan: Ultra Fast in-Memory Table Scan using on-Chip Vector Processing Units , 2009, Proc. VLDB Endow..

[9]  Sven Helmer,et al.  The implementation and performance of compressed databases , 2000, SGMD.

[10]  Johannes Gehrke,et al.  Query optimization in compressed database systems , 2001, SIGMOD '01.

[11]  Jaideep Srivastava,et al.  Efficient Aggregation Algorithms for Compressed Data Warehouses , 2002, IEEE Trans. Knowl. Data Eng..

[12]  Garret Swart,et al.  How to wring a table dry: entropy compression of relations and querying of compressed relations , 2006, VLDB.

[13]  Goetz Graefe,et al.  Data compression and database performance , 1991, [Proceedings] 1991 Symposium on Applied Computing.