Data compression in database systems

This paper addresses the question of how information-theoretically-derived compact representations can be applied in practice to improve storage and processing efficiency in DBMS. Compact data representation has the potential for savings in storage, access and processing costs throughout the systems architecture and may alter the balance of usage between disk and solid state storage. To realise the potential performance benefits, however, novel systems engineering must be adopted to ensure that compression/decompression overheads are limited. This paper describes a basic approach to storage and processing of relations in a highly compressed form. A vertical columnwise representation is adopted in which columns can dynamically vary incrementally in both length and width. To achieve good performance query processing is carried out directly on the compressed relational representation (using a compressed representation of the query), thus avoiding decompression overheads. Measurements of performance of the Hi-base prototype implementation are compared with those obtained from conventional DBMS.