A General Framework for Dynamic Succinct and Compressed Data Structures

Succinct data structures are becoming increasingly popular in big data processing applications due to their low memory consumption. However, a feature that is currently lacking from most implementations of succinct data structures is dynamism. In this paper we design, implement, and test a general framework that allows for practical dynamic succinct structures. Firstly, a key component of our approach is careful memory management, which is often overlooked in the succinct data structures literature. Most succinct data structures allocate and deallocate relatively small data blocks each time a modify, insert, or delete operation occurs. We demonstrate experimentally that the space cost of neglecting memory management can be over 25% for dynamic data structures of this type. Secondly, using our memory management approach, we describe implementations of compressed modifiable bit vectors, and extended compressed random access memory (recently proposed by Jansson, Sadakane, and Sung [ICALP 2012]). Finally, we implement and test our data structures using several popular compression libraries, and both synthetic data (for the compressed modifiable bit vector) and a real-world temporal graph (for the extended compressed random access memory). Our data structures provide an easy to use interface that allow standard algorithms (in our example, breadth-first search in a graph) to be ∗Universitat des Saarlandes, Saarbrucken, Germany; Email: philologos14@gmail.com; Corresponding Author †Bell Labs, Dublin, Ireland; Email: pat.nicholson@alcatel-lucent.com run on top of the compressed data, decreasing memory consumption at the expense of running time.