Compressed Prefix Sums

We consider the prefix sumsproblem: given a (static) sequence of positive integers $\vec{x} = (x_1, \ldots, x_n)$, such that $\sum_{i=1}^n x_i = m$, we wish to support the operation ${\sf sum}(\vec{x},j)$, which returns $\sum_{i=1}^{j} x_i$. Our interest is in minimising the space required for storing $\vec{x}$, where `minimal space' is defined according to some compressibility criteria, while supporting sum as rapidly as possible. There are two main compressibility criteria: (a) the succinctspace bound, $B(m, n) = \lceil \log_2 {{m-1}\choose{n-1}} \rceil$ bits, applies to any sequence $\vec{x}$ whose elements add up to m; (b) data-awaremeasures, which depend on the values in $\vec{x}$, and can be lower than the succinct bound for some sequences. Appropriate data-aware measures have been studied extensively in the information retrieval (IR) community [17]. We demonstrate a close connection between the data-aware measure that is the best in practice for an important IR application and the succinct bound. We give theoretical solutions that use space close to other data-aware compressibility measures (often within o(n) bits), and support sum in doubly-logarithmic (or better) time, and experimental evaluations of practical variants thereof. A bit-vectoris a data structure that supports ` rank / select ' on a bit-string, and is fundamental to succinct and compressed data structures. We describe a new bit-vector that is robust and efficient.