A linear algorithm for optimal context clustering with application to bi-level image coding

The memory required to store the context model for a PPM-style compressor increases exponentially with the order of the model (i.e., length of context). It is a challenging research problem to find ways to reduce the memory requirement of a large context model without sacrificing its coding efficiency. In this paper, we focus on bi-level image coding and investigate context reduction by clustering: that is, contexts predicting similar probability distributions are grouped together to share a common entropy coder. We give an O(kn) algorithm for optimally grouping n contexts into k clusters so that the total loss in coding efficiency is minimized. Previously no algorithm was known for solving this problem. We demonstrate the effectiveness of clustering by implementing a two-level compression scheme. Experimental results on the CCITT test images show that, using the same amount of memory, our scheme achieves better compression than the two-level PPM method of A. Moffat (1991).