Coding of sources with two-sided geometric distributions and unknown parameters

Lossless compression is studied for a countably infinite alphabet source with an unknown, off-centered, two-sided geometric (TSG) distribution, which is a commonly used statistical model for image prediction residuals. We demonstrate that arithmetic coding based on a simple strategy of model adaptation, essentially attains the theoretical lower bound to the universal coding redundancy associated with this model. We then focus on more practical codes for the TSG model, that operate on a symbol-by-symbol basis, and study the problem of adaptively selecting a code from a given discrete family. By taking advantage of the structure of the optimum Huffman tree for a known TSG distribution, which enables simple calculation of the codeword of every given source symbol, an efficient adaptive strategy is derived.

[1]  J. Rissanen Stochastic Complexity and Modeling , 1986 .

[2]  A.N. Netravali,et al.  Picture coding: A review , 1980, Proceedings of the IEEE.

[3]  Neri Merhav,et al.  Optimal prefix codes for sources with two-sided geometric distributions , 2000, IEEE Trans. Inf. Theory.

[4]  Robert F. Rice,et al.  Some practical universal noiseless coding techniques , 1979 .

[5]  Neri Merhav,et al.  A strong version of the redundancy-capacity theorem of universal coding , 1995, IEEE Trans. Inf. Theory.

[6]  Nasir D. Memon,et al.  Context-based, adaptive, lossless image coding , 1997, IEEE Trans. Commun..

[7]  Jorma Rissanen,et al.  Universal coding, information, prediction, and estimation , 1984, IEEE Trans. Inf. Theory.

[8]  Jorma Rissanen,et al.  Complexity of strings in the class of Markov sources , 1986, IEEE Trans. Inf. Theory.

[9]  Guillermo Sapiro,et al.  LOCO-I: a low complexity, context-based, lossless image compression algorithm , 1996, Proceedings of Data Compression Conference - DCC '96.

[10]  Solomon W. Golomb,et al.  Run-length encodings (Corresp.) , 1966, IEEE Trans. Inf. Theory.

[11]  Stephen Todd,et al.  Parameter Reduction and Context Selection for Compression of Gray-Scale Images , 1985, IBM J. Res. Dev..

[12]  Lee D. Davisson,et al.  Universal noiseless coding , 1973, IEEE Trans. Inf. Theory.

[13]  R. F. Rice,et al.  Some practical universal noiseless coding techniques, part 2 , 1983 .

[14]  Meir Feder,et al.  A universal finite memory source , 1995, IEEE Trans. Inf. Theory.

[15]  JORMA RISSANEN,et al.  A universal data compression system , 1983, IEEE Trans. Inf. Theory.

[16]  Neri Merhav,et al.  Optimal prefix codes for two-sided geometric distributions , 1997, Proceedings of IEEE International Symposium on Information Theory.

[17]  Gadiel Seroussi,et al.  On adaptive strategies for an extended family of Golomb-type codes , 1997, Proceedings DCC '97. Data Compression Conference.

[18]  J. B. O'Neal,et al.  Predictive quantizing systems (differential pulse code modulation) for the transmission of television signals , 1966 .

[19]  Donald E. Knuth,et al.  Dynamic Huffman Coding , 1985, J. Algorithms.

[20]  Jorma Rissanen,et al.  Applications of universal context modeling to lossless compression of gray-scale images , 1995, Conference Record of The Twenty-Ninth Asilomar Conference on Signals, Systems and Computers.

[21]  Raphail E. Krichevsky,et al.  The performance of universal encoding , 1981, IEEE Trans. Inf. Theory.

[22]  Padhraic Smyth,et al.  A High-Speed Distortionless Predictive Image-Compression Scheme , 1990 .

[23]  Neri Merhav,et al.  Optimal sequential probability assignment for individual sequences , 1994, IEEE Trans. Inf. Theory.

[24]  P.G. Howard,et al.  Fast and efficient lossless image compression , 1993, [Proceedings] DCC `93: Data Compression Conference.

[25]  David C. van Voorhis,et al.  Optimal source codes for geometrically distributed integer alphabets (Corresp.) , 1975, IEEE Trans. Inf. Theory.

[26]  S. Golomb Run-length encodings. , 1966 .

[27]  Guillermo Sapiro,et al.  The LOCO-I lossless image compression algorithm: principles and standardization into JPEG-LS , 2000, IEEE Trans. Image Process..