Practical Implementations of Arithmetic Coding

We provide a tutorial on arithmetic coding, showing how it provides nearly optimal data compression and how it can be matched with almost any probabilistic model. We indicate the main disadvantage of arithmetic coding, its slowness, and give the basis of a fast, space-efficient, approximate arithmetic coder with only minimal loss of compression efficiency. Our coder is based on the replacement of arithmetic by table lookups coupled with a new deterministic probability estimation scheme.

[1]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[2]  D. Huffman.  A Method for the Construction of Minimum-Redundancy Codes , 1952 .

[3]  Norman Abramson,et al.  Information theory and coding , 1963 .

[4]  Peter Elias,et al.  Universal codeword sets and representations of the integers , 1975, IEEE Trans. Inf. Theory.

[5]  Richard Clark Pasco,et al.  Source coding algorithms for fast data compression , 1976 .

[6]  Jorma Rissanen,et al.  Generalized Kraft Inequality and Arithmetic Coding , 1976, IBM J. Res. Dev..

[7]  Abraham Lempel,et al.  A universal algorithm for sequential data compression , 1977, IEEE Trans. Inf. Theory.

[8]  Richard C. Pasco.  Source coding algorithms for fast data compression (Ph.D. Thesis abstr.) , 1977, IEEE Trans. Inf. Theory.

[9]  Abraham Lempel,et al.  Compression of individual sequences via variable-rate coding , 1978, IEEE Trans. Inf. Theory.

[10]  Robert G. Gallager,et al.  Variations on a theme by Huffman , 1978, IEEE Trans. Inf. Theory.

[11]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[12]  Glen G. Langdon,et al.  Arithmetic Coding , 1979 .

[13]  Frank Rubin,et al.  Arithmetic stream coding using fixed precision registers , 1979, IEEE Trans. Inf. Theory.

[14]  Mauro Guazzo,et al.  A general minimum-redundancy source-coding algorithm , 1980, IEEE Trans. Inf. Theory.

[15]  Glen G. Langdon,et al.  Universal modeling and coding , 1981, IEEE Trans. Inf. Theory.

[16]  Jorma Rissanen,et al.  Compression of Black-White Images with Arithmetic Coding , 1981, IEEE Trans. Commun..

[17]  Glen G. Langdon,et al.  A note on the Ziv-Lempel model for compressing individual sequences , 1983, IEEE Trans. Inf. Theory.

[18]  J. Rissanen.  A UNIVERSAL PRIOR FOR INTEGERS AND ESTIMATION BY MINIMUM DESCRIPTION LENGTH , 1983 .

[19]  R. Nigel Horspool,et al.  Algorithms for Adaptive Huffman Codes , 1984, Inf. Process. Lett..

[20]  Jorma Rissanen,et al.  Universal coding, information, prediction, and estimation , 1984, IEEE Trans. Inf. Theory.

[21]  Philippe Flajolet,et al.  Probabilistic Counting Algorithms for Data Base Applications , 1985, J. Comput. Syst. Sci..

[22]  Z. Galil,et al.  Combinatorial Algorithms on Words , 1985 .

[23]  Philippe Flajolet,et al.  Approximate counting: A detailed analysis , 1985, BIT.

[24]  Mark N. Wegman,et al.  Variations on a theme by Ziv and Lempel , 1985 .

[25]  Donald E. Knuth,et al.  Dynamic Huffman Coding , 1985, J. Algorithms.

[26]  Timothy Bell,et al.  A unifying theory and improvements for existing approaches to text compression , 1986 .

[27]  Frank Thomson Leighton,et al.  Estimating a probability using finite memory , 1986, IEEE Trans. Inf. Theory.

[28]  Robert E. Tarjan,et al.  A Locally Adaptive Data , 1986 .

[29]  Inder Jeet Taneja,et al.  Bounds on the redundancy of Huffman codes , 1986, IEEE Trans. Inf. Theory.

[30]  Peter Elias,et al.  Interval and recency rank source coding: Two on-line adaptive variable-length schemes , 1987, IEEE Trans. Inf. Theory.

[31]  R. Nigel Horspool,et al.  Data Compression Using Dynamic Markov Modelling , 1987, Comput. J..

[32]  Jeffrey Scott Vitter,et al.  Design and analysis of dynamic Huffman codes , 1987, JACM.

[33]  Ian H. Witten,et al.  Arithmetic coding for data compression , 1987, CACM.

[34]  Anselm Blumer,et al.  The Rényi redundancy of generalized Huffman codes , 1988, IEEE Trans. Inf. Theory.

[35]  Joan L. Mitchell,et al.  Optimal Hardware and Software Arithmetic Coding Procedures for the Q-Coder , 1988, IBM J. Res. Dev..

[36]  Ronald Arps,et al.  A Multi-Purpose VLSI Chip for Adaptive Data Compression of Bilevel Images , 1988, IBM J. Res. Dev..

[37]  Joan L. Mitchell,et al.  Probability Estimation for the Q-Coder , 1988, IBM J. Res. Dev..

[38]  Glen G. Langdon,et al.  An Overview of the Basic Principles of the Q-Coder Adaptive Binary Arithmetic Coder , 1988, IBM J. Res. Dev..

[39]  Joan L. Mitchell,et al.  Software Implementations of the Q-Coder , 1988, IBM J. Res. Dev..

[40]  Jorma Rissanen,et al.  A multiplication-free multialphabet arithmetic code , 1989, IEEE Trans. Commun..

[41]  Alistair Moffat,et al.  Word‐based text compression , 1989, Softw. Pract. Exp..

[42]  Timothy C. Bell,et al.  A Note on the DMC Data Compression Scheme , 1989, Computer/law journal.

[43]  Clark D. Thomborson,et al.  Enhancements to Ziv-Lempel data compression , 1989, [1989] Proceedings of the Thirteenth Annual International Computer Software & Applications Conference.

[44]  Jeffrey Scott Vitter,et al.  Algorithm 673: Dynamic Huffman coding , 1989, TOMS.

[45]  Ian H. Witten,et al.  Modeling for text compression , 1989, CSUR.

[46]  Alistair Moffat,et al.  Implementing the PPM data compression scheme , 1990, IEEE Trans. Commun..

[47]  Alistair Moffat,et al.  Linear time adaptive arithmetic coding , 1990, IEEE Trans. Inf. Theory.

[48]  Ian H. Witten,et al.  Text Compression , 1990, 125 Problems in Text Algorithms.

[49]  Ian H. Witten,et al.  The zero-frequency problem: Estimating the probabilities of novel events in adaptive text compression , 1991, IEEE Trans. Inf. Theory.

[50]  Ehud D. Karnin,et al.  High efficiency, multiplication free approximation of arithmetic coding , 1991, [1991] Proceedings. Data Compression Conference.

[51]  Glen G. Langdon,et al.  Probabilistic and Q-Coder algorithms for binary source adaptation , 1991, [1991] Proceedings. Data Compression Conference.

[52]  Daniel S. Hirschberg,et al.  Streamlining context models for data compression , 1991, [1991] Proceedings. Data Compression Conference.

[53]  Jeffrey Scott Vitter,et al.  New methods for lossless image compression using arithmetic coding , 1991, [1991] Proceedings. Data Compression Conference.

[54]  Jeffrey Scott Vitter,et al.  Analysis of arithmetic coding for data compression , 1991, [1991] Proceedings. Data Compression Conference.