Tunstall adaptive coding and miscoding

In the first part of this paper, we tackle the case where a variable length-to-block Tunstall code is used to encode the wrong source (miscoding). It turns out that, exactly as happens in the case with Huffman coding, the asymptotic excess rate is given by the informational divergence between the probability distribution ruling the source and the probability distribution for which the Tunstall code had been devised. We also prove asymptotic equality between the individual rate of a codeword and the corresponding self-information. This allows us to bound the maximal and the minimal length in a Tunstall tree. In the second part of the paper we study the case where the probability distribution of the source changes in time. If this happens, it is necessary to frequently update the current code, to ensure that the optimality conditions concerning its rate are met. This coding procedure is known as adaptive coding. We propose some schemes for adaptive Tunstall coding, based on the structure of the coding tree, on the "ordering property", analogous to Gallager's (1978) sibling property of the Huffman code, and on the "Tunstall regions", analogous to the attraction regions due to Longo and Galasso (1982).

[1]  Frederick Jelinek,et al.  On variable-length-to-block coding , 1972, IEEE Trans. Inf. Theory.

[2]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[3]  Gyula O. H. Katona,et al.  Huffman codes and self-information , 1976, IEEE Trans. Inf. Theory.

[4]  Frans M. J. Willems,et al.  A universal variable-to-fixed length source code based on Lawrence's algorithm , 1992, IEEE Trans. Inf. Theory.

[5]  Francesco Fabris,et al.  An application of the Hopfield model to Huffman codes , 1993, IEEE Trans. Inf. Theory.

[6]  H. N. Shapiro Introduction to the theory of numbers , 1983 .

[7]  Robert G. Gallager,et al.  Variations on a theme by Huffman , 1978, IEEE Trans. Inf. Theory.

[8]  Frederick Jelinek,et al.  Algorithms for Source Coding , 1975 .

[9]  Meir Feder,et al.  An adaptive universal variable-to-fix lossless coding scheme , 1994, Proceedings of 1994 IEEE International Symposium on Information Theory.

[10]  Frans M. J. Willems,et al.  Variable to fixed-length codes for Markov sources , 1987, IEEE Trans. Inf. Theory.

[11]  Francesco Fabris The Longo-Galasso Criterion for Huffman Codes: an Extended Version , 1989 .

[12]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[13]  Giuseppe Longo,et al.  An application of informational divergence to Huffman codes , 1982, IEEE Trans. Inf. Theory.

[14]  Daniel S. Hirschberg,et al.  Data compression , 1987, CSUR.