Performance of universal codes over infinite alphabets

It was known that universal compression of strings generated by independent and identically distributed sources over infinite alphabets entails infinite per-symbol redundancy. Alternative compression schemes, which decompose the description of such strings into a description of the symbols appearing in the string, and a description of the arrangement of the symbols form were presented. Two descriptions of the symbol arrangement were considered: shapes and patterns. Roughly speaking, shapes describe the relative magnitude of the symbols while patterns describe only the order in which they appear. The per-symbol worst-case redundancy of compressing shapes is a positive constant less than one, and the per-symbol redundancy of compressing patterns diminishes to zero as the block-length increases were proven. Some results on sequential pattern compression were also mentioned.

[1]  T. Uyematsu,et al.  Asymptotical optimality of two variations of Lempel-Ziv codes for sources with countably infinite alphabet , 2002, Proceedings IEEE International Symposium on Information Theory,.

[2]  W. Szpankowski,et al.  The precise minimax redundancy , 2002, Proceedings IEEE International Symposium on Information Theory,.

[3]  Richard M. Wilson,et al.  A course in combinatorics , 1992 .

[4]  Jorma Rissanen,et al.  Universal coding, information, prediction, and estimation , 1984, IEEE Trans. Inf. Theory.

[5]  Erik Ordentlich,et al.  Universal portfolios with side information , 1996, IEEE Trans. Inf. Theory.

[6]  John C. Kieffer,et al.  A unified approach to weak universal source coding , 1978, IEEE Trans. Inf. Theory.

[7]  Peter Elias,et al.  Universal codeword sets and representations of the integers , 1975, IEEE Trans. Inf. Theory.

[8]  László Györfi,et al.  On Universal Noiseless Source Coding for Infinite Source Alphabets , 1993, Eur. Trans. Telecommun..

[9]  A. Barron,et al.  Asymptotic minimax regret for data compression, gambling and prediction , 1997, Proceedings of IEEE International Symposium on Information Theory.

[10]  Raphail E. Krichevsky,et al.  The performance of universal encoding , 1981, IEEE Trans. Inf. Theory.

[11]  Jorma Rissanen,et al.  Fisher information and stochastic complexity , 1996, IEEE Trans. Inf. Theory.

[12]  Neri Merhav,et al.  Universal Prediction , 1998, IEEE Trans. Inf. Theory.

[13]  Dean P. Foster,et al.  Universal codes for finite sequences of integers drawn from a monotone distribution , 2002, IEEE Trans. Inf. Theory.

[14]  Lee D. Davisson,et al.  Universal noiseless coding , 1973, IEEE Trans. Inf. Theory.

[15]  A. Orlitsky,et al.  Universal compression of unknown alphabets , 2002, Proceedings IEEE International Symposium on Information Theory,.

[16]  T. Cover Universal Portfolios , 1996 .