On a partition of a finite set and its relationships to encoding tasks and the Rényi entropy

In the problem of encoding tasks introduced by Bunte and Lapidoth, a finite alphabet χ is partitioned to M disjoint subsets {ℒ<sub>m</sub>}<sub>m</sub><sup>M</sup>=1 called lists. In this paper we consider the partition of χ into M lists satisfying L(x) ≤ λ(x) for all x ∈ χ for an arbitrarily given mapping λ: χ → [1, ∞), where L(x) denotes the cardinality of the list ℒ<sub>m</sub> satisfying x ∈ ℒ<sub>m</sub>. We investigate the minimum number M*(λ) of the lists meeting this constraint and give a lower and an upper bounds on M*(λ) described in terms of λ. We also show that the partition algorithm given by Bunte and Lapidoth attains M*(λ) with a slight modification. Relationships between M*(λ) and the Rényi entropy are also discussed.

[1]  Erdal Arikan An inequality on guessing and its application to sequential decoding , 1996, IEEE Trans. Inf. Theory.

[2]  I. Csiszár Generalized Cutoff Rates and Renyi's Information Measures , 1993, Proceedings. IEEE International Symposium on Information Theory.

[3]  Amos Lapidoth,et al.  Rényi entropy and quantization for densities , 2014, 2014 IEEE Information Theory Workshop (ITW 2014).

[4]  Amos Lapidoth,et al.  Encoding Tasks and Rényi Entropy , 2014, IEEE Transactions on Information Theory.

[5]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[6]  A. Rényi On Measures of Entropy and Information , 1961 .

[7]  Thomas M. Cover,et al.  Elements of Information Theory: Cover/Elements of Information Theory, Second Edition , 2005 .

[8]  L. L. Campbell,et al.  A Coding Theorem and Rényi's Entropy , 1965, Inf. Control..