On a Construction of Entropic Vectors Using Lattice-Generated Distributions

The problem of determining the region of entropic vectors is a central one in information theory. There has been a great deal of interest in the development of non-Shannon information inequalities, which provide outer bounds to the aforementioned region; however, there has been less work on developing inner bounds. This paper develops an inner bound that applies to any number of random variables and which is tight for 2 and 3 random variables (the only cases where the entropy region is known). The construction is based on probability distributions generated by a lattice. The region is shown to be a polytope generated by a set of linear inequalities. Study of the region for 4 and more random variables is currently under investigation.

[1]  Raymond W. Yeung,et al.  A framework for linear information inequalities , 1997, IEEE Trans. Inf. Theory.

[2]  Malcolm C. Smith.  Applications of Algebraic Function Theory in Multivariable Control , 1984 .

[3]  Nikolai K. Vereshchagin,et al.  A new class of non-Shannon-type inequalities for entropies , 2002, Commun. Inf. Syst..

[4]  Frantisek Matús,et al.  Conditional Independences among Four Random Variables II , 1995, Combinatorics, Probability and Computing.

[5]  Randall Dougherty,et al.  Six New Non-Shannon Information Inequalities , 2006, 2006 IEEE International Symposium on Information Theory.

[6]  Milan Studený,et al.  Conditional Independences among Four Random Variables I , 1995, Combinatorics, Probability and Computing.

[7]  Raymond W. Yeung,et al.  On a relation between information inequalities and group theory , 2002, IEEE Trans. Inf. Theory.

[8]  Zhen Zhang,et al.  On Characterization of Entropy Function via Information Inequalities , 1998, IEEE Trans. Inf. Theory.

[9]  Zhen Zhang,et al.  A non-Shannon-type conditional inequality of information quantities , 1997, IEEE Trans. Inf. Theory.

[10]  F. Mat.  Two Constructions on Limits of Entropy Functions , 2007, IEEE Trans. Inf. Theory.

[11]  Satoru Fujishige,et al.  Polymatroidal Dependence Structure of a Set of Random Variables , 1978, Inf. Control..

[12]  Zhen Zhang,et al.  On a new non-Shannon-type information inequality , 2002, Proceedings IEEE International Symposium on Information Theory,.

[13]  Ho-Leung Chan,et al.  A combinatorial approach to information inequalities , 1999, 1999 Information Theory and Networking Workshop (Cat. No.99EX371).

[14]  Zhen Zhang,et al.  Distributed Source Coding for Satellite Communications , 1999, IEEE Trans. Inf. Theory.