Entropy Region and Convolution

The entropy region is constructed from vectors of random variables by collecting Shannon entropies of all subvectors. Its shape is studied here by means of polymatroidal constructions, notably by convolution. The closure of the region is decomposed into the direct sum of tight and modular parts, reducing the study to the tight part. The relative interior of the reduction belongs to the entropy region. Behavior of the decomposition under self-adhesivity is clarified. Results are specialized and extended to the region constructed from four tuples of random variables. This and computer experiments help to visualize approximations of a symmetrized part of the entropy region. The four-atom conjecture on the minimal Ingleton score is refuted.

[1]  James G. Oxley,et al.  Matroid theory , 1992 .

[2]  Demetres Christofides,et al.  Multiple unicasts, graph guessing games, and non-Shannon inequalities , 2013, 2013 International Symposium on Network Coding (NetCod).

[3]  Rahim Tafazolli,et al.  Network Coding Theory: A Survey , 2013, IEEE Communications Surveys & Tutorials.

[4]  Satoru Fujishige,et al.  Polymatroidal Dependence Structure of a Set of Random Variables , 1978, Inf. Control..

[5]  Prasad Tetali,et al.  Information-theoretic inequalities in additive combinatorics , 2010, 2010 IEEE Information Theory Workshop on Information Theory (ITW 2010, Cairo).

[6]  F. Matús.  PROBABILISTIC CONDITIONAL INDEPENDENCE STRUCTURES AND MATROID THEORY: BACKGROUND1 , 1993 .

[7]  F. Matús,et al.  Two Constructions on Limits of Entropy Functions , 2007, IEEE Transactions on Information Theory.

[8]  C.J.H. Mann,et al.  Probabilistic Conditional Independence Structures , 2005 .

[9]  Zhen Zhang,et al.  On a new non-Shannon-type information inequality , 2002, Proceedings IEEE International Symposium on Information Theory,.

[10]  Babak Hassibi,et al.  On the Ingleton-Violating Finite Groups and Group Network Codes , 2012, ArXiv.

[11]  Milan Studený,et al.  Conditional Independences among Four Random Variables I , 1995, Combinatorics, Probability and Computing.

[12]  M. Lunelli,et al.  Representation of matroids , 2002, math/0202294.

[13]  Nikolai K. Vereshchagin,et al.  Inequalities for Shannon Entropy and Kolmogorov Complexity , 1997, J. Comput. Syst. Sci..

[14]  Frantisek Matús,et al.  Adhesivity of polymatroids , 2007, Discret. Math..

[15]  Amos Beimel,et al.  Secret Sharing and Non-Shannon Information Inequalities , 2011, IEEE Transactions on Information Theory.

[16]  Maximilien Gadouleau,et al.  Graph-Theoretical Constructions for Graph Entropy and Network Coding Based Communications , 2011, IEEE Transactions on Information Theory.

[17]  Terence Chan.  Recent Progresses in Characterising Information Inequalities , 2011, Entropy.

[18]  Zhen Zhang,et al.  On Characterization of Entropy Function via Information Inequalities , 1998, IEEE Trans. Inf. Theory.

[19]  Andrei E. Romashchenko,et al.  Conditional Information Inequalities for Entropic and Almost Entropic Points , 2012, IEEE Transactions on Information Theory.

[20]  László Lovász,et al.  Submodular functions and convexity , 1982, ISMP.

[21]  Tarik Kaced.  Equivalence of two proof techniques for non-shannon-type inequalities , 2013, 2013 IEEE International Symposium on Information Theory.

[22]  Matthew Thill,et al.  On the Ingleton-Violations in Finite Groups , 2012 .

[23]  S. Weber,et al.  Relationships among bounds for the region of entropic vectors in four variables , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[24]  Zhen Zhang,et al.  A non-Shannon-type conditional inequality of information quantities , 1997, IEEE Trans. Inf. Theory.

[25]  A. Ingleton,et al.  Conditions for representability and transversality of matroids , 1971 .

[26]  Andrei E. Romashchenko,et al.  On essentially conditional information inequalities , 2011, 2011 IEEE International Symposium on Information Theory Proceedings.

[27]  Raymond W. Yeung,et al.  On a relation between information inequalities and group theory , 2002, IEEE Trans. Inf. Theory.

[28]  Babak Hassibi,et al.  On a Construction of Entropic Vectors Using Lattice-Generated Distributions , 2007, 2007 IEEE International Symposium on Information Theory.

[29]  R. Rockafellar.  Convex Analysis: (pms-28) , 1970 .

[30]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[31]  F. Mattt,et al.  Conditional Independences among Four Random Variables Iii: Final Conclusion , 1999 .

[32]  Randall Dougherty,et al.  Six New Non-Shannon Information Inequalities , 2006, 2006 IEEE International Symposium on Information Theory.

[33]  Frantisek Matús,et al.  Infinitely Many Information Inequalities , 2007, 2007 IEEE International Symposium on Information Theory.

[34]  Nikolai K. Vereshchagin,et al.  A new class of non-Shannon-type inequalities for entropies , 2002, Commun. Inf. Syst..

[35]  Carles Padró,et al.  Matroids Can Be Far from Ideal Secret Sharing , 2008, TCC.

[36]  Raymond W. Yeung,et al.  A First Course in Information Theory , 2002 .

[37]  Søren Riis,et al.  Information flows, graphs and their guessing numbers , 2006, 2006 4th International Symposium on Modeling and Optimization in Mobile, Ad Hoc and Wireless Networks.

[38]  Frantisek Matús,et al.  Classes of Matroids Closed Under Minors and Principal Extensions , 2018, Comb..

[39]  László Csirmaz.  Book Inequalities , 2014, IEEE Transactions on Information Theory.

[40]  Randall Dougherty,et al.  Non-Shannon Information Inequalities in Four Random Variables , 2011, ArXiv.

[41]  T. H. Chan,et al.  Balanced information inequalities , 2003, IEEE Trans. Inf. Theory.

[42]  Alex J. Grant,et al.  Truncation Technique for Characterizing Linear Polymatroids , 2011, IEEE Transactions on Information Theory.

[43]  Nigel Boston,et al.  Large violations of the Ingleton inequality , 2012, 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[44]  Imre Csiszár,et al.  Information Theory - Coding Theorems for Discrete Memoryless Systems, Second Edition , 2011 .

[45]  Andrei E. Romashchenko,et al.  On the non-robustness of essentially conditional information inequalities , 2012, 2012 IEEE Information Theory Workshop.

[46]  László Csirmaz,et al.  Using multiobjective optimization to map the entropy region , 2013, Computational Optimization and Applications.