Discrete Entropy Power Inequalities via Sperner Theory

A ranked poset is called strongly Sperner if the size of $k$-family cannot exceed the sum of $k$-largest Whitney numbers. In a sense of function ordering, a function $f$ is (weakly) majorized by $g$ if the the sum of $k$-largest values in $f$ cannot exceed the sum of $k$-largest values in $g$. Two definitions arise from different contexts, but each share a strong similarity with each other. Furthermore, the product of two weighted posets assumes a structural similarity with a convolution of two functions. Elements in the product of weighted posets with ranks capture underlying structures of the building blocks in the convolution. Combining all together, we are able to derive several types of entropy inequalities including discrete entropy power inequalities, and discuss more applications.

[1]  Mokshay M. Madiman,et al.  Log-concavity, ultra-log-concavity, and a maximum entropy property of discrete compound Poisson measures , 2009, Discret. Appl. Math..

[2]  Varun Jog,et al.  The Entropy Power Inequality and Mrs. Gerber's Lemma for Groups of Order 2n , 2014, IEEE Trans. Inf. Theory.

[3]  Peng Xu,et al.  An Exact Upper Bound on the Lp Lebesgue Constant and The ∞-Rényi Entropy Power Inequality for Integer Valued Random Variables , 2018, ArXiv.

[4]  Jae Oh Woo,et al.  A discrete entropy power inequality for uniform distributions , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[5]  Patrick P. Bergmans,et al.  A simple converse for broadcast channels with additive white Gaussian noise (Corresp.) , 1974, IEEE Trans. Inf. Theory.

[6]  E. Lieb.  Proof of an entropy conjecture of Wehrl , 1978 .

[7]  Hoi H. Nguyen,et al.  A new approach to an old problem of Erdős and Moser , 2011, J. Comb. Theory, Ser. A.

[8]  G. H. Hardy,et al.  Notes on the Theory of Series (VIII): An Inequality , 1928 .

[9]  Richard J. Gardner,et al.  A Brunn-Minkowski inequality for the integer lattice , 2001 .

[10]  W. Beckner.  Inequalities in Fourier analysis , 1975 .

[11]  Oliver Johnson,et al.  Monotonicity, Thinning, and Discrete Versions of the Entropy Power Inequality , 2009, IEEE Transactions on Information Theory.

[12]  Oliver Johnson,et al.  Entropy and the law of small numbers , 2005, IEEE Transactions on Information Theory.

[13]  Imre Z. Ruzsa.  Sumsets and entropy , 2009 .

[14]  R. Stanley.  Enumerative Combinatorics: Volume 1 , 2011 .

[15]  A. J. Stam.  Some Inequalities Satisfied by the Quantities of Information of Fisher and Shannon , 1959, Inf. Control..

[16]  Jae Oh Woo,et al.  Entropy Inequalities for Sums in Prime Cyclic Groups , 2017, SIAM J. Discret. Math..

[17]  V. V. Buldygin,et al.  Brunn-Minkowski inequality , 2000 .

[18]  Liyao Wang,et al.  Beyond the Entropy Power Inequality, via Rearrangements , 2013, IEEE Transactions on Information Theory.

[19]  Mokshay Madiman,et al.  Entropy Bounds on Abelian Groups and the Ruzsa Divergence , 2015, IEEE Transactions on Information Theory.

[20]  Peng Xu,et al.  Forward and Reverse Entropy Power Inequalities in Convex Geometry , 2016, ArXiv.

[21]  Shlomo Shamai,et al.  The Capacity Region of the Gaussian Multiple-Input Multiple-Output Broadcast Channel , 2006, IEEE Transactions on Information Theory.

[22]  Mokshay M. Madiman,et al.  Generalized Entropy Power Inequalities and Monotonicity Properties of Information , 2006, IEEE Transactions on Information Theory.

[23]  Jae Oh Woo,et al.  A lower bound on the Rényi entropy of convolutions in the integers , 2014, 2014 IEEE International Symposium on Information Theory.

[24]  J. Littlewood,et al.  On the Number of Real Roots of a Random Algebraic Equation , 1938 .

[25]  Sergey G. Bobkov,et al.  Variants of the Entropy Power Inequality , 2017, IEEE Transactions on Information Theory.

[26]  Amir Dembo,et al.  Information theoretic inequalities , 1991, IEEE Trans. Inf. Theory.

[27]  O. Johnson.  Information Theory And The Central Limit Theorem , 2004 .

[28]  Vsevolod F. Lev,et al.  Linear equations over $\mathbb{F}_p$ and moments of exponential sums , 2001 .

[29]  Jiange Li,et al.  Rényi entropy power inequality and a reverse , 2017, ArXiv.

[30]  Mokshay M. Madiman,et al.  Compound Poisson Approximation via Information Functionals , 2010, ArXiv.

[31]  Varun Jog,et al.  The Entropy Power Inequality and Mrs. Gerber's Lemma for groups of order 2n , 2013, 2013 IEEE International Symposium on Information Theory.

[32]  Jeff Kahn,et al.  An Entropy Approach to the Hard-Core Model on Bipartite Graphs , 2001, Combinatorics, Probability and Computing.

[33]  Mokshay Madiman,et al.  Combinatorial Entropy Power Inequalities: A Preliminary Study of the Stam Region , 2017, IEEE Transactions on Information Theory.

[34]  K. Ball,et al.  Solution of Shannon's problem on the monotonicity of entropy , 2004 .

[35]  Daniel J. Kleitman,et al.  Normalized Matching in Direct Products of Partial Orders , 1973 .

[36]  Robert A. Proctor,et al.  Solution of Two Difficult Combinatorial Problems with Linear Algebra , 1982 .

[37]  Mokshay M. Madiman,et al.  Information Inequalities for Joint Distributions, With Interpretations and Applications , 2008, IEEE Transactions on Information Theory.

[38]  Richard P. Stanley,et al.  Weyl Groups, the Hard Lefschetz Theorem, and the Sperner Property , 1980, SIAM J. Algebraic Discret. Methods.

[39]  Emre Telatar,et al.  A New Entropy Power Inequality for Integer-Valued Random Variables , 2014, IEEE Trans. Inf. Theory.

[40]  Giuseppe Toscani,et al.  A Strengthened Entropy Power Inequality for Log-Concave Densities , 2014, IEEE Transactions on Information Theory.

[41]  M. Madiman,et al.  Rogozin's convolution inequality for locally compact groups , 2017, 1705.00642.

[42]  Robert A. Proctor,et al.  Product partial orders with the sperner property , 1980, Discret. Math..

[43]  Terence Tao,et al.  Sumset and Inverse Sumset Theory for Shannon Entropy , 2009, Combinatorics, Probability and Computing.

[44]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[45]  P. Erdös.  On a lemma of Littlewood and Offord , 1945 .

[46]  Shlomo Shamai,et al.  The Interplay Between Information and Estimation Measures , 2013, Found. Trends Signal Process..

[47]  Ron Peled,et al.  Rigidity of proper colorings of $${\mathbb {Z}}^{d}$$ , 2018, Inventiones mathematicae.