Forward and Reverse Entropy Power Inequalities in Convex Geometry

The entropy power inequality, which plays a fundamental role in information theory and probability, may be seen as an analogue of the Brunn-Minkowski inequality. Motivated by this connection to Convex Geometry, we survey various recent developments on forward and reverse entropy power inequalities not just for the Shannon-Boltzmann entropy but also more generally for Renyi entropy. In the process, we discuss connections between the so-called functional (or integral) and probabilistic (or entropic) analogues of some classical inequalities in geometric functional analysis.

[1]  A. Prékopa.  On logarithmic concave measures and functions , 1973 .

[2]  Emanuel Milman.  Sharp Isoperimetric Inequalities and Model Spaces for Curvature-Dimension-Diameter Condition , 2011 .

[3]  W. Beckner.  Inequalities in Fourier analysis , 1975 .

[4]  Michael Christ.  Near equality in the two-dimensional Brunn-Minkowski inequality , 2012 .

[5]  V. Milman,et al.  New volume ratio properties for convex symmetric bodies in ℝn , 1987 .

[6]  David Jerison,et al.  Quantitative stability for sumsets in R[superscript n] , 2015 .

[7]  Vitali Milman.  Geometrization of Probability , 2007 .

[8]  Cédric Villani,et al.  A short proof of the "Concavity of entropy power" , 2000, IEEE Trans. Inf. Theory.

[9]  Q. Shao,et al.  Gaussian processes: Inequalities, small ball probabilities and applications , 2001 .

[10]  S. Bobkov,et al.  Concentration of the information in data with log-concave distributions , 2010, 1012.5457.

[11]  C. Borell.  Convex measures on locally convex spaces , 1974 .

[12]  E. Lieb,et al.  On extensions of the Brunn-Minkowski and Prékopa-Leindler theorems, including inequalities for log concave functions, and with an application to the diffusion equation , 1976 .

[13]  S. Bobkov,et al.  Reverse Brunn–Minkowski and reverse entropy power inequalities for convex measures , 2011, 1109.5287.

[14]  Mokshay M. Madiman,et al.  Entropies of Weighted Sums in Cyclic Groups and an Application to Polar Codes , 2017, Entropy.

[15]  Varun Jog,et al.  The Entropy Power Inequality and Mrs. Gerber's Lemma for groups of order 2n , 2013, 2013 IEEE International Symposium on Information Theory.

[16]  V. Milman,et al.  Isotropic position and inertia ellipsoids and zonoids of the unit ball of a normed n-dimensional space , 1989 .

[17]  Erwin Lutwak,et al.  Moment-Entropy Inequalities for a Random Vector , 2007, IEEE Transactions on Information Theory.

[18]  Richard J. Gardner,et al.  A Brunn-Minkowski inequality for the integer lattice , 2001 .

[19]  Dimitri Shlyakhtenko,et al.  Shannon's monotonicity problem for free and classical entropy , 2007, Proceedings of the National Academy of Sciences.

[20]  C. Petty,et al.  AFFINE ISOPERIMETRIC PROBLEMS , 1985 .

[21]  Michael Christ,et al.  Near-extremizers of Young's Inequality for R^d , 2011, 1112.4875.

[22]  H. Groemer,et al.  On the Brunn-Minkowski theorem , 1988 .

[23]  B. Simon.  Convexity: An Analytic Viewpoint , 2011 .

[24]  T. Tao,et al.  From the Littlewood-Offord problem to the Circular Law: universality of the spectral distribution of random matrices , 2008, 0810.2994.

[25]  Yaming Yu,et al.  On an inequality of Karlin and Rinott concerning weighted sums of i.i.d. random variables , 2008, Advances in Applied Probability.

[26]  K. Ball.  Cube slicing in ⁿ , 1986 .

[27]  Mokshay Madiman,et al.  A Combinatorial Approach to Small Ball Inequalities for Sums and Differences , 2016, Combinatorics, Probability and Computing.

[28]  Yonutz V. Stanchescu.  An Upper Bound for d-dimensional Difference Sets , 2001 .

[29]  V. I. Diskant.  Stability of the solution of the Minkowski equation , 1973 .

[30]  O. Johnson.  Information Theory And The Central Limit Theorem , 2004 .

[31]  Elisabeth M. Werner,et al.  R\'enyi Divergence and $L_p$-affine surface area for convex bodies , 2011, 1105.1124.

[32]  Grigoris Paouris,et al.  On sharp bounds for marginal densities of product measures , 2015, 1507.07949.

[33]  Thomas A. Courtade,et al.  Strengthening the entropy power inequality , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[34]  C. Villani,et al.  Ricci curvature for metric-measure spaces via optimal transport , 2004, math/0412127.

[35]  Olivier Rioul,et al.  Information Theoretic Proofs of Entropy Power Inequalities , 2007, IEEE Transactions on Information Theory.

[36]  E. Lieb,et al.  Best Constants in Young's Inequality, Its Converse, and Its Generalization to More than Three Functions , 1976 .

[37]  Michael Kelly,et al.  A Fourier analytic proof of the Blaschke-Santaló Inequality , 2013, 1312.0244.

[38]  H. Busemann,et al.  A Theorem on Convex Bodies of the Brunn-Minkowski Type. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[39]  Sergey G. Bobkov,et al.  On the problem of reversibility of the entropy power inequality , 2011, ArXiv.

[40]  M. Madiman,et al.  Rogozin's convolution inequality for locally compact groups , 2017, 1705.00642.

[41]  Vitali Milman,et al.  $\alpha$-concave functions and a functional extension of mixed volumes , 2013, 1302.0823.

[42]  Sergey G. Bobkov,et al.  Dimensional behaviour of entropy and information , 2011, ArXiv.

[43]  Thomas A. Courtade,et al.  Wasserstein Stability of the Entropy Power Inequality for Log-Concave Densities , 2016, ArXiv.

[44]  Mokshay M. Madiman,et al.  Entropy and set cardinality inequalities for partition‐determined functions , 2008, Random Struct. Algorithms.

[45]  Imre Z. Ruzsa,et al.  Generalized arithmetical progressions and sumsets , 1994 .

[46]  Antonia Maria Tulino,et al.  Monotonic Decrease of the Non-Gaussianness of the Sum of Independent Random Variables: A Simple Proof , 2006, IEEE Transactions on Information Theory.

[47]  E. Carlen,et al.  Entropy production by block variable summation and central limit theorems , 1991 .

[48]  Prasad Tetali,et al.  Information-theoretic inequalities in additive combinatorics , 2010, 2010 IEEE Information Theory Workshop on Information Theory (ITW 2010, Cairo).

[49]  M. Rudelson,et al.  Non-asymptotic theory of random matrices: extreme singular values , 2010, 1003.2990.

[50]  Stanislaw J. Szarek,et al.  Shannon’s entropy power inequality via restricted minkowski sums , 2000 .

[51]  Tinne Hoff Kjeldsen,et al.  Egg-Forms and Measure-Bodies: Different Mathematical Practices in the Early History of the Modern Theory of Convexity , 2009, Science in Context.

[52]  Giuseppe Toscani,et al.  The Concavity of Rényi Entropy Power , 2014, IEEE Transactions on Information Theory.

[53]  Andrea Colesanti,et al.  Log-Concave Functions , 2017 .

[54]  Mokshay M. Madiman,et al.  Compound Poisson Approximation via Information Functionals , 2010, ArXiv.

[55]  Amir Dembo,et al.  Information theoretic inequalities , 1991, IEEE Trans. Inf. Theory.

[56]  B. Klartag,et al.  Functional affine-isoperimetry and an inverse logarithmic Sobolev inequality , 2011, 1110.5551.

[57]  Meir Feder,et al.  On the Volume of the Minkowski Sum of Line Sets and the Entropy-Power Inequality , 1998, IEEE Trans. Inf. Theory.

[58]  Van Hoang Nguyen,et al.  Entropy jumps for isotropic log-concave random vectors and spectral gap , 2012, 1206.5098.

[59]  Emanuel Milman,et al.  Beyond traditional Curvature-Dimension I: new model spaces for isoperimetric and concentration inequalities in negative dimension , 2014, 1409.4109.

[60]  Matthieu Fradelizi,et al.  Some functional forms of Blaschke–Santaló inequality , 2006 .

[61]  Mokshay M. Madiman,et al.  The entropies of the sum and the difference of two IID random variables are not too different , 2010, 2010 IEEE International Symposium on Information Theory.

[62]  M. Meyer,et al.  A geometric property of the boundary of symmetric convex bodies and convexity of flotation surfaces , 1991 .

[63]  Terence Tao,et al.  Additive combinatorics , 2007, Cambridge studies in advanced mathematics.

[64]  Nelson M. Blachman,et al.  The convolution inequality for entropy powers , 1965, IEEE Trans. Inf. Theory.

[65]  C. Vignat,et al.  Some results concerning maximum Renyi entropy distributions , 2005, math/0507400.

[66]  O. Johnson.  Log-concavity and the maximum entropy property of the Poisson distribution , 2006, math/0603647.

[67]  Grigoris Paouris,et al.  The isotropic position and the reverse Santaló inequality , 2011 .

[68]  V. D. Milman.  ON MILMAN ’ S ELLIPSOIDS AND M – POSITION OF CONVEX BODIES , 2010 .

[69]  W. H. Young.  On the Multiplication of Successions of Fourier Constants , 1912 .

[70]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[71]  Mokshay M. Madiman,et al.  Log-concavity, ultra-log-concavity, and a maximum entropy property of discrete compound Poisson measures , 2009, Discret. Appl. Math..

[72]  Sergey G. Bobkov,et al.  Variants of the Entropy Power Inequality , 2017, IEEE Transactions on Information Theory.

[73]  Shlomo Reisner,et al.  THE CONVEX INTERSECTION BODY OF A CONVEX BODY , 2011, Glasgow Mathematical Journal.

[74]  Michael Christ.  Near Equality in the Brunn-Minkowski Inequality , 2012 .

[75]  Yaming Yu,et al.  On the Entropy of Compound Distributions on Nonnegative Integers , 2009, IEEE Transactions on Information Theory.

[76]  B. Bollobás.  THE VOLUME OF CONVEX BODIES AND BANACH SPACE GEOMETRY (Cambridge Tracts in Mathematics 94) , 1991 .

[77]  Alfred O. Hero,et al.  On Solutions to Multivariate Maximum alpha-Entropy Problems , 2003, EMMCVPR.

[78]  Bo'az Klartag,et al.  The Santalo point of a function, and a functional form of the Santalo inequality , 2004 .

[79]  V. V. Buldygin,et al.  Brunn-Minkowski inequality , 2000 .

[80]  M. Rudelson,et al.  The smallest singular value of a random rectangular matrix , 2008, 0802.3956.

[81]  M. Meyer,et al.  Characterizations of ellipsoids by section-centroid location , 1989 .

[82]  S. Bobkov,et al.  Quermassintegrals of quasi-concave functions and generalized Prékopa–Leindler inequalities , 2012, 1210.6364.

[83]  L. Leindler.  On a Certain Converse of Hölder’s Inequality , 1972 .

[84]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[85]  A. Figalli,et al.  A mass transportation approach to quantitative isoperimetric inequalities , 2010 .

[86]  Max H. M. Costa,et al.  A new entropy power inequality , 1985, IEEE Trans. Inf. Theory.

[87]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[88]  Varun Jog,et al.  The Entropy Power Inequality and Mrs. Gerber's Lemma for Groups of Order 2n , 2014, IEEE Trans. Inf. Theory.

[89]  Imre Z. Ruzsa.  Sumsets and entropy , 2009 .

[90]  Karl-Theodor Sturm,et al.  On the geometry of metric measure spaces , 2006 .

[91]  A. J. Stam.  Some Inequalities Satisfied by the Quantities of Information of Fisher and Shannon , 1959, Inf. Control..

[92]  V. Vu,et al.  Small Ball Probability, Inverse Theorems, and Applications , 2012, 1301.0019.

[93]  Mokshay M. Madiman,et al.  Generalized Entropy Power Inequalities and Monotonicity Properties of Information , 2006, IEEE Transactions on Information Theory.

[94]  Franck Barthe.  RESTRICTED PREKOPA-LEINDLER INEQUALITY , 1999 .

[95]  Zhen Zhang,et al.  On the maximum entropy of the sum of two dependent random variables , 1994, IEEE Trans. Inf. Theory.

[96]  Liyao Wang,et al.  Optimal Concentration of Information Content For Log-Concave Densities , 2015, ArXiv.

[97]  G. Pisier.  The volume of convex bodies and Banach space geometry , 1989 .

[98]  Igal Sason,et al.  On Rényi Entropy Power Inequalities , 2016, IEEE Transactions on Information Theory.

[99]  A. Burchard,et al.  A Short Course on Rearrangement Inequalities , 2009 .

[100]  F. Barthe.  Optimal young's inequality and its converse: a simple proof , 1997, math/9704210.

[101]  Peng Xu,et al.  Reverse entropy power inequalities for s-concave densities , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[102]  R. Starr.  Quasi-Equilibria in Markets with Non-Convex Preferences , 1969 .

[103]  A. Yu. Zaitsev,et al.  Arak’s inequalities for concentration functions and the Littlewood–Offord problem , 2015, 1506.09034.

[104]  Joseph Lehec.  Partitions and functional Santaló inequalities , 2009 .

[105]  Karl-Theodor Sturm,et al.  On the geometry of metric measure spaces. II , 2006 .

[106]  Mokshay Madiman,et al.  The Monotonicity of Information in the Central Limit Theorem and Entropy Power Inequalities , 2006, 2006 IEEE International Symposium on Information Theory.

[107]  S. Szarek,et al.  Volumes of restricted Minkowski sums and the free analogue of the entropy power inequality , 1995, math/9510203.

[108]  Sergey G. Bobkov,et al.  The Entropy Per Coordinate of a Random Vector is Highly Constrained Under Convexity Conditions , 2010, IEEE Transactions on Information Theory.

[109]  Michel Ledoux,et al.  The geometry of Euclidean convolution inequalities and entropy , 2009 .

[110]  Terence Tao,et al.  Sumset and Inverse Sumset Theory for Shannon Entropy , 2009, Combinatorics, Probability and Computing.

[111]  A. Figalli,et al.  A refined Brunn-Minkowski inequality for convex sets , 2009 .

[112]  C. Borell.  Convex set functions ind-space , 1975 .

[113]  Mokshay Madiman,et al.  Entropy Bounds on Abelian Groups and the Ruzsa Divergence , 2015, IEEE Transactions on Information Theory.

[114]  Mokshay M. Madiman,et al.  Information concentration for convex measures , 2016, 2016 IEEE International Symposium on Information Theory (ISIT).

[115]  Mokshay Madiman,et al.  On the entropy of sums , 2008, 2008 IEEE Information Theory Workshop.

[116]  Liyao Wang,et al.  Beyond the Entropy Power Inequality, via Rearrangements , 2013, IEEE Transactions on Information Theory.

[117]  Jae Oh Woo,et al.  A lower bound on the Rényi entropy of convolutions in the integers , 2014, 2014 IEEE International Symposium on Information Theory.

[118]  Sergey G. Bobkov,et al.  Entropy Power Inequality for the Rényi Entropy , 2015, IEEE Transactions on Information Theory.

[119]  Rafael Villa,et al.  Rogers–Shephard inequality for log-concave functions , 2014, 1410.2556.

[120]  Yaming Yu,et al.  Monotonic Convergence in an Information-Theoretic Law of Small Numbers , 2008, IEEE Transactions on Information Theory.

[121]  Piotr Nayar,et al.  A reverse entropy power inequality for log-concave random vectors , 2015, 1509.05926.

[122]  Sergey G. Bobkov,et al.  Convex bodies and norms associated to convex measures , 2010 .

[123]  G. C. Shephard,et al.  The difference body of a convex body , 1957 .

[124]  Joseph Lehec.  A direct proof of the functional Santalo inequality , 2009 .

[125]  Christoph Haberl,et al.  Lp intersection bodies , 2008 .

[126]  Oliver Johnson.  An Information-Theoretic Central Limit Theorem for Finitely Susceptible FKG Systems , 2001 .

[127]  Shiri Artstein-Avidan,et al.  On Godbersen’s conjecture , 2014, 1408.2135.

[128]  A. Barron,et al.  Fisher information inequalities and the central limit theorem , 2001, math/0111020.

[129]  E. Lieb.  Proof of an entropy conjecture of Wehrl , 1978 .

[130]  Andrea Colesanti,et al.  Functional Inequalities related to the Rogers-Shephard Inequality , 2005 .

[131]  Matthieu Fradelizi,et al.  Concentration of information content for convex measures , 2015, Electronic Journal of Probability.

[132]  Friedrich Götze,et al.  Неравенства Арака для функций концентрации и проблема Литтлвуда - Оффорда@@@Arak inequalities for concentration functions and the Littlewood - Offord problem , 2017 .

[133]  Jae Oh Woo,et al.  A discrete entropy power inequality for uniform distributions , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[134]  Vitali Milman,et al.  Mixed integrals and related inequalities , 2012, 1210.4346.

[135]  Matthieu Fradelizi,et al.  Some inequalities about mixed volumes , 2003 .

[136]  Emre Telatar,et al.  A New Entropy Power Inequality for Integer-Valued Random Variables , 2014, IEEE Trans. Inf. Theory.

[137]  Giuseppe Toscani,et al.  A Strengthened Entropy Power Inequality for Log-Concave Densities , 2014, IEEE Transactions on Information Theory.

[138]  Tinne Hoff Kjeldsen,et al.  From measuring tool to geometrical object: Minkowski’s development of the concept of convex bodies , 2008 .

[139]  Alexander Segal.  Remark on Stability of Brunn–Minkowski and Isoperimetric Inequalities for Convex Bodies , 2012 .

[140]  A. Prékopa.  Logarithmic concave measures with applications to stochastic programming , 1971 .

[141]  Oliver Johnson.  An information-theoretic central limit theorem for finitely susceptible FKG systems@@@An information-theoretic central limit theorem for finitely susceptible FKG systems , 2005 .

[142]  Vitali Milman,et al.  Isomorphic symmetrization and geometric inequalities , 1988 .

[143]  Matthieu Fradelizi,et al.  On the monotonicity of Minkowski sums towards convexity , 2017 .

[144]  Matthieu Fradelizi,et al.  On the analogue of the concavity of entropy power in the Brunn-Minkowski theory , 2013, Adv. Appl. Math..

[145]  Assaf Naor,et al.  On the rate of convergence in the entropic central limit theorem , 2004 .

[146]  T. Cover,et al.  IEEE TRANSACTIONSON INFORMATIONTHEORY,VOL. IT-30,N0. 6,NOVEmER1984 Correspondence On the Similarity of the Entropy Power Inequality The preceeding equations allow the entropy power inequality and the Brunn-Minkowski Inequality to be rewritten in the equiv , 2022 .

[147]  W. R. Emerson,et al.  Asymptotic behavior of products ^{}=+\cdots+ in locally compact abelian groups , 1969 .

[148]  Erwin Lutwak,et al.  Affine Moments of a Random Vector , 2013, IEEE Transactions on Information Theory.

[149]  Elisabeth M. Werner,et al.  Divergence for s-concave and log concave functions , 2013, 1307.5409.

[150]  D. Hensley.  Slicing convex bodies—bounds for slice area in terms of the body’s covariance , 1980 .

[151]  Erwin Lutwak,et al.  The Brunn-Minkowski-Firey theory. I. Mixed volumes and the Minkowski problem , 1993 .

[152]  Daniel Pérez Palomar,et al.  Hessian and Concavity of Mutual Information, Differential Entropy, and Entropy Power in Linear Vector Gaussian Channels , 2009, IEEE Transactions on Information Theory.

[153]  Liyao Wang,et al.  Fractional generalizations of Young and Brunn-Minkowski inequalities , 2010, ArXiv.

[154]  Oliver Johnson.  A conditional entropy power inequality for dependent variables , 2004, IEEE Transactions on Information Theory.

[155]  Yuliy Baryshnikov,et al.  Hadwiger’s Theorem for definable functions ☆ , 2012, 1203.6120.

[156]  Erwin Lutwak,et al.  Moment-entropy inequalities , 2004 .

[157]  Mokshay M. Madiman,et al.  The entropy power of a sum is fractionally superadditive , 2009, 2009 IEEE International Symposium on Information Theory.

[158]  Varun Jog,et al.  On the geometry of convex typical sets , 2015, 2015 IEEE International Symposium on Information Theory (ISIT).

[159]  K. Ball,et al.  Solution of Shannon's problem on the monotonicity of entropy , 2004 .

[160]  K. Ball.  Logarithmically concave functions and sections of convex sets in $R^{n}$ , 1988 .

[161]  Matthieu Fradelizi,et al.  Do Minkowski averages get progressively more convex , 2015, 1512.03718.

[162]  V. Milman,et al.  Geometry of Log-concave Functions and Measures , 2005 .