Beyond the Entropy Power Inequality, via Rearrangements

A lower bound on the Rényi differential entropy of a sum of independent random vectors is demonstrated in terms of rearrangements. For the special case of Boltzmann-Shannon entropy, this lower bound is better than that given by the entropy power inequality. Several applications are discussed, including a new proof of the classical entropy power inequality and an entropy inequality involving symmetrization of Lévy processes.

[1]  W. Beckner.  Inequalities in Fourier analysis , 1975 .

[2]  R. Cooke.  Real and Complex Analysis , 2011 .

[3]  S. Sobolev.  On a theorem in functional analysis , 1938 .

[4]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[5]  Sergey G. Bobkov,et al.  On the problem of reversibility of the entropy power inequality , 2011, ArXiv.

[6]  Peter Harremoës,et al.  Rényi Divergence and Kullback-Leibler Divergence , 2012, IEEE Transactions on Information Theory.

[7]  Sergey G. Bobkov,et al.  Dimensional behaviour of entropy and information , 2011, ArXiv.

[8]  Toshiro Watanabe.  The isoperimetric inequality for isotropic unimodal Lévy processes , 1983 .

[9]  Max H. M. Costa,et al.  A new entropy power inequality , 1985, IEEE Trans. Inf. Theory.

[10]  Erwin Lutwak,et al.  Affine Moments of a Random Vector , 2013, IEEE Transactions on Information Theory.

[11]  Sergey G. Bobkov,et al.  The Entropy Per Coordinate of a Random Vector is Highly Constrained Under Convexity Conditions , 2010, IEEE Transactions on Information Theory.

[12]  Bruce E. Hajek,et al.  Paging and Registration in Cellular Networks: Jointly Optimal Policies and an Iterative Algorithm , 2007, IEEE Transactions on Information Theory.

[13]  E. Carlen,et al.  Entropy production by block variable summation and central limit theorems , 1991 .

[14]  Stanislaw J. Szarek,et al.  Shannon’s entropy power inequality via restricted minkowski sums , 2000 .

[15]  Erwin Lutwak,et al.  Moment-Entropy Inequalities for a Random Vector , 2007, IEEE Transactions on Information Theory.

[16]  Mokshay M. Madiman,et al.  The Ruzsa divergence for random elements in locally compact abelian groups , 2015, ArXiv.

[17]  C. Vignat,et al.  Some results concerning maximum Renyi entropy distributions , 2005, math/0507400.

[18]  Sergey G. Bobkov,et al.  Entropy Power Inequality for the Rényi Entropy , 2015, IEEE Transactions on Information Theory.

[19]  R. Bañuelos,et al.  Symmetrization of Lévy processes and applications , 2010 .

[20]  Amir Dembo,et al.  Large Deviations Techniques and Applications , 1998 .

[21]  A. Burchard,et al.  A Short Course on Rearrangement Inequalities , 2009 .

[22]  O. Johnson.  Information Theory And The Central Limit Theorem , 2004 .

[23]  Jose A. Costa,et al.  On Solutions to Multivariate Maximum α-Entropy Problems , 2003 .

[24]  Liyao Wang,et al.  Fractional generalizations of Young and Brunn-Minkowski inequalities , 2010, ArXiv.

[25]  K. Chong,et al.  Some Extensions of a Theorem of Hardy, Littlewood and Pólya and Their Applications , 1974, Canadian Journal of Mathematics.

[26]  M. Ledoux,et al.  Logarithmic Sobolev Inequalities , 2014 .

[27]  G. Pólya,et al.  Inequalities (Cambridge Mathematical Library) , 1934 .

[28]  C. A. Rogers.  A Single Integral Inequality , 1957 .

[29]  A. Burchard,et al.  Comparison theorems for exit times , 2001 .

[30]  Giuseppe Toscani,et al.  The Concavity of Rényi Entropy Power , 2014, IEEE Transactions on Information Theory.

[31]  Pierre-Louis Lions,et al.  On optimization problems with prescribed rearrangements , 1989 .

[32]  Alfred O. Hero,et al.  On Solutions to Multivariate Maximum alpha-Entropy Problems , 2003, EMMCVPR.

[33]  E. Lieb,et al.  Best Constants in Young's Inequality, Its Converse, and Its Generalization to More than Three Functions , 1976 .

[34]  Giuseppe Toscani,et al.  An information-theoretic proof of Nash's inequality , 2012, ArXiv.

[35]  Friedrich Götze,et al.  Fisher information and the central limit theorem , 2012 .

[36]  A. J. Stam.  Some Inequalities Satisfied by the Quantities of Information of Fisher and Shannon , 1959, Inf. Control..

[37]  Thomas M. Cover,et al.  Elements of information theory (2. ed.) , 2006 .

[38]  Elliott H. Lieb,et al.  A General Rearrangement Inequality for Multiple Integrals , .

[39]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[40]  Jae Oh Woo,et al.  A lower bound on the Rényi entropy of convolutions in the integers , 2014, 2014 IEEE International Symposium on Information Theory.

[41]  Alexander Drewitz,et al.  Symmetric rearrangements around infinity with applications to Lévy processes , 2011, 1110.5592.

[42]  S. Bobkov,et al.  Bounds on the Maximum of the Density for Sums of Independent Random Variables , 2014 .

[43]  S. Bobkov,et al.  Reverse Brunn–Minkowski and reverse entropy power inequalities for convex measures , 2011, 1109.5287.

[44]  Sergey G. Bobkov,et al.  On Concentration Functions of Random Variables , 2015 .

[45]  Amir Dembo,et al.  Information theoretic inequalities , 1991, IEEE Trans. Inf. Theory.

[46]  Almut Burchard,et al.  Cases of equality in the riesz rearrangement inequality , 1996 .

[47]  F. Riesz.  Sur Une Inégalité Intégarale , 1930 .