The Entropy Per Coordinate of a Random Vector is Highly Constrained Under Convexity Conditions

The entropy per coordinate in a log-concave random vector of any dimension with given density at the mode is shown to have a range of just 1. Uniform distributions on convex bodies are at the lower end of this range, the distribution with i.i.d. exponentially distributed coordinates is at the upper end, and the normal is exactly in the middle. Thus, in terms of the amount of randomness as measured by entropy per coordinate, any log-concave random vector of any dimension contains randomness that differs from that in the normal random variable with the same maximal density value by at most 1/2. As applications, we obtain an information-theoretic formulation of the famous hyperplane conjecture in convex geometry, entropy bounds for certain infinitely divisible distributions, and quantitative estimates for the behavior of the density at the mode on convolution. More generally, one may consider so-called convex or hyperbolic probability measures on Euclidean spaces; we give new constraints on entropy per coordinate for this class of measures, which generalize our results under the log-concavity assumption, expose the extremal role of multivariate Pareto-type distributions, and give some applications.

[1]  D. Varberg.  Convex Functions , 1973 .

[2]  Robert M. Gray,et al.  Toeplitz and Circulant Matrices: A Review , 2005, Found. Trends Commun. Inf. Theory.

[3]  Robert M. Gray,et al.  Toeplitz And Circulant Matrices: A Review (Foundations and Trends(R) in Communications and Information Theory) , 2006 .

[4]  Shiri Artstein-Avidan,et al.  The "M"-ellipsoid, symplectic capacities and volume , 2006 .

[5]  Erwin Lutwak,et al.  Moment-Entropy Inequalities for a Random Vector , 2007, IEEE Transactions on Information Theory.

[6]  Bjorn G. Hansen,et al.  On Log-Concave and Log-Convex Infinitely Divisible Sequences and Densities , 1988 .

[7]  C. Vignat,et al.  Some results concerning maximum Renyi entropy distributions , 2005, math/0507400.

[8]  O. Johnson.  Log-concavity and the maximum entropy property of the Poisson distribution , 2006, math/0603647.

[9]  Zhen Zhang,et al.  On the maximum entropy of the sum of two dependent random variables , 1994, IEEE Trans. Inf. Theory.

[10]  S. Bobkov.  Large deviations and isoperimetry over convex probability measures with heavy tails , 2007 .

[11]  Jean-François Bercher,et al.  An entropic view of Pickands’ theorem , 2008, 2008 IEEE International Symposium on Information Theory.

[12]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[13]  A. J. Stam.  Some Inequalities Satisfied by the Quantities of Information of Fisher and Shannon , 1959, Inf. Control..

[14]  E. Lieb,et al.  On extensions of the Brunn-Minkowski and Prékopa-Leindler theorems, including inequalities for log concave functions, and with an application to the diffusion equation , 1976 .

[15]  E. Beckenbach.  CONVEX FUNCTIONS , 2007 .

[16]  Ilya S. Molchanov,et al.  Convex and star-shaped sets associated with multivariate stable distributions, I: Moments and densities , 2009, J. Multivar. Anal..

[17]  Mokshay M. Madiman,et al.  Log-concavity, ultra-log-concavity, and a maximum entropy property of discrete compound Poisson measures , 2009, Discret. Appl. Math..

[18]  A. Prékopa.  Logarithmic concave measures with applications to stochastic programming , 1971 .

[19]  Yaming Yu.  On an inequality of Karlin and Rinott concerning weighted sums of i.i.d. random variables , 2008, Advances in Applied Probability.

[20]  Erwin Lutwak,et al.  Moment-entropy inequalities , 2004 .

[21]  Nicole Tomczak-Jaegermann,et al.  Geometric inequalities for a class of exponential measures , 2004 .

[22]  L. Leindler.  On a Certain Converse of Hölder’s Inequality , 1972 .

[23]  Sunghyu Han,et al.  Submitted to Ieee Transactions on Information Theory Upper Bounds for the Length of S-extremal Codes over F 2 , F 4 , and F 2 + Uf 2 , 2007 .

[24]  S. Bobkov.  Isoperimetric and Analytic Inequalities for Log-Concave Probability Measures , 1999 .

[25]  K. Ball.  Logarithmically concave functions and sections of convex sets in $R^{n}$ , 1988 .

[26]  T. Cover,et al.  IEEE TRANSACTIONSON INFORMATIONTHEORY,VOL. IT-30,N0. 6,NOVEmER1984 Correspondence On the Similarity of the Entropy Power Inequality The preceeding equations allow the entropy power inequality and the Brunn-Minkowski Inequality to be rewritten in the equiv , 2022 .

[27]  Alfred O. Hero,et al.  On Solutions to Multivariate Maximum alpha-Entropy Problems , 2003, EMMCVPR.

[28]  Makoto Yamazato.  On Strongly Unimodal Infinitely Divisible Distributions , 1982 .

[29]  Raluca Vernic,et al.  On a Multivariate Pareto Distribution , 2009 .

[30]  L. Berwald,et al.  Verallgemeinerung eines Mittelwertsatzes von J. Favard für Positive Konkave Funktionen , 1947 .

[31]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[32]  A. Prékopa.  On logarithmic concave measures and functions , 1973 .

[33]  Hsiaw-Chan Yeh.  Some properties and characterizations for generalized multivariate Pareto distributions , 2004 .

[34]  D. Applebaum.  Lévy Processes and Stochastic Calculus: Preface , 2009 .

[35]  Matthieu Fradelizi,et al.  Sections of convex bodies through their centroid , 1997 .

[36]  Mokshay M. Madiman,et al.  On the entropy and log-concavity of compound Poisson measures , 2008, ArXiv.

[37]  D. Bakry,et al.  A simple proof of the Poincaré inequality for a large class of probability measures , 2008 .

[38]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[39]  Sergey G. Bobkov,et al.  Extremal properties of half-spaces for log-concave distributions , 1996 .

[40]  M. Kanter,et al.  Unimodality and dominance for symmetric random vectors , 1977 .

[41]  J. Bourgain.  On the distribution of polynomials on high dimensional convex sets , 1991 .

[42]  Amir Dembo,et al.  Information theoretic inequalities , 1991, IEEE Trans. Inf. Theory.

[43]  Bo'az Klartag,et al.  Symmetrization and Isotropic Constants of Convex Bodies , 2004 .

[44]  Jean Bourgain,et al.  ON HIGH DIMENSIONAL MAXIMAL FUNCTIONS ASSOCIATED TO CONVEX BODIES , 1986 .

[45]  V. Milman,et al.  Geometry of Log-concave Functions and Measures , 2005 .

[46]  V. Milman,et al.  Isotropic position and inertia ellipsoids and zonoids of the unit ball of a normed n-dimensional space , 1989 .

[47]  Mordecai Avriel,et al.  r-convex functions , 1972, Math. Program..

[48]  Axthonv G. Oettinger,et al.  IEEE Transactions on Information Theory , 1998 .

[49]  D. Hensley.  Slicing convex bodies—bounds for slice area in terms of the body’s covariance , 1980 .

[50]  S. Dar.  On the isotropic constant of non-symmetric convex bodies , 1997 .

[51]  C. Borell.  Convex set functions ind-space , 1975 .

[52]  C. Borell.  Convex measures on locally convex spaces , 1974 .

[53]  C. Borell.  Complements of Lyapunov's inequality , 1973 .

[54]  S. Bobkov,et al.  Reverse Brunn–Minkowski and reverse entropy power inequalities for convex measures , 2011, 1109.5287.

[55]  Vitali Milman,et al.  Isomorphic symmetrization and geometric inequalities , 1988 .

[56]  佐藤 健一.  Lévy processes and infinitely divisible distributions , 2013 .

[57]  Samuel Kotz,et al.  Multivariate Pareto Distributions , 2005 .

[58]  Sergey G. Bobkov,et al.  Convex bodies and norms associated to convex measures , 2010 .

[59]  Marius Junge.  Volume estimates for log-concave densities with application to iterated convolutions , 1995 .

[60]  G. Pisier.  The volume of convex bodies and Banach space geometry , 1989 .

[61]  B. Klartag.  On convex perturbations with a bounded isotropic constant , 2006 .