Concentration of the information in data with log-concave distributions

on which the distribution of Xis supported. In this case, eh(X) is essentially the number of bits neededto represent X by a coding scheme that minimizes average code length([Sha48]). In the continuous case (with reference measure dx), one may stillcall eh(X) the information content even though the coding interpretation nolonger holds. In statistics, one may think of the information content as thelog likelihood function.The average value of the information content of Xis known more com-monly as the entropy. Indeed, the entropy of Xis defined byh(X) = −Zf(x)logf(x)dx= −Elogf(X).

[1]  B. McMillan.  The Basic Theorems of Information Theory , 1953 .

[2]  L. Breiman.  The Individual Ergodic Theorem of Information Theory , 1957 .

[3]  Shu-Teh Chen Moy,et al.  Generalizations of Shannon-McMillan theorem , 1961 .

[4]  Richard E. Barlow,et al.  Moment inequalities of Pólya frequency functions. , 1961 .

[5]  A. W. Marshall,et al.  Properties of Probability Distributions with Monotone Hazard Rate , 1963 .

[6]  C. Borell.  Complements of Lyapunov's inequality , 1973 .

[7]  C. Borell.  Convex measures on locally convex spaces , 1974 .

[8]  J. Kieffer.  A SIMPLE PROOF OF THE MOY-PEREZ GENERALIZATION OF THE SHANNON-MCMILLAN THEOREM , 1974 .

[9]  A. Barron.  THE STRONG ERGODIC THEOREM FOR DENSITIES: GENERALIZED SHANNON-MCMILLAN-BREIMAN THEOREM' , 1985 .

[10]  J. Lindenstrauss,et al.  Geometric Aspects of Functional Analysis , 1987 .

[11]  T. Cover,et al.  A sandwich proof of the Shannon-McMillan-Breiman theorem , 1988 .

[12]  Thomas M. Cover,et al.  Gaussian feedback capacity , 1989, IEEE Trans. Inf. Theory.

[13]  Miklós Simonovits,et al.  Random Walks in a Convex Body and an Improved Volume Algorithm , 1993, Random Struct. Algorithms.

[14]  Miklós Simonovits,et al.  Isoperimetric problems for convex bodies and a localization lemma , 1995, Discret. Comput. Geom..

[15]  Sergey G. Bobkov,et al.  Extremal properties of half-spaces for log-concave distributions , 1996 .

[16]  S. G. Bobkov,et al.  Spectral Gap and Concentration for Some Spherically Symmetric Probability Measures , 2003 .

[17]  V. Milman,et al.  Geometry of Log-concave Functions and Measures , 2005 .

[18]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[19]  Sergey G. Bobkov,et al.  The Entropy Per Coordinate of a Random Vector is Highly Constrained Under Convexity Conditions , 2010, IEEE Transactions on Information Theory.