Information theoretic inequalities

The role of inequalities in information theory is reviewed, and the relationship of these inequalities to inequalities in other branches of mathematics is developed. The simple inequalities for differential entropy are applied to the standard multivariate normal to furnish new and simpler proofs of the major determinant inequalities in classical mathematics. The authors discuss differential entropy inequalities for random subsets of samples. These inequalities when specialized to multivariate normal variables provide the determinant inequalities that are presented. The authors focus on the entropy power inequality (including the related Brunn-Minkowski, Young's, and Fisher information inequalities) and address various uncertainty principles and their interrelations. >

[1]  A. Oppenheim.  Inequalities Connected with Definite Hermitian Forms , 1930 .

[2]  K. Fan.  On a Theorem of Weyl Concerning Eigenvalues of Linear Transformations: II. , 1949, Proceedings of the National Academy of Sciences of the United States of America.

[3]  K. Fan.  Some inequalities concerning positive-definite Hermitian matrices , 1955 .

[4]  R. Bellman.  NOTES ON MATRIX THEORY--IV (AN INEQUALITY DUE TO BERGSTROM) , 1955 .

[5]  L. Mirsky.  On a Generalization of Hadamard's determinantal inequality due to Szász , 1957 .

[6]  A. J. Stam.  Some Inequalities Satisfied by the Quantities of Information of Fisher and Shannon , 1959, Inf. Control..

[7]  Moshe Zakai,et al.  A Class of Definitions of "Duration" (or "Uncertainty") and the Associated Uncertainty Relations , 1960, Inf. Control..

[8]  Nelson M. Blachman,et al.  The convolution inequality for entropy powers , 1965, IEEE Trans. Inf. Theory.

[9]  S. Kullback,et al.  A lower bound for discrimination information in terms of variation (Corresp.) , 1967, IEEE Trans. Inf. Theory.

[10]  Solomon Kullback,et al.  Correction to A Lower Bound for Discrimination Information in Terms of Variation , 1970, IEEE Trans. Inf. Theory.

[11]  Patrick P. Bergmans,et al.  A simple converse for broadcast channels with additive white Gaussian noise (Corresp.) , 1974, IEEE Trans. Inf. Theory.

[12]  W. Beckner.  Inequalities in Fourier analysis , 1975 .

[13]  E. Lieb,et al.  On extensions of the Brunn-Minkowski and Prékopa-Leindler theorems, including inequalities for log concave functions, and with an application to the diffusion equation , 1976 .

[14]  E. Lieb,et al.  Best Constants in Young's Inequality, Its Converse, and Its Generalization to More than Three Functions , 1976 .

[15]  E. Lieb.  Proof of an entropy conjecture of Wehrl , 1978 .

[16]  A. Wehrl.  General properties of entropy , 1978 .

[17]  Te Sun Han.  Nonnegative Entropy Measures of Multivariate Symmetric Correlations , 1978, Inf. Control..

[18]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .

[19]  I. Olkin,et al.  A Convexity Proof of Hadamard's Inequality , 1982 .

[20]  Abbas El Gamal,et al.  An information - theoretic proof of Hadamard's inequality , 1983, IEEE Trans. Inf. Theory.

[21]  T. Cover,et al.  IEEE TRANSACTIONSON INFORMATIONTHEORY,VOL. IT-30,N0. 6,NOVEmER1984 Correspondence On the Similarity of the Entropy Power Inequality The preceeding equations allow the entropy power inequality and the Brunn-Minkowski Inequality to be rewritten in the equiv , 2022 .

[22]  Max H. M. Costa,et al.  A new entropy power inequality , 1985, IEEE Trans. Inf. Theory.

[23]  A. Barron.  ENTROPY AND THE CENTRAL LIMIT THEOREM , 1986 .

[24]  T. Cover,et al.  Determinant inequalities via information theory , 1988 .

[25]  Amir Dembo,et al.  Simple proof of the concavity of the entropy power with respect to Gaussian noise , 1989, IEEE Trans. Inf. Theory.

[26]  D. Donoho,et al.  Uncertainty principles and signal recovery , 1989 .

[27]  E. Lieb.  Gaussian kernels have only Gaussian maximizers , 1990 .

[28]  E. Carlen,et al.  Entropy production by block variable summation and central limit theorems , 1991 .

[29]  E. Carlen.  Some integral identities and inequalities for entire functions and their application to the coherent state transform , 1991 .

[30]  E. Carlen.  Superadditivity of Fisher's information and logarithmic Sobolev inequalities , 1991 .