Crame/spl acute/r-Rao and moment-entropy inequalities for Renyi entropy and generalized Fisher information

The moment-entropy inequality shows that a continuous random variable with given second moment and maximal Shannon entropy must be Gaussian. Stam's inequality shows that a continuous random variable with given Fisher information and minimal Shannon entropy must also be Gaussian. The Crame/spl acute/r-Rao inequality is a direct consequence of these two inequalities. In this paper, the inequalities above are extended to Renyi entropy, p/sup th/ moment, and generalized Fisher information. Generalized Gaussian random densities are introduced and shown to be the extremal densities for the new inequalities. An extension of the Crame/spl acute/r-Rao inequality is derived as a consequence of these moment and Fisher information inequalities.

[1]  C. Villani,et al.  A MASS-TRANSPORTATION APPROACH TO SHARP SOBOLEV AND GAGLIARDO-NIRENBERG INEQUALITIES , 2004 .

[2]  R. Fisher,et al.  On the Mathematical Foundations of Theoretical Statistics , 1922 .

[3]  Erwin Lutwak,et al.  Information-theoretic inequalities for contoured probability distributions , 2002, IEEE Trans. Inf. Theory.

[4]  E. Arıkan.  An inequality on guessing and its application to sequential decoding , 1995, Proceedings of 1995 IEEE International Symposium on Information Theory.

[5]  W. Rudin.  Real and complex analysis, 3rd ed. , 1987 .

[6]  C. R. Rao,et al.  Information and the Accuracy Attainable in the Estimation of Statistical Parameters , 1992 .

[7]  A. Hero,et al.  A characterization of the multivariate distributions maximizing Renyi entropy , 2002, Proceedings IEEE International Symposium on Information Theory,.

[8]  Rory A. Fisher,et al.  Theory of Statistical Estimation , 1925, Mathematical Proceedings of the Cambridge Philosophical Society.

[9]  E. Lutwak,et al.  Sharp Affine LP Sobolev Inequalities , 2002 .

[10]  Erdal Arikan.  An inequality on guessing and its application to sequential decoding , 1996, IEEE Trans. Inf. Theory.

[11]  E. Lutwak,et al.  Sharp Affine L P Sobolev Inequalities , 2002 .

[12]  E. Lutwak,et al.  The Cramer-Rao inequality for star bodies , 2002 .

[13]  F. Götze.  Differential-geometrical methods in statistics. Lecture notes in statistics - A. Shun-ichi. , 1987 .

[14]  Shun-ichi Amari,et al.  Differential-geometrical methods in statistics , 1985 .

[15]  A. J. Stam.  Some Inequalities Satisfied by the Quantities of Information of Fisher and Shannon , 1959, Inf. Control..

[16]  Erwin Lutwak,et al.  L p Affine Isoperimetric Inequalities , 2000 .

[17]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[18]  Jonathan M. Borwein,et al.  Maximum Entropy Reconstruction Using Derivative Information, Part 1: Fisher Information and Convex Duality , 1996, Math. Oper. Res..

[19]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[20]  H. Cramér.  Mathematical methods of statistics , 1947 .

[21]  J. L. Doob,et al.  Probability and statistics , 1934 .

[22]  Erwin Lutwak,et al.  Moment-Entropy Inequalities for a Random Vector , 2007, IEEE Transactions on Information Theory.

[23]  Manuel del Pino,et al.  Best constants for Gagliardo–Nirenberg inequalities and applications to nonlinear diffusions☆ , 2002 .