The Cramér-Rao Inequality on Singular Statistical Models

We introduce the notions of essential tangent space and reduced Fisher metric and extend the classical Cramer-Rao inequality to 2-integrable (possibly singular) statistical models for general \(\varphi \)-estimators, where \(\varphi \) is a V-valued feature function and V is a topological vector space. We show the existence of a \(\varphi \)-efficient estimator on strictly singular statistical models associated with a finite sample space and on a class of infinite dimensional exponential models that have been discovered by Fukumizu. We conclude that our general Cramer-Rao inequality is optimal.

[1]  H. Cramér.  Mathematical methods of statistics , 1947 .

[2]  K. Do,et al.  Efficient and Adaptive Estimation for Semiparametric Models. , 1994 .

[3]  O. Barndorff-Nielsen.  Information And Exponential Families , 1970 .

[4]  J. Neveu.  Bases mathématiques du calcul des probabilités , 1966 .

[5]  K. Fukumizu.  Algebraic and Geometric Methods in Statistics: Exponential manifold by reproducing kernel Hilbert spaces , 2009 .

[6]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[7]  Jean-François Bercher,et al.  On generalized Cramér–Rao inequalities, generalized Fisher information and characterizations of generalized q-Gaussian distributions , 2012, ArXiv.

[8]  C. R. Rao,et al.  Information and the Accuracy Attainable in the Estimation of Statistical Parameters , 1992 .

[9]  S. Lang.  Fundamentals of differential geometry , 1998 .

[10]  Shun-ichi Amari,et al.  Information Geometry and Its Applications , 2016 .

[11]  Sumio Watanabe,et al.  Almost All Learning Machines are Singular , 2007, 2007 IEEE Symposium on Foundations of Computational Intelligence.

[12]  Shun-ichi Amari,et al.  Differential geometrical theory of statistics , 1987 .

[13]  N. Ay,et al.  Information geometry and sufficient statistics , 2012, Probability Theory and Related Fields.

[14]  R. F.,et al.  Mathematical Statistics , 1944, Nature.

[15]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[16]  渡邊 澄夫.  Algebraic geometry and statistical learning theory , 2009 .

[17]  Hông Vân Lê,et al.  The uniqueness of the Fisher metric as information metric , 2013, 1306.1465.

[18]  Aapo Hyvärinen,et al.  Density Estimation in Infinite Dimensional Exponential Families , 2013, J. Mach. Learn. Res..

[19]  Giovanni Pistone,et al.  An Infinite-Dimensional Geometric Structure on the Space of all the Probability Measures Equivalent to a Given One , 1995 .

[20]  Nicolas Boumal,et al.  On Intrinsic Cramér-Rao Bounds for Riemannian Submanifolds and Quotient Manifolds , 2013, IEEE Transactions on Signal Processing.

[21]  L. Wasserman.  All of Nonparametric Statistics , 2005 .

[22]  Shun-ichi Amari,et al.  Methods of information geometry , 2000 .

[23]  Nihat Ay,et al.  Parametrized measure models , 2015, Bernoulli.