Density estimation via exponential model selection

We address the problem of estimating some unknown density on a bounded interval using some exponential models of piecewise polynomials. We consider a finite collection of such models based on a family of partitions. And we study the maximum-likelihood estimator built on a data-driven selected model among this collection. In doing so, we validate Akaike's criterion if the partitions that we consider are regular and we modify it if the partitions are irregular. We deduce the rate of convergence of the squared Hellinger risk of our estimator in the regular case when the logarithm of the density belongs to some Besov space.

[1]  John C. W. Rayner,et al.  Smooth test of goodness of fit , 1989 .

[2]  P. Massart,et al.  Risk bounds for model selection via penalization , 1999 .

[3]  A. Barron,et al.  APPROXIMATION OF DENSITY FUNCTIONS BY SEQUENCES OF EXPONENTIAL FAMILIES , 1991 .

[4]  P. Massart,et al.  Minimum contrast estimators on sieves: exponential bounds and rates of convergence , 1998 .

[5]  George G. Lorentz,et al.  Constructive Approximation , 1993, Grundlehren der mathematischen Wissenschaften.

[6]  Lawrence D. Brown.  Fundamentals of Statistical Exponential Families , 1987 .

[7]  B. R. Crain.  An Information Theoretic Approach to Approximating a Probability Distribution , 1977 .

[8]  B. R. Crain.  Exponential Models, Maximum Likelihood Estimation, and the Haar Condition , 1976 .

[9]  C. J. Stone.  Uniform Error Bounds Involving Logspline Models , 1989 .

[10]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[11]  P. Massart,et al.  From Model Selection to Adaptive Estimation , 1997 .

[12]  C. J. Stone,et al.  Large-Sample Inference for Log-Spline Models , 1990 .

[13]  Kent E. Morrison.  q-Exponential Families , 2004, Electron. J. Comb..

[14]  B. R. Crain.  More on Estimation of Distributions Using Orthogonal Expansions , 1976 .

[15]  Yuhong Yang,et al.  An Asymptotic Property of Model Selection Criteria , 1998, IEEE Trans. Inf. Theory.

[16]  Edmund Taylor Whittaker,et al.  A Course of Modern Analysis , 2021 .

[17]  B. R. Crain,et al.  Estimation of Distributions Using Orthogonal Expansions , 1974 .

[18]  Gwenaelle Castellan.  Sélection d'histogrammes ou de modèles exponentiels de polynômes par morceaux à l'aide d'un critère de type Akaike , 2000 .

[19]  G. Lugosi,et al.  Nonasymptotic universal smoothing factors, kernel complexity and yatracos classes , 1997 .

[20]  J. Lamperti.  ON CONVERGENCE OF STOCHASTIC PROCESSES , 1962 .

[21]  S. Portnoy.  Asymptotic Behavior of Likelihood Methods for Exponential Families when the Number of Parameters Tends to Infinity , 1988 .

[22]  Lucien Birgé.  Approximation dans les espaces métriques et théorie de l'estimation , 1983 .

[23]  O. Bousquet.  A Bennett concentration inequality and its application to suprema of empirical processes , 2002 .

[24]  Wavelet density estimation by approximation of log-densities , 1996 .

[25]  N. Čencov.  Statistical Decision Rules and Optimal Inference , 2000 .