A new method for estimation and model selection:$$\rho $$ρ-estimation

The aim of this paper is to present a new estimation procedure that can be applied in various statistical frameworks including density and regression and which leads to both robust and optimal (or nearly optimal) estimators. In density estimation, they asymptotically coincide with the celebrated maximum likelihood estimators at least when the statistical model is regular enough and contains the true density to estimate. For very general models of densities, including non-compact ones, these estimators are robust with respect to the Hellinger distance and converge at optimal rate (up to a possible logarithmic factor) in all cases we know. In the regression setting, our approach improves upon the classical least squares in many respects. In simple linear regression for example, it provides an estimation of the coefficients that are both robust to outliers and simultaneously rate-optimal (or nearly rate-optimal) for a large class of error distributions including Gaussian, Laplace, Cauchy and uniform among others.

[1]  Edmund Taylor Whittaker,et al.  A Course of Modern Analysis , 2021 .

[2]  A. Kolmogorov,et al.  Entropy and "-capacity of sets in func-tional spaces , 1961 .

[3]  L. Lecam.  On the Assumptions Used to Prove Asymptotic Normality of Maximum Likelihood Estimates , 1970 .

[4]  J. Hájek.  Local asymptotic minimax and admissibility in estimation , 1972 .

[5]  L. Lecam.  Convergence of Estimates Under Dimensionality Restrictions , 1973 .

[6]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[7]  R. Z. Khasʹminskiĭ,et al.  Statistical estimation : asymptotic theory , 1981 .

[8]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[9]  Lucien Birgé.  Approximation dans les espaces métriques et théorie de l'estimation , 1983 .

[10]  R. Dudley.  A course on empirical processes , 1984 .

[11]  Lucien Birgé.  Stabilité et instabilité du risque minimax pour des variables indépendantes équidistribuées , 1984 .

[12]  L. L. Cam,et al.  Asymptotic Methods In Statistical Decision Theory , 1986 .

[13]  L. L. Cam,et al.  Maximum likelihood : an introduction , 1990 .

[14]  Grace L. Yang,et al.  Asymptotics In Statistics , 1990 .

[15]  Andrew R. Barron,et al.  Complexity Regularization with Application to Artificial Neural Networks , 1991 .

[16]  P. Massart,et al.  Rates of convergence for minimum contrast estimators , 1993 .

[17]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[18]  P. Massart,et al.  From Model Selection to Adaptive Estimation , 1997 .

[19]  P. Massart,et al.  Minimum contrast estimators on sieves: exponential bounds and rates of convergence , 1998 .

[20]  A. V. D. Vaart,et al.  Asymptotic Statistics: Frontmatter , 1998 .

[21]  P. Massart,et al.  Risk bounds for model selection via penalization , 1999 .

[22]  Yuhong Yang,et al.  Information-theoretic determination of minimax rates of convergence , 1999 .

[23]  A. V. D. Vaart,et al.  Convergence rates of posterior distributions , 2000 .

[24]  Y. Baraud.  Model selection for regression on a random design , 2002 .

[25]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[26]  E. Rio,et al.  Concentration around the mean for maxima of empirical processes , 2005, math/0506594.

[27]  V. Koltchinskii,et al.  Concentration inequalities and asymptotic results for ratio type empirical processes , 2006, math/0606788.

[28]  L. Cam.  Asymptotic Normality of Experiments , 2006 .

[29]  V. Koltchinskii.  Local Rademacher complexities and oracle inequalities in risk minimization , 2006, 0708.0083.

[30]  V. Koltchinskii.  Rejoinder: Local Rademacher complexities and oracle inequalities in risk minimization , 2006, 0708.0135.

[31]  L. Birge,et al.  Model selection via testing: an alternative to (penalized) maximum likelihood estimators , 2006 .

[32]  P. Massart,et al.  Concentration inequalities and model selection , 2007 .

[33]  P. Massart,et al.  Risk bounds for statistical learning , 2007, math/0702683.

[34]  P. Massart,et al.  Minimal Penalties for Gaussian Model Selection , 2007 .

[35]  A. W. van der Vaart,et al.  A note on bounds for VC dimensions. , 2009, Institute of Mathematical Statistics collections.

[36]  Y. Baraud.  Estimator selection with respect to Hellinger-type risks , 2009, 0905.1486.

[37]  M. Sart.  Model selection for Poisson processes with covariates , 2011, 1112.5634.

[38]  Jean-Yves Audibert,et al.  Robust linear least squares regression , 2010, 1010.0074.

[39]  M. Sart.  Estimation of the transition density of a Markov chain , 2012, 1210.5165.

[40]  L. Birgé.  Robust tests for model selection , 2013 .

[41]  M. Sart.  Robust estimation on a parametric model via testing , 2013, 1308.2927.