Strong Universal Consistent Estimate of the Minimum Mean Squared Error

Consider the regression problem with a response variable Y and a feature vector X. For the regression function m(x) = E{Y ∣X = x}, we introduce new and simple estimators of the minimum mean squared error Mean squared error—( Minimum mean squared error—( \({L}^{{\ast}} = \mathbf{E}\{{(Y - m(\mathbf{X}))}^{2}\}\), and prove their strong consistenciesConsistency—(. We bound the rate of convergenceRate of convergence, too.

[1]  S. Dudoit,et al.  Asymptotics of cross-validated risk estimation in estimator selection and performance assessment , 2005 .

[2]  Amaury Lendasse,et al.  Residual variance estimation using a nearest neighbor statistic , 2010, J. Multivar. Anal..

[3]  Györfi László,et al.  The estimation problem of minimum mean squared error , 2003 .

[4]  Harro Walk,et al.  Strong Laws of Large Numbers by Elementary Tauberian Arguments , 2005 .

[5]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[6]  A. Tsybakov,et al.  Nonparametric recursive variance estimation , 1995 .

[7]  B. Efron,et al.  The Jackknife Estimate of Variance , 1981 .

[8]  E. Landau,et al.  Über die Bedeutung einiger neuen Grenzwertsätze der Herren Hardy und Axer , 2022 .

[9]  G. Lugosi,et al.  On the Strong Universal Consistency of Nearest Neighbor Regression Function Estimates , 1994 .

[10]  Ulrich Stadtmüller,et al.  Estimation of Heteroscedasticity in Regression Analysis , 1987 .

[11]  Michael H. Neumann.  Fully Data-Driven Nonparametric Variance Estimators , 1994 .

[12]  P. Ferrario,et al.  Nonparametric partitioning estimation of residual and local variance based on first and second nearest neighbours , 2012 .

[13]  Michel Verleysen,et al.  Residual variance estimation in machine learning , 2009, Neurocomputing.

[14]  Amaury Lendasse,et al.  On Nonparametric Residual Variance Estimation , 2008, Neural Processing Letters.

[15]  Y. Chow.  Local Convergence of Martingales and the Law of Large Numbers , 1965 .

[16]  Anton Schick,et al.  Estimating the error variance in nonparametric regression by a covariate-matched u-statistic , 2003 .