How Far are Automatically Chosen Regression Smoothing Parameters from their Optimum

Abstract We address the problem of smoothing parameter selection for nonparametric curve estimators in the specific context of kernel regression estimation. Call the “optimal bandwidth” the minimizer of the average squared error. We consider several automatically selected bandwidths that approximate the optimum. How far are the automatically selected bandwidths from the optimum? The answer is studied theoretically and through simulations. The theoretical results include a central limit theorem that quantifies the convergence rate and gives the differences asymptotic distribution. The convergence rate turns out to be excruciatingly slow. This is not too disappointing, because this rate is of the same order as the convergence rate of the difference between the minimizers of the average squared error and the mean average squared error. In some simulations by John Rice, the selectors considered here performed quite differently from each other. We anticipated that these differences would be reflected in differ...

[1]  P. Whittle,et al.  Bounds for the Moments of Linear and Quadratic Forms in Independent Variables , 1960 .

[2]  E. Nadaraya.  On Estimating Regression , 1964 .

[3]  G. S. Watson,et al.  Smooth regression analysis , 1964 .

[4]  H. Akaike.  Statistical predictor identification , 1970 .

[5]  M. Priestley,et al.  Non‐Parametric Function Fitting , 1972 .

[6]  C. L. Mallows.  Some comments on C_p , 1973 .

[7]  H. Akaike.  A new look at the statistical model identification , 1974 .

[8]  G. Wahba.  Smoothing noisy data with spline functions , 1975 .

[9]  R. M. Clark.  A calibration curve for radiocarbon dates , 1975, Antiquity.

[10]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[11]  H. Müller,et al.  Kernel estimation of regression functions , 1979 .

[12]  R. Shibata.  An optimal selection of regression variables , 1981 .

[13]  M. Rudemo.  Empirical Choice of Histograms and Kernel Density Estimators , 1982 .

[14]  J. Rice.  Bandwidth Choice for Nonparametric Regression , 1984 .

[15]  A. Bowman.  An alternative method of cross-validation for the smoothing of density estimates , 1984 .

[16]  P. Hall.  Central limit theorem for integrated square error of multivariate nonparametric density estimators , 1984 .

[17]  Ker-Chau Li,et al.  From Stein's Unbiased Risk Estimates to the Method of Generalized Cross Validation , 1985 .

[18]  W. Härdle,et al.  Asymptotic nonequivalence of some bandwidth selectors in nonparametric regression , 1985 .

[19]  G. Wahba.  A Comparison of GCV and GML for Choosing the Smoothing Parameter in the Generalized Spline Smoothing Problem , 1985 .

[20]  W. Härdle,et al.  Optimal Bandwidth Selection in Nonparametric Regression Function Estimation , 1985 .

[21]  James Stephen Marron,et al.  Random approximations to some measures of accuracy in nonparametric curve estimation , 1986 .

[22]  J. Marron,et al.  Extent to which least-squares cross-validation minimises integrated square error in nonparametric density estimation , 1987 .

[23]  James Stephen Marron,et al.  On the Amount of Noise Inherent in Bandwidth Selection for a Kernel Density Estimator , 1987 .

[24]  D. W. Scott,et al.  Biased and Unbiased Cross-Validation in Density Estimation , 1987 .

[25]  D. Pollard,et al.  $U$-Processes: Rates of Convergence , 1987 .

[26]  Ker-Chau Li,et al.  Asymptotic Optimality for $C_p, C_L$, Cross-Validation and Generalized Cross-Validation: Discrete Index Set , 1987 .