A Unifying Approach to Nonparametric Regression Estimation

Abstract There exists a variety of nonparametric regression estimators, with cubic smoothing splines, k-nearest-neighbor (k-NN) estimators, and various types of kernel estimators among the most popular. A class of kernel estimators with local bandwidth depending on the density of the design points is introduced, where the degree of design adaptation may be expressed by a single parameter α in [0, 1]. The adaptation to the design is such that the bandwidth is made larger, to a degree depending on α, where the design is thin. Special values of this parameter correspond approximately to the ordinary (fixed-width) kernel estimator, the smoothing spline, and the k-NN estimator. Hence this method offers a synthesis of some classical methods. The same method allows estimation of derivatives by using appropriate kernels. The influence of this degree of design adaptation on the integrated mean squared error is investigated. There is no uniformly optimal solution. The optimal solution depends in a complex way on de...

[1]  Ulrich Stadtmüller,et al.  Estimation of Heteroscedasticity in Regression Analysis , 1987 .

[2]  H. Müller,et al.  Variable Bandwidth Kernel Estimators of Regression Curves , 1987 .

[3]  P. Speckman.  Spline Smoothing and Optimal Rates of Convergence in Nonparametric Regression Models , 1985 .

[4]  R. John,et al.  Boundary modification for kernel regression , 1984 .

[5]  G. Wahba.  Smoothing noisy data with spline functions , 1975 .

[6]  T. Gasser,et al.  Residual variance and residual pattern in nonlinear regression , 1986 .

[7]  Y. Mack,et al.  Local Properties of k-NN Regression Estimates , 1981 .

[8]  M. Hutchinson,et al.  Smoothing noisy data with spline functions , 1985 .

[9]  G. Collomb.  Estimation Non-paramétrique de la Régression: Revue Bibliographique@@@Estimation Non-parametrique de la Regression: Revue Bibliographique , 1981 .

[10]  B. Silverman,et al.  Spline Smoothing: The Equivalent Variable Kernel Method , 1984 .

[11]  John Rice,et al.  Integrated mean squared error of a smoothing spline , 1981 .

[12]  Bernard W. Silverman,et al.  A Fast and Efficient Cross-Validation Method for Smoothing Parameter Choice in Spline Regression , 1984 .

[13]  H. Müller,et al.  Kernels for Nonparametric Curve Estimation , 1985 .

[14]  G. S. Watson,et al.  Smooth regression analysis , 1964 .

[15]  Thomas M. Cover,et al.  Estimation by the nearest neighbor rule , 1968, IEEE Trans. Inf. Theory.

[16]  E. Nadaraya.  On Estimating Regression , 1964 .

[17]  W. Härdle,et al.  Optimal Bandwidth Selection in Nonparametric Regression Function Estimation , 1985 .

[18]  B. Silverman,et al.  Some Aspects of the Spline Smoothing Approach to Non‐Parametric Regression Curve Fitting , 1985 .

[19]  H. Müller,et al.  Estimating regression functions and their derivatives by the kernel method , 1984 .

[20]  H. Müller,et al.  Kernel estimation of regression functions , 1979 .

[21]  Pi-Erh Lin,et al.  Nonparametric estimation of a regression function , 1981 .

[22]  C. Reinsch.  Smoothing by spline functions , 1967 .