Nonparametric function recovering from noisy observations

Abstract We consider the nonparametric regression model Yi=g(xi)+ζi, where g is a bounded function, over the interval [0,1], to be estimated, xi's are nonrandom and ζi's are independent identically distributed random variables with Eζi=0. This paper studies the behavior of the general family of nonparametric estimates gn(x)=Σi=1nYiwni(x), where the weight functions {wni} are of the form wni(x)=wni(x;x1,…,xn), i=1,…,n. The family of estimates includes all known estimates proposed by Priestley and Chao (1972), Clark (1977), Gasser and Muller (1979), Cheng and Lin (1981) as well as Georgiev (1984b, 1985). Sufficient conditions for mean square and complete convergence are derived. New results for the Priestley-Chao and Gasser-Muller-Cheng-Lin estimates are obtained. Also proposed is a class of new nearest neighbor estimates of g. Finally, a simulation experiment demonstrates the remarkable success of the nearest neighbor technique with bandwidth depending on the local density of the design points.

[1]  J. Rice.  Bandwidth Choice for Nonparametric Regression , 1984 .

[2]  L. Devroye,et al.  Distribution-Free Consistency Results in Nonparametric Discrimination and Regression Function Estimation , 1980 .

[3]  M. E. Muller,et al.  A Note on the Generation of Random Normal Deviates , 1958 .

[4]  A. Krzyżak,et al.  Distribution-Free Pointwise Consistency of Kernel Regression Estimate , 1984 .

[5]  G. Collomb.  Estimation Non-paramétrique de la Régression: Revue Bibliographique@@@Estimation Non-parametrique de la Regression: Revue Bibliographique , 1981 .

[6]  M. Priestley,et al.  Non‐Parametric Function Fitting , 1972 .

[7]  Pi-Erh Lin,et al.  Nonparametric estimation of a regression function , 1981 .

[8]  B. Silverman,et al.  Spline Smoothing: The Equivalent Variable Kernel Method , 1984 .

[9]  J. K. Benedetti.  On the Nonparametric Estimation of Regression Functions , 1977 .

[10]  Gérard Collomb,et al.  Nonparametric regression: An up–to–date bibliography , 1985 .

[11]  Alexander A. Georgiev.  The strong pointwise convergence of nearest neighbor function fitting algorithm with applications to system identification , 1985, Kybernetika.

[12]  C. Spiegelman,et al.  Consistent Window Estimation in Nonparametric Regression , 1980 .

[13]  S. Yakowitz,et al.  A comparison of kriging with nonparametric regression methods , 1985 .

[14]  Ker-Chau Li.  Consistency for Cross-Validated Nearest Neighbor Estimates in Nonparametric Regression , 1984 .

[15]  Local Properties of Function Fitting Estimates with Application to System Identification , 1985 .

[16]  W. Wong.  On the Consistency of Cross-Validation in Kernel Nonparametric Regression , 1983 .

[17]  S. Yakowitz,et al.  Contributions to the Theory of Nonparametric Regression, with Application to System Identification , 1979 .

[18]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[19]  Alexander A. Georgiev,et al.  On the recovery of functions and their derivatives from imperfect measurements , 1984, IEEE Transactions on Systems, Man, and Cybernetics.

[20]  R. M. Clark.  Non‐Parametric Estimation of a Smooth Regression Function , 1977 .

[21]  H. Müller,et al.  Kernel estimation of regression functions , 1979 .

[22]  W. Stout.  Almost sure convergence , 1974 .

[23]  A. Georgiev.  Kernel estimates of functions and their derivatives with applications , 1984 .

[24]  G. Bennett.  Probability Inequalities for the Sum of Independent Random Variables , 1962 .