A universal strong law of large numbers for conditional expectations via nearest neighbors

For k"n-nearest neighbor estimates of a regression Y on X (d-dimensional random vector X, integrable real random variable Y) based on observed independent copies of (X,Y), strong universal pointwise consistency is shown, i.e., strong consistency P"X-almost everywhere for general distribution of (X,Y). With tie-breaking by indices, this means validity of a universal strong law of large numbers for conditional expectations E(Y|X=x).

[1]  N. Etemadi.  An elementary proof of the strong law of large numbers , 1981 .

[2]  E. F. Schuster,et al.  On a universal strong law of large numbers for conditional expectations , 1998 .

[3]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[4]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[5]  L. Györfi,et al.  Strong universal pointwise consistency of some regression function estimates , 1999 .

[6]  G. Lugosi,et al.  On the Strong Universal Consistency of Nearest Neighbor Regression Function Estimates , 1994 .

[7]  Winfried Stute,et al.  ON ALMOST SURE CONVERGENCE OF CONDITIONAL EMPIRICAL DISTRIBUTION FUNCTIONS , 1986 .

[8]  L. Devroye.  On the Almost Everywhere Convergence of Nonparametric Regression Function Estimates , 1981 .

[9]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[10]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[11]  A. Krzyżak,et al.  Distribution-Free Pointwise Consistency of Kernel Regression Estimate , 1984 .

[12]  Harro Walk.  Strong Universal Pointwise Consistency of Recursive Regression Estimates , 2001 .

[13]  B. Efron,et al.  The Jackknife Estimate of Variance , 1981 .

[14]  J. Steele.  An Efron-Stein inequality for nonsymmetric statistics , 1986 .

[15]  Paul H. Algoet,et al.  Universal Schemes for Learning the Best Nonlinear Predictor Given the Infinite Past and Side Information , 1999, IEEE Trans. Inf. Theory.

[16]  Richard L. Wheeden.  Measure and integral , 1977 .

[17]  H. Mukerjee.  A strong law of large numbers for nonparametric regression , 1989 .