Rates of Uniform Consistency for k-NN Regression

We derive high-probability finite-sample uniform rates of consistency for $k$-NN regression that are optimal up to logarithmic factors under mild assumptions. We moreover show that $k$-NN regression adapts to an unknown lower intrinsic dimension automatically. We then apply the $k$-NN regression rates to establish new results about estimating the level sets and global maxima of a function from noisy observations.

[1]  A. Tsybakov.  On nonparametric estimation of density level sets , 1997 .

[2]  Nadia L. Kudraszow,et al.  Uniform consistency of kNN regressors for functional variables , 2013 .

[3]  G. Lugosi,et al.  On the Strong Universal Consistency of Nearest Neighbor Regression Function Estimates , 1994 .

[4]  Rebecca Willett,et al.  Minimax optimal level set estimation , 2005, SPIE Optics + Photonics.

[5]  Joseph P. Romano.  On weak convergence and optimality of kernel density estimates of the mode , 1988 .

[6]  Larry A. Wasserman,et al.  Minimax Manifold Estimation , 2010, J. Mach. Learn. Res..

[7]  P. Cheng.  Strong consistency of nearest neighbor regression function estimators , 1984 .

[8]  Arnaud Guyader,et al.  Rates of Convergence of the Functional $k$-Nearest Neighbor Estimate , 2010, IEEE Transactions on Information Theory.

[9]  Heng Lian,et al.  Convergence of functional k-nearest neighbor regression estimate with functional responses , 2011, Electronic Journal of Statistics.

[10]  Sanjoy Dasgupta,et al.  Optimal rates for k-NN density and mode estimation , 2014, NIPS.

[11]  Sivaraman Balakrishnan,et al.  Cluster Trees on Manifolds , 2013, NIPS.

[12]  Luc Devroye,et al.  The uniform convergence of nearest neighbor regression function estimators and their application in optimization , 1978, IEEE Trans. Inf. Theory.

[13]  Robert D. Nowak,et al.  Adaptive Hausdorff Estimation of Density Level Sets , 2009, COLT.

[14]  Sanjeev R. Kulkarni,et al.  Rates of convergence of nearest neighbor estimation under arbitrary sampling , 1995, IEEE Trans. Inf. Theory.

[15]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[16]  Samory Kpotufe,et al.  k-NN Regression Adapts to Local Intrinsic Dimension , 2011, NIPS.

[17]  Sanjoy Dasgupta,et al.  Rates of convergence for the cluster tree , 2010, NIPS.

[18]  B. Silverman,et al.  Weak and strong uniform consistency of kernel regression estimates , 1982 .

[19]  Heinrich Jiang,et al.  Density Level Set Estimation on Manifolds with DBSCAN , 2017, ICML.

[20]  Heinrich Jiang,et al.  Uniform Convergence Rates for Kernel Density Estimation , 2017, ICML.