Optimal rates for k-NN density and mode estimation

We present two related contributions of independent interest: (1) high-probability finite sample rates for k-NN density estimation, and (2) practical mode estimators - based on k-NN - which attain minimax-optimal rates under surprisingly general distributional conditions.

[1]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[2]  H. Chernoff.  Estimation of the mode , 1964 .

[3]  C. Quesenberry,et al.  A nonparametric estimate of a multivariate density function , 1965 .

[4]  U. Grenander.  Some Direct Estimates of the Mode , 1965 .

[5]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[6]  L. Devroye,et al.  The Strong Uniform Consistency of Nearest Neighbor Density Estimates. , 1977 .

[7]  J. Yackel,et al.  Large Sample Properties of Nearest Neighbor Density Function Estimators , 1977 .

[8]  L. Devroye.  Recursive estimation of the mode of a multivariate density , 1979 .

[9]  W. Eddy.  Optimum Kernel Estimators of the Mode , 1980 .

[10]  R. Varga,et al.  Proof of Theorem 2 , 1983 .

[11]  D. Donoho,et al.  Geometrizing Rates of Convergence, III , 1991 .

[12]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  P. Hall,et al.  On the minimisation of $L\sp p$ error in mode estimation , 1995 .

[14]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Gábor Lugosi,et al.  Introduction to Statistical Learning Theory , 2004, Advanced Lectures on Machine Learning.

[16]  G. Biau,et al.  On the asymptotic properties of a simple estimate of the Mode , 2004 .

[17]  Jussi Klemelä.  Adaptive estimation of the mode of a multivariate density , 2005 .

[18]  Surajit Ray,et al.  A Nonparametric Statistical Approach to Clustering via Mode Identification , 2007, J. Mach. Learn. Res..

[19]  Ulrike von Luxburg,et al.  Optimal construction of k-nearest-neighbor graphs for identifying noisy clusters , 2009, Theoretical Computer Science.

[20]  Sanjoy Dasgupta,et al.  Rates of convergence for the cluster tree , 2010, NIPS.

[21]  Ulrike von Luxburg,et al.  Pruning nearest neighbor cluster trees , 2011, ICML.

[22]  L. Devroye,et al.  A weighted k-nearest neighbor density estimate for geometric inference , 2011 .

[23]  Leonidas J. Guibas,et al.  Persistence-Based Clustering in Riemannian Manifolds , 2013, JACM.

[24]  Sivaraman Balakrishnan,et al.  Cluster Trees on Manifolds , 2013, NIPS.

[25]  Larry A. Wasserman,et al.  Non‐parametric inference for density modes , 2013, ArXiv.

[26]  Ulrike von Luxburg,et al.  Consistent Procedures for Cluster Tree Estimation and Pruning , 2014, IEEE Transactions on Information Theory.

[27]  David Mason,et al.  On the Estimation of the Gradient Lines of a Density and the Consistency of the Mean-Shift Algorithm , 2016, J. Mach. Learn. Res..