Uniform Convergence Rates for Kernel Density Estimation

Kernel density estimation (KDE) is a popular nonparametric density estimation method. We (1) derive finite-sample high-probability density estimation bounds for multivariate KDE under mild density assumptions which hold uniformly in x ∈ R and bandwidth matrices. We apply these results to (2) mode, (3) density level set, and (4) class probability estimation and attain optimal rates up to logarithmic factors. We then (5) provide an extension of our results under the manifold hypothesis. Finally, we (6) give uniform convergence results for local intrinsic dimension estimation.

[1]  Ingo Steinwart,et al.  Consistency and Rates for Clustering with DBSCAN , 2012, AISTATS.

[2]  Csaba Szepesvári,et al.  Manifold-Adaptive Dimension Estimation , 2007, ICML '07.

[3]  Sanjoy Dasgupta,et al.  Rates of convergence for the cluster tree , 2010, NIPS.

[4]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[5]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  G. Biau,et al.  On the asymptotic properties of a simple estimate of the Mode , 2004 .

[7]  Peter J. Bickel,et al.  Maximum Likelihood Estimation of Intrinsic Dimension , 2004, NIPS.

[8]  B. Cadre.  Kernel estimation of density level sets , 2005, math/0501221.

[9]  Matthias Hein,et al.  Intrinsic dimensionality estimation of submanifolds in Rd , 2005, ICML.

[10]  Balázs Kégl,et al.  Intrinsic Dimension Estimation Using Packing Numbers , 2002, NIPS.

[11]  Sanjoy Dasgupta,et al.  Optimal rates for k-NN density and mode estimation , 2014, NIPS.

[12]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[13]  Michael E. Houle,et al.  Dimensionality, Discriminability, Density and Distance Distributions , 2013, 2013 IEEE 13th International Conference on Data Mining Workshops.

[14]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[15]  Jose A. Costa,et al.  Estimating Local Intrinsic Dimension with k-Nearest Neighbor Graphs , 2005, IEEE/SP 13th Workshop on Statistical Signal Processing, 2005.

[16]  Philippe Rigollet,et al.  Generalization Error Bounds in Semi-supervised Classification Under the Cluster Assumption , 2006, J. Mach. Learn. Res..

[17]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Larry A. Wasserman,et al.  Non‐parametric inference for density modes , 2013, ArXiv.

[19]  Samory Kpotufe,et al.  Modal-set estimation with an application to clustering , 2016, AISTATS.

[20]  E. Giné,et al.  Rates of strong uniform consistency for multivariate kernel density estimators , 2002 .

[21]  B. Silverman,et al.  Using Kernel Density Estimates to Investigate Multimodality , 1981 .

[22]  Uwe Einmahl,et al.  Uniform in bandwidth consistency of kernel-type function estimators , 2005 .

[23]  A. Rinaldo,et al.  Generalized density clustering , 2009, 0907.3454.

[24]  W. Eddy.  Optimum Kernel Estimators of the Mode , 1980 .

[25]  Surajit Ray,et al.  A Nonparametric Statistical Approach to Clustering via Mode Identification , 2007, J. Mach. Learn. Res..

[26]  A. Cuevas,et al.  A plug-in approach to support estimation , 1997 .

[27]  Heinrich Jiang,et al.  Density Level Set Estimation on Manifolds with DBSCAN , 2017, ICML.

[28]  J. Carmichael,et al.  FINDING NATURAL CLUSTERS , 1968 .

[29]  Gábor Lugosi,et al.  Introduction to Statistical Learning Theory , 2004, Advanced Lectures on Machine Learning.

[30]  H. Chernoff.  Estimation of the mode , 1964 .

[31]  M. Hazelton.  Variable kernel density estimation , 2003 .

[32]  J. Simonoff.  Smoothing Methods in Statistics , 1998 .

[33]  David Mason,et al.  On the Estimation of the Gradient Lines of a Density and the Consistency of the Mean-Shift Algorithm , 2016, J. Mach. Learn. Res..

[34]  Ingo Steinwart,et al.  Adaptive Density Level Set Clustering , 2011, COLT.

[35]  A. Tsybakov,et al.  Fast learning rates for plug-in classifiers , 2007, 0708.2321.

[36]  Sivaraman Balakrishnan,et al.  Cluster Trees on Manifolds , 2013, NIPS.

[37]  Hannu Oja,et al.  Classification Based on Hybridization of Parametric and Nonparametric Classifiers , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[39]  Stephen Smale,et al.  Finding the Homology of Submanifolds with High Confidence from Random Samples , 2008, Discret. Comput. Geom..

[40]  Sanjoy Dasgupta,et al.  Rates of Convergence for Nearest Neighbor Classification , 2014, NIPS.

[41]  Larry A. Wasserman,et al.  Minimax Manifold Estimation , 2010, J. Mach. Learn. Res..

[42]  Richard G. Baraniuk,et al.  Random Projections of Smooth Manifolds , 2009, Found. Comput. Math..

[43]  Ken-ichi Kawarabayashi,et al.  Estimating Local Intrinsic Dimensionality , 2015, KDD.

[44]  Dirk P. Kroese,et al.  Kernel density estimation via diffusion , 2010, 1011.2602.

[45]  A. Tsybakov.  On nonparametric estimation of density level sets , 1997 .

[46]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[47]  P. Rigollet,et al.  Fast rates for plug-in estimators of density level sets , 2008 .

[48]  Robert D. Nowak,et al.  Adaptive Hausdorff Estimation of Density Level Sets , 2009, COLT.