Modal-set estimation with an application to clustering

We present a first procedure that can estimate -- with statistical consistency guarantees -- any local-maxima of a density, under benign distributional conditions. The procedure estimates all such local maxima, or $\textit{modal-sets}$, of any bounded shape or dimension, including usual point-modes. In practice, modal-sets can arise as dense low-dimensional structures in noisy data, and more generally serve to better model the rich variety of locally-high-density structures in data. The procedure is then shown to be competitive on clustering applications, and moreover is quite stable to a wide range of settings of its tuning parameter.

[1]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[2]  H. Chernoff.  Estimation of the mode , 1964 .

[3]  J. Carmichael,et al.  FINDING NATURAL CLUSTERS , 1968 .

[4]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[5]  L. Devroye.  Recursive estimation of the mode of a multivariate density , 1979 .

[6]  W. Eddy.  Optimum Kernel Estimators of the Mode , 1980 .

[7]  J. Hartigan.  Consistency of Single Linkage for High-Density Clusters , 1981 .

[8]  L. Hubert,et al.  Comparing partitions , 1985 .

[9]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[11]  Paul E. Green,et al.  K-modes Clustering , 2001, J. Classif..

[12]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  G. Biau,et al.  On the asymptotic properties of a simple estimate of the Mode , 2004 .

[14]  Surajit Ray,et al.  A Nonparametric Statistical Approach to Clustering via Mode Identification , 2007, J. Mach. Learn. Res..

[15]  P. Rigollet,et al.  Fast rates for plug-in estimators of density level sets , 2008 .

[16]  Robert D. Nowak,et al.  Adaptive Hausdorff Estimation of Density Level Sets , 2009, COLT.

[17]  Ulrike von Luxburg,et al.  Optimal construction of k-nearest-neighbor graphs for identifying noisy clusters , 2009, Theoretical Computer Science.

[18]  A. Rinaldo,et al.  Generalized density clustering , 2009, 0907.3454.

[19]  Sanjoy Dasgupta,et al.  Rates of convergence for the cluster tree , 2010, NIPS.

[20]  James Bailey,et al.  Information Theoretic Measures for Clusterings Comparison: Variants, Properties, Normalization and Correction for Chance , 2010, J. Mach. Learn. Res..

[21]  W. Stuetzle,et al.  A Generalized Single Linkage Method for Estimating the Cluster Tree of a Density , 2010 .

[22]  Ashutosh Kumar Singh,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2010 .

[23]  Ingo Steinwart,et al.  Adaptive Density Level Set Clustering , 2011, COLT.

[24]  Ulrike von Luxburg,et al.  Pruning nearest neighbor cluster trees , 2011, ICML.

[25]  Deniz Erdogmus,et al.  Locally Defined Principal Curves and Surfaces , 2011, J. Mach. Learn. Res..

[26]  Ingo Steinwart,et al.  Consistency and Rates for Clustering with DBSCAN , 2012, AISTATS.

[27]  Larry A. Wasserman,et al.  Nonparametric Ridge Estimation , 2012, ArXiv.

[28]  Rebecca Nugent,et al.  Stability of density-based clustering , 2010, J. Mach. Learn. Res..

[29]  Sivaraman Balakrishnan,et al.  Minimax rates for homology inference , 2011, AISTATS.

[30]  Leonidas J. Guibas,et al.  Persistence-Based Clustering in Riemannian Manifolds , 2013, JACM.

[31]  Sivaraman Balakrishnan,et al.  Cluster Trees on Manifolds , 2013, NIPS.

[32]  Larry A. Wasserman,et al.  Non‐parametric inference for density modes , 2013, ArXiv.

[33]  L. Wasserman,et al.  A Comprehensive Approach to Mode Clustering , 2014, 1406.1780.

[34]  L. Wasserman,et al.  Feature Selection For High-Dimensional Clustering , 2014, 1406.2240.

[35]  Sanjoy Dasgupta,et al.  Optimal rates for k-NN density and mode estimation , 2014, NIPS.

[36]  Ulrike von Luxburg,et al.  Consistent Procedures for Cluster Tree Estimation and Pruning , 2014, IEEE Transactions on Information Theory.

[37]  Mikhail Belkin,et al.  Beyond Hartigan Consistency: Merge Distortion Metric for Hierarchical Clustering , 2015, COLT.

[38]  L. Wasserman,et al.  Statistical Inference using the Morse-Smale Complex , 2015, 1506.08826.

[39]  David Mason,et al.  On the Estimation of the Gradient Lines of a Density and the Consistency of the Mean-Shift Algorithm , 2016, J. Mach. Learn. Res..

[40]  Frédéric Chazal,et al.  Robust Topological Inference: Distance To a Measure and Kernel Distance , 2014, J. Mach. Learn. Res..

[41]  Vincent Kanade,et al.  Clustering Algorithms , 2021, Wireless RF Energy Transfer in the Massive IoT Era.