Foundations of Comparison-Based Hierarchical Clustering

We address the classical problem of hierarchical clustering, but in a framework where one does not have access to a representation of the objects or their pairwise similarities. Instead, we assume that only a set of comparisons between objects is available, that is, statements of the form "objects $i$ and $j$ are more similar than objects $k$ and $l$." Such a scenario is commonly encountered in crowdsourcing applications. The focus of this work is to develop comparison-based hierarchical clustering algorithms that do not rely on the principles of ordinal embedding. We show that single and complete linkage are inherently comparison-based and we develop variants of average linkage. We provide statistical guarantees for the different methods under a planted hierarchical partition model. We also empirically demonstrate the performance of the proposed approaches on several datasets.

[1]  Svante Janson,et al.  The infamous upper tail , 2002, Random Struct. Algorithms.

[2]  Ulrike von Luxburg,et al.  Comparison-Based Nearest Neighbor Search , 2017, AISTATS.

[3]  Sanjoy Dasgupta,et al.  Interactive Bayesian Hierarchical Clustering , 2016, ICML.

[4]  Gordon D. A. Brown,et al.  Absolute identification by relative judgment. , 2005, Psychological review.

[5]  Sivaraman Balakrishnan,et al.  Noise Thresholds for Spectral Clustering , 2011, NIPS.

[6]  Ulrike von Luxburg,et al.  Local Ordinal Embedding , 2014, ICML.

[7]  Hannes Heikinheimo,et al.  Crowdsourced Nonparametric Density Estimation Using Relative Distances , 2015, HCOMP.

[8]  Ulrike von Luxburg,et al.  Kernel functions based on triplet similarity comparisons , 2016, ArXiv.

[9]  Forrest W. Young.  Multidimensional Scaling: History, Theory, and Applications , 1987 .

[10]  R. Shepard.  The analysis of proximities: Multidimensional scaling with an unknown distance function. I. , 1962 .

[11]  Ulrike von Luxburg,et al.  Comparison-Based Random Forests , 2018, ICML.

[12]  Claire Mathieu,et al.  Hierarchical Clustering , 2017, SODA.

[13]  R. Shepard.  The analysis of proximities: Multidimensional scaling with an unknown distance function. II , 1962 .

[14]  Robert D. Nowak,et al.  Low-dimensional embedding using adaptively selected ordinal data , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[15]  Ehsan Amid,et al.  Multiview Triplet Embedding: Learning Attributes in Multiple Maps , 2015, ICML.

[16]  Aurko Roy,et al.  Hierarchical Clustering via Spreading Metrics , 2016, NIPS.

[17]  Ulrike von Luxburg,et al.  Dimensionality estimation without distances , 2015, AISTATS.

[18]  Antti Ukkonen,et al.  Crowdsourced Correlation Clustering with Relative Distance Comparisons , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[19]  David Kempe,et al.  Adaptive Hierarchical Clustering Using Ordinal Queries , 2017, SODA.

[20]  Yudong Chen,et al.  Statistical-Computational Tradeoffs in Planted Problems and Submatrix Localization with a Growing Number of Clusters and Submatrices , 2014, J. Mach. Learn. Res..

[21]  Sanjoy Dasgupta,et al.  A cost function for similarity-based hierarchical clustering , 2015, STOC.

[22]  Kilian Q. Weinberger,et al.  Stochastic triplet embedding , 2012, 2012 IEEE International Workshop on Machine Learning for Signal Processing.

[23]  Elizaveta Levina,et al.  On semidefinite relaxations for the block model , 2014, ArXiv.

[24]  Amin Karbasi,et al.  Comparison Based Learning from Weak Oracles , 2018, AISTATS.

[25]  Avrim Blum,et al.  Foundations of Data Science , 2020 .

[26]  Ulrike von Luxburg,et al.  Uniqueness of Ordinal Embedding , 2014, COLT.

[27]  M. Cugmas,et al.  On comparing partitions , 2015 .

[28]  Ulrike von Luxburg,et al.  Lens Depth Function and k-Relative Neighborhood Graph: Versatile Tools for Ordinal Data Analysis , 2016, J. Mach. Learn. Res..

[29]  Adam Tauman Kalai,et al.  Adaptively Learning the Crowd Kernel , 2011, ICML.

[30]  Ery Arias-Castro,et al.  Some theory for ordinal embedding , 2015, 1501.02861.

[31]  Robert D. Nowak,et al.  Active Ranking using Pairwise Comparisons , 2011, NIPS.

[32]  Patrick J. F. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 2003 .

[33]  Lalit Jain,et al.  Finite Sample Prediction and Recovery Bounds for Ordinal Embedding , 2016, NIPS.

[34]  Benjamin Moseley,et al.  Approximation Bounds for Hierarchical Clustering: Average Linkage, Bisecting K-means, and Local Search , 2017, NIPS.

[35]  David J. Kriegman,et al.  Generalized Non-metric Multidimensional Scaling , 2007, AISTATS.

[36]  Ulrike von Luxburg,et al.  Consistent Procedures for Cluster Tree Estimation and Pruning , 2014, IEEE Transactions on Information Theory.

[37]  Hannes Heikinheimo,et al.  The Crowd-Median Algorithm , 2013, HCOMP.

[38]  Subhransu Maji,et al.  Jointly Learning Multiple Measures of Similarities from Triplet Comparisons , 2015 .

[39]  J. Hartigan.  Consistency of Single Linkage for High-Density Clusters , 1981 .

[40]  Katherine A. Heller,et al.  Bayesian hierarchical clustering , 2005, ICML.