Learning Nearest Neighbor Graphs from Noisy Distance Samples

We consider the problem of learning the nearest neighbor graph of a dataset of n items. The metric is unknown, but we can query an oracle to obtain a noisy estimate of the distance between any pair of items. This framework applies to problem domains where one wants to learn people's preferences from responses commonly modeled as noisy distance judgments. In this paper, we propose an active algorithm to find the graph with high probability and analyze its query complexity. In contrast to existing work that forces Euclidean structure, our method is valid for general metrics, assuming only symmetry and the triangle inequality. Furthermore, we demonstrate efficiency of our method empirically and theoretically, needing only O(n log(n)Delta^-2) queries in favorable settings, where Delta^-2 accounts for the effect of noise. Using crowd-sourced data collected for a subset of the UT Zappos50K dataset, we apply our algorithm to learn which shoes people believe are most similar and show that it beats both an active baseline and ordinal embedding.

[1]  Kristen Grauman,et al.  Semantic Jitter: Dense Supervision for Visual Comparisons via Synthetic Images , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[2]  Shrikanth Narayanan,et al.  Toward Robust Interpretable Human Movement Pattern Analysis in a Workplace Setting , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  Inderjit S. Dhillon,et al.  The Metric Nearness Problem , 2008, SIAM J. Matrix Anal. Appl..

[4]  Lalit Jain,et al.  If it ain't broke, don't fix it: Sparse metric repair , 2017, 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[5]  Kenneth L. Clarkson,et al.  Fast algorithms for the all nearest neighbors problem , 1983, 24th Annual Symposium on Foundations of Computer Science (sfcs 1983).

[6]  Xin-She Yang,et al.  Introduction to Algorithms , 2021, Nature-Inspired Optimization Algorithms.

[7]  Kilian Q. Weinberger,et al.  Stochastic triplet embedding , 2012, 2012 IEEE International Workshop on Machine Learning for Signal Processing.

[8]  R. Shepard.  The analysis of proximities: Multidimensional scaling with an unknown distance function. II , 1962 .

[9]  Yury Lifshits,et al.  Disorder inequality: a combinatorial approach to nearest neighbor search , 2008, WSDM '08.

[10]  Shivaram Kalyanakrishnan,et al.  Information Complexity in Bandit Subset Selection , 2013, COLT.

[11]  Richard G. Baraniuk,et al.  Adaptive Estimation for Approximate k-Nearest-Neighbor Computations , 2019, AISTATS.

[12]  Andreas Krause,et al.  Actively Learning Hemimetrics with Applications to Eliciting User Preferences , 2016, ICML.

[13]  Aurélien Garivier,et al.  Informational confidence bounds for self-normalized averages and applications , 2013, 2013 IEEE Information Theory Workshop (ITW).

[14]  Vandana,et al.  Survey of Nearest Neighbor Techniques , 2010, ArXiv.

[15]  Matthew Malloy,et al.  On Finding the Largest Mean Among Many , 2013, ArXiv.

[16]  David Tse,et al.  Adaptive Monte-Carlo Optimization , 2018, ArXiv.

[17]  David Tse,et al.  Medoids in almost linear time via multi-armed bandits , 2017, AISTATS.

[18]  Paul Barford,et al.  A Learning-Based Approach for IP Geolocation , 2010, PAM.

[19]  Adam Tauman Kalai,et al.  Adaptively Learning the Crowd Kernel , 2011, ICML.

[20]  Mark Crovella,et al.  Hyperbolic Embedding and Routing for Dynamic Graphs , 2009, IEEE INFOCOM 2009.

[21]  Lalit Jain,et al.  Finite Sample Prediction and Recovery Bounds for Ordinal Embedding , 2016, NIPS.

[22]  Michael E. Houle,et al.  Rank-Based Similarity Search: Reducing the Dimensional Dependence , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Kristen Grauman,et al.  Fine-Grained Visual Comparisons with Local Learning , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Pravin M. Vaidya,et al.  AnO(n logn) algorithm for the all-nearest-neighbors Problem , 1989, Discret. Comput. Geom..

[25]  Hanan Samet,et al.  A fast all nearest neighbor algorithm for applications involving large point-clouds , 2007, Comput. Graph..

[26]  Suhas N. Diggavi,et al.  Randomized Algorithms for Comparison-based Search , 2011, NIPS.

[27]  Ulrike von Luxburg,et al.  Lens Depth Function and k-Relative Neighborhood Graph: Versatile Tools for Ordinal Data Analysis , 2016, J. Mach. Learn. Res..

[28]  R. Shepard.  The analysis of proximities: Multidimensional scaling with an unknown distance function. I. , 1962 .

[29]  Robert D. Nowak,et al.  Best-arm identification algorithms for multi-armed bandits in the fixed confidence setting , 2014, 2014 48th Annual Conference on Information Sciences and Systems (CISS).

[30]  Milos Hauskrecht,et al.  Active Perceptual Similarity Modeling with Auxiliary Information , 2015, ArXiv.

[31]  J. Kruskal.  Nonmetric multidimensional scaling: A numerical method , 1964 .

[32]  Shie Mannor,et al.  Action Elimination and Stopping Conditions for the Multi-Armed Bandit and Reinforcement Learning Problems , 2006, J. Mach. Learn. Res..

[33]  Lalit Jain,et al.  NEXT: A System for Real-World Development, Evaluation, and Application of Active Learning , 2015, NIPS.