Low-dimensional embedding using adaptively selected ordinal data

Low-dimensional embedding based on non-metric data (e.g., non-metric multidimensional scaling) is a problem that arises in many applications, especially those involving human subjects. This paper investigates the problem of learning an embedding of n objects into d-dimensional Euclidean space that is consistent with pairwise comparisons of the type “object a is closer to object b than c.” While there are O(n3) such comparisons, experimental studies suggest that relatively few are necessary to uniquely determine the embedding up to the constraints imposed by all possible pairwise comparisons (i.e., the problem is typically over-constrained). This paper is concerned with quantifying the minimum number of pairwise comparisons necessary to uniquely determine an embedding up to all possible comparisons. The comparison constraints stipulate that, with respect to each object, the other objects are ranked relative to their proximity. We prove that at least Q(dn log n) pairwise comparisons are needed to determine the embedding of all n objects. The lower bounds cannot be achieved by using randomly chosen pairwise comparisons. We propose an algorithm that exploits the low-dimensional geometry in order to accurately embed objects based on relatively small number of sequentially selected pairwise comparisons and demonstrate its performance with experiments.

[1]  Joshua B. Tenenbaum,et al.  Sparse multidimensional scaling using land-mark points , 2004 .

[2]  C. Coombs A theory of data. , 1965, Psychology Review.

[3]  R. Buck Partition of Space , 1943 .

[4]  P. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 1999 .

[5]  Bruce A. Schneider,et al.  Spatial and conjoint models based on pairwise comparisons of dissimilarities and combined effects: Complete and incomplete designs , 1991 .

[6]  Michel Wedel,et al.  The effects of alternative methods of collecting similarity data for Multidimensional Scaling , 1995 .

[7]  R. M. Johnson,et al.  Pairwise nonmetric multidimensional scaling , 1973 .

[8]  Brian McFee,et al.  Distance Metric Learning from Pairwise Proximities , 2008 .

[9]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[10]  Adam Tauman Kalai,et al.  Adaptively Learning the Crowd Kernel , 2011, ICML.

[11]  László Györfi,et al.  A Probabilistic Theory of Pattern Recognition , 1996, Stochastic Modelling and Applied Probability.

[12]  Robert D. Nowak,et al.  Active Ranking using Pairwise Comparisons , 2011, NIPS.

[13]  Gordon D. A. Brown,et al.  Absolute identification by relative judgment. , 2005, Psychological review.

[14]  R. Shepard Metric structures in ordinal data , 1966 .

[15]  David J. Kriegman,et al.  Generalized Non-metric Multidimensional Scaling , 2007, AISTATS.

[16]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[17]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.