Learning Ordinal Embedding from Sets

Ordinal embedding is the task of computing a meaningful multidimensional representation of objects, for which only qualitative constraints on their distance functions are known. In particular, we consider comparisons of the form “Which object from the pair (j,k) is more similar to object i?”. In this paper, we generalize this framework to the case where the ordinal constraints are not given at the level of individual points, but at the level of sets, and propose a distributional triplet embedding approach in a scalable learning framework. We show that the query complexity of our approach is on par with the single-item approach. Without having access to features of the items to be embedded, we show the applicability of our model on toy datasets for the task of reconstruction and demonstrate the validity of the obtained embeddings in experiments on synthetic and real-world datasets.

[1]  R. Shepard.  Metric structures in ordinal data , 1966 .

[2]  R. Shepard.  The analysis of proximities: Multidimensional scaling with an unknown distance function. I. , 1962 .

[3]  Ulrike von Luxburg,et al.  Local Ordinal Embedding , 2014, ICML.

[4]  Marco Cuturi,et al.  Generalizing Point Embeddings using the Wasserstein Space of Elliptical Distributions , 2018, NeurIPS.

[5]  Nir Ailon,et al.  Deep Metric Learning Using Triplet Network , 2014, SIMBAD.

[6]  Adam Tauman Kalai,et al.  Adaptively Learning the Crowd Kernel , 2011, ICML.

[7]  David J. Kriegman,et al.  Generalized Non-metric Multidimensional Scaling , 2007, AISTATS.

[8]  Johannes Fürnkranz,et al.  Permutation Learning via Lehmer Codes , 2020, ECAI.

[9]  Hossein Mobahi,et al.  Learning with a Wasserstein Loss , 2015, NIPS.

[10]  Ulrike von Luxburg,et al.  Kernel functions based on triplet comparisons , 2016, NIPS.

[11]  Ryan L. Murphy,et al.  Janossy Pooling: Learning Deep Permutation-Invariant Functions for Variable-Size Inputs , 2018, ICLR.

[12]  Samy Bengio,et al.  Order Matters: Sequence to sequence for sets , 2015, ICLR.

[13]  Robert D. Nowak,et al.  Low-dimensional embedding using adaptively selected ordinal data , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[14]  Yee Whye Teh,et al.  Set Transformer , 2018, ICML.

[15]  Ewan Klein,et al.  Natural Language Processing with Python , 2009 .

[16]  Navdeep Jaitly,et al.  Pointer Networks , 2015, NIPS.

[17]  Scott W. Linderman,et al.  Learning Latent Permutations with Gumbel-Sinkhorn Networks , 2018, ICLR.

[18]  Jonathon S. Hare,et al.  Learning Representations of Sets through Optimized Permutations , 2018, ICLR.

[19]  J. Dittmann.  Explicit formulae for the Bures metric , 1998, quant-ph/9808044.

[20]  Jason Weston,et al.  Memory Networks , 2014, ICLR.

[21]  I. Olkin,et al.  The distance between two random vectors with given dispersion matrices , 1982 .

[22]  Andrew Markham,et al.  Robust Attentional Aggregation of Deep Feature Sets for Multi-view 3D Reconstruction , 2018, International Journal of Computer Vision.

[23]  R. Beran.  Minimum Hellinger distance estimates for parametric models , 1977 .

[24]  Alexander J. Smola,et al.  Deep Sets , 2017, 1703.06114.

[25]  C. Qi.  Deep Learning on Point Sets for 3 D Classification and Segmentation , 2016 .

[26]  Kilian Q. Weinberger,et al.  Stochastic triplet embedding , 2012, 2012 IEEE International Workshop on Machine Learning for Signal Processing.

[27]  Lalit Jain,et al.  Finite Sample Prediction and Recovery Bounds for Ordinal Embedding , 2016, NIPS.

[28]  Fu Jie Huang,et al.  A Tutorial on Energy-Based Learning , 2006 .

[29]  Ulrike von Luxburg,et al.  Large scale representation learning from triplet comparisons , 2019, ArXiv.

[30]  Johannes Fürnkranz,et al.  Elliptical Ordinal Embedding , 2021, DS.

[31]  Antti Ukkonen,et al.  Crowdsourced Correlation Clustering with Relative Distance Comparisons , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[32]  Thorsten Joachims,et al.  Accurately interpreting clickthrough data as implicit feedback , 2005, SIGIR '05.

[33]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[34]  Leonidas J. Guibas,et al.  PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Hannes Heikinheimo,et al.  The Crowd-Median Algorithm , 2013, HCOMP.

[36]  Sanjoy Dasgupta,et al.  More like this: machine learning approaches to music similarity , 2012 .

[37]  Jonathon S. Hare,et al.  FSPool: Learning Set Representations with Featurewise Sort Pooling , 2019, ICLR.

[38]  Michalis Vazirgiannis,et al.  Rep the Set: Neural Networks for Learning Set Representations , 2019, AISTATS.

[39]  Hannes Heikinheimo,et al.  Crowdsourced Nonparametric Density Estimation Using Relative Distances , 2015, HCOMP.

[40]  Jonathon S. Hare,et al.  Deep Set Prediction Networks , 2019, NeurIPS.