Representation Tradeoffs for Hyperbolic Embeddings

Hyperbolic embeddings offer excellent quality with few dimensions when embedding hierarchical data structures like synonym or type hierarchies. Given a tree, we give a combinatorial construction that embeds the tree in hyperbolic space with arbitrarily low distortion without using optimization. On WordNet, our combinatorial embedding obtains a mean-average-precision of 0.989 with only two dimensions, while Nickel et al.'s recent construction obtains 0.87 using 200 dimensions. We provide upper and lower bounds that allow us to characterize the precision-dimensionality tradeoff inherent in any hyperbolic embedding. To embed general metric spaces, we propose a hyperbolic generalization of multidimensional scaling (h-MDS). We show how to perform exact recovery of hyperbolic points from distances, provide a perturbation analysis, and give a recovery result that allows us to reduce dimensionality. The h-MDS approach offers consistently low distortion even with few dimensions across several datasets. Finally, we extract lessons from the algorithms and theory above to design a PyTorch-based implementation that can handle incomplete information and is scalable.

[1]  M. DePamphilis,et al.  HUMAN DISEASE , 1957, The Ulster Medical Journal.

[2]  R. Sibson.  Studies in the Robustness of Multidimensional Scaling: Procrustes Statistics , 1978 .

[3]  R. Sibson.  Studies in the Robustness of Multidimensional Scaling: Perturbational Analysis of Classical Scaling , 1979 .

[4]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[5]  W. Fischer,et al.  Sphere Packings, Lattices and Groups , 1990 .

[6]  R. Benedetti,et al.  Lectures on Hyperbolic Geometry , 1992 .

[7]  Ramana Rao,et al.  Laying out and visualizing large trees using a hyperbolic space , 1994, UIST '94.

[8]  Nathan Linial,et al.  The geometry of graphs and some of its algorithmic applications , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[9]  Jon Kleinberg,et al.  Authoritative sources in a hyperlinked environment , 1999, SODA '98.

[10]  A. Barabasi,et al.  Lethality and centrality in protein networks , 2001, Nature.

[11]  P. Thomas Fletcher,et al.  Principal geodesic analysis for the study of nonlinear statistics of shape , 2004, IEEE Transactions on Medical Imaging.

[12]  Jörg A. Walter.  H-MDS: a new approach for interactive visualization with multidimensional scaling in the hyperbolic space , 2004, Inf. Syst..

[13]  Bethany S. Dohleman.  Exploratory social network analysis with Pajek , 2006 .

[14]  A. O. Houcine.  On hyperbolic groups , 2006 .

[15]  Ittai Abraham,et al.  Reconstructing approximate tree metrics , 2007, PODC '07.

[16]  Christos Faloutsos,et al.  Graph evolution: Densification and shrinking diameters , 2006, TKDD.

[17]  A. Barabasi,et al.  The human disease network , 2007, Proceedings of the National Academy of Sciences.

[18]  Evgeniy Gabrilovich,et al.  Computing Semantic Relatedness Using Wikipedia-based Explicit Semantic Analysis , 2007, IJCAI.

[19]  A. Munk,et al.  INTRINSIC SHAPE ANALYSIS: GEODESIC PCA FOR RIEMANNIAN MANIFOLDS MODULO ISOMETRIC LIE GROUP ACTIONS , 2007 .

[20]  Robert D. Kleinberg.  Geographic Routing Using Hyperbolic Space , 2007, IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications.

[21]  David Eppstein,et al.  Succinct Greedy Graph Drawing in the Hyperbolic Plane , 2008, GD.

[22]  Amin Vahdat,et al.  On curvature and temperature of complex networks , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[23]  Mark Crovella,et al.  Hyperbolic Embedding and Routing for Dynamic Graphs , 2009, IEEE INFOCOM 2009.

[24]  Emmanuel J. Candès,et al.  The Power of Convex Relaxation: Near-Optimal Matrix Completion , 2009, IEEE Transactions on Information Theory.

[25]  Amin Vahdat,et al.  Hyperbolic Geometry of Complex Networks , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[26]  Mark Crovella,et al.  Multidimensional Scaling in the Poincaré Disk , 2011, ArXiv.

[27]  Rik Sarkar,et al.  Low Distortion Delaunay Embedding of Trees in Hyperbolic Plane , 2011, GD.

[28]  Wei Chen,et al.  On the Hyperbolicity of Small-World and Treelike Random Graphs , 2012, Internet Math..

[29]  Danqi Chen,et al.  Reasoning With Neural Tensor Networks for Knowledge Base Completion , 2013, NIPS.

[30]  P. Thomas Fletcher,et al.  Probabilistic Principal Geodesic Analysis , 2013, NIPS.

[31]  Edwin R. Hancock,et al.  Spherical and Hyperbolic Embeddings of Data , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Ryan A. Rossi,et al.  The Network Data Repository with Interactive Graph Analytics and Visualization , 2015, AAAI.

[33]  Christopher De Sa,et al.  Global Convergence of Stochastic Gradient Descent for Some Non-convex Matrix Problems , 2014, ICML.

[34]  Feodor F. Dragan,et al.  Metric tree‐like structures in real‐world networks: an empirical study , 2016, Networks.

[35]  Subhash Suri,et al.  Metric Embedding, Hyperbolic Space, and Social Networks , 2014, Comput. Geom..

[36]  Michelle L. Hart,et al.  Preliminary insights from DNA barcoding into the diversity of mosses colonising modern building surfaces , 2016 .

[37]  Douwe Kiela,et al.  Poincaré Embeddings for Learning Hierarchical Representations , 2017, NIPS.

[38]  Marc Peter Deisenroth,et al.  Neural Embeddings of Graphs in Hyperbolic Space , 2017, ArXiv.

[39]  Maureen T. Carroll.  Geometry , 2017, MAlkahtani Mathematics.

[40]  Matthew Jenssen,et al.  On kissing numbers and spherical codes in high dimensions , 2018, Advances in Mathematics.

[41]  X. Pennec.  Barycentric subspace analysis on manifolds , 2016, The Annals of Statistics.

[42]  Thomas Hofmann,et al.  Hyperbolic Entailment Cones for Learning Hierarchical Embeddings , 2018, ICML.

[43]  Siu Cheung Hui,et al.  Hyperbolic Representation Learning for Fast and Efficient Neural Question Answering , 2017, WSDM.