Hyperbolic Entailment Cones for Learning Hierarchical Embeddings

Learning graph representations via low-dimensional embeddings that preserve relevant network properties is an important class of problems in machine learning. We here present a novel method to embed directed acyclic graphs. Following prior work, we first advocate for using hyperbolic spaces which provably model tree-like structures better than Euclidean geometry. Second, we view hierarchical relations as partial orders defined using a family of nested geodesically convex cones. We prove that these entailment cones admit an optimal shape with a closed form expression both in the Euclidean and hyperbolic spaces. Moreover, they canonically define the embedding learning process. Experiments show significant improvements of our method over strong recent baselines both in terms of representational capacity and generalization.

[1]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[2]  Amin Vahdat,et al.  Hyperbolic Geometry of Complex Networks , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[4]  M. Spivak A comprehensive introduction to differential geometry , 1979 .

[5]  Brian H. Bowditch,et al.  A Course On Geometric Group Theory , 2006 .

[6]  Sanja Fidler,et al.  Skip-Thought Vectors , 2015, NIPS.

[7]  Yuval Shavitt,et al.  On Hyperbolic Embedding of Internet Graph for Distance Estimation and Overlay Construction , 2007 .

[8]  Yuval Shavitt,et al.  Hyperbolic embedding of internet graph for distance estimation and overlay construction , 2008, TNET.

[9]  Christopher De Sa,et al.  Representation Tradeoffs for Hyperbolic Embeddings , 2018, ICML.

[10]  Silvere Bonnabel,et al.  Stochastic Gradient Descent on Riemannian Manifolds , 2011, IEEE Transactions on Automatic Control.

[11]  Amin Vahdat,et al.  Greedy forwarding in scale-free networks embedded in hyperbolic metric spaces , 2009, SIGMETRICS Perform. Evaluation Rev..

[12]  Mark Crovella,et al.  Hyperbolic Embedding and Routing for Dynamic Graphs , 2009, IEEE INFOCOM 2009.

[13]  Phil Blunsom,et al.  Reasoning about Entailment with Neural Attention , 2015, ICLR.

[14]  Douwe Kiela,et al.  Poincaré Embeddings for Learning Hierarchical Representations , 2017, NIPS.

[15]  J. Robbin,et al.  INTRODUCTION TO DIFFERENTIAL GEOMETRY , 2011 .

[16]  Ido Dagan,et al.  Improving Hypernymy Detection with an Integrated Path-based and Distributional Method , 2016, ACL.

[17]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[18]  Ramana Rao,et al.  A focus+context technique based on hyperbolic geometry for visualizing large hierarchies , 1995, CHI '95.

[19]  Tobias Friedrich,et al.  Efficient Embedding of Scale-Free Graphs in the Hyperbolic Plane , 2018, IEEE/ACM Transactions on Networking.

[20]  W. Floyd,et al.  HYPERBOLIC GEOMETRY , 1996 .

[21]  A. O. Houcine On hyperbolic groups , 2006 .

[22]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[23]  Louis J. Billera,et al.  Geometry of the Space of Phylogenetic Trees , 2001, Adv. Appl. Math..

[24]  James W. Anderson,et al.  Hyperbolic geometry , 1999 .

[25]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[26]  Hans-Peter Kriegel,et al.  A Three-Way Model for Collective Learning on Multi-Relational Data , 2011, ICML.

[27]  M. Hamann On the tree-likeness of hyperbolic spaces , 2011, Mathematical Proceedings of the Cambridge Philosophical Society.

[28]  Thomas Hofmann,et al.  Deep Joint Entity Disambiguation with Local Neural Attention , 2017, EMNLP.

[29]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[30]  Sanja Fidler,et al.  Order-Embeddings of Images and Language , 2015, ICLR.

[31]  Peter D. Hoff,et al.  Latent Space Approaches to Social Network Analysis , 2002 .

[32]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[33]  Wanxiang Che,et al.  Learning Semantic Hierarchies via Word Embeddings , 2014, ACL.

[34]  B. Andrews,et al.  The Ricci Flow in Riemannian Geometry , 2011 .

[35]  Rik Sarkar,et al.  Low Distortion Delaunay Embedding of Trees in Hyperbolic Plane , 2011, GD.

[36]  George A. Miller,et al.  Introduction to WordNet: An On-line Lexical Database , 1990 .