Hyperbolic Disk Embeddings for Directed Acyclic Graphs

Obtaining continuous representations of structural data such as directed acyclic graphs (DAGs) has gained attention in machine learning and artificial intelligence. However, embedding complex DAGs in which both ancestors and descendants of nodes are exponentially increasing is difficult. Tackling in this problem, we develop Disk Embeddings, which is a framework for embedding DAGs into quasi-metric spaces. Existing state-of-the-art methods, Order Embeddings and Hyperbolic Entailment Cones, are instances of Disk Embedding in Euclidean space and spheres respectively. Furthermore, we propose a novel method Hyperbolic Disk Embeddings to handle exponential growth of relations. The results of our experiments show that our Disk Embedding models outperform existing methods especially in complex DAGs other than trees.

[1]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[2]  Silvere Bonnabel,et al.  Stochastic Gradient Descent on Riemannian Manifolds , 2011, IEEE Transactions on Automatic Control.

[3]  Jens Blanck,et al.  Domain Representability of Metric Spaces , 1997, Ann. Pure Appl. Log..

[4]  Kevin Chen-Chuan Chang,et al.  A Comprehensive Survey of Graph Embedding: Problems, Techniques, and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[5]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[6]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[7]  D J PRICE,et al.  NETWORKS OF SCIENTIFIC PAPERS. , 1965, Science.

[8]  Thomas Hofmann,et al.  Deep Joint Entity Disambiguation with Local Neural Attention , 2017, EMNLP.

[9]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[10]  Jian Pei,et al.  A Survey on Network Embedding , 2017, IEEE Transactions on Knowledge and Data Engineering.

[11]  Rik Sarkar,et al.  Low Distortion Delaunay Embedding of Trees in Hyperbolic Plane , 2011, GD.

[12]  Bonnie Kirkpatrick.  Haplotypes versus genotypes on pedigrees , 2010, Algorithms for Molecular Biology.

[13]  Sanja Fidler,et al.  Skip-Thought Vectors , 2015, NIPS.

[14]  A. O. Houcine.  On hyperbolic groups , 2006 .

[15]  P. Spirtes,et al.  Causation, prediction, and search , 1993 .

[16]  Douwe Kiela,et al.  Poincaré Embeddings for Learning Hierarchical Representations , 2017, NIPS.

[17]  A. Merchant,et al.  Directed acyclic graphs (DAGs): an aid to assess confounding in dental research. , 2002, Community dentistry and oral epidemiology.

[18]  Reinhold Heckmann,et al.  Approximation of Metric Spaces by Partial Metric Spaces , 1999, Appl. Categorical Struct..

[19]  Jean Goubault-Larrecq.  Metrics, quasi-metrics, hemi-metrics , 2013 .

[20]  Zhendong Mao,et al.  Knowledge Graph Embedding: A Survey of Approaches and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[21]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[22]  Abbas Edalat,et al.  A Computational Model for Metric Spaces , 1998, Theor. Comput. Sci..

[23]  Sanja Fidler,et al.  Order-Embeddings of Images and Language , 2015, ICLR.

[24]  Peter D. Hoff,et al.  Latent Space Approaches to Social Network Analysis , 2002 .

[25]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[26]  Klaus Weihrauch,et al.  Embedding Metric Spaces Into CPO's , 1981, Theor. Comput. Sci..

[27]  Gem Stapleton,et al.  Drawing Euler Diagrams with Circles: The Theory of Piercings , 2011, IEEE Transactions on Visualization and Computer Graphics.

[28]  Andrew McCallum,et al.  Word Representations via Gaussian Embedding , 2014, ICLR.

[29]  Douwe Kiela,et al.  Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry , 2018, ICML.

[30]  Tiansi Dong,et al.  Imposing Category Trees Onto Word-Embeddings Using A Geometric Construction , 2018, ICLR.

[31]  Thomas Hofmann,et al.  Hyperbolic Entailment Cones for Learning Hierarchical Embeddings , 2018, ICML.

[32]  Tim S. Evans,et al.  Embedding graphs in Lorentzian spacetime , 2016, PloS one.

[33]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[34]  Hideki Tsuiki,et al.  Lawson topology of the space of formal balls and the hyperbolic topology , 2008, Theor. Comput. Sci..

[35]  A. Philip Dawid,et al.  Beware of the DAG! , 2008, NIPS Causality: Objectives and Assessment.

[36]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[37]  Hans-Peter Kriegel,et al.  A Three-Way Model for Collective Learning on Multi-Relational Data , 2011, ICML.

[38]  J. Robins.  A graphical approach to the identification and estimation of causal parameters in mortality studies with sustained exposure periods. , 1987, Journal of chronic diseases.