Lorentzian Graph Convolutional Networks

Graph convolutional networks (GCNs) have received considerable research attention recently. Most GCNs learn the node representations in Euclidean geometry, but that could have a high distortion in the case of embedding graphs with scale-free or hierarchical structure. Recently, some GCNs are proposed to deal with this problem in non-Euclidean geometry, e.g., hyperbolic geometry. Although hyperbolic GCNs achieve promising performance, existing hyperbolic graph operations actually cannot rigorously follow the hyperbolic geometry, which may limit the ability of hyperbolic geometry and thus hurt the performance of hyperbolic GCNs. In this paper, we propose a novel hyperbolic GCN named Lorentzian graph convolutional network (LGCN), which rigorously guarantees the learned node features follow the hyperbolic geometry. Specifically, we rebuild the graph operations of hyperbolic GCNs with Lorentzian version, e.g., the feature transformation and non-linear activation. Also, an elegant neighborhood aggregation method is designed based on the centroid of Lorentzian distance. Moreover, we prove some proposed graph operations are equivalent in different types of hyperbolic geometry, which fundamentally indicates their correctness. Experiments on six datasets show that LGCN performs better than the state-of-the-art methods. LGCN has lower distortion to learn the representation of tree-likeness graphs compared with existing hyperbolic GCNs. We also find that the performance of some hyperbolic GCNs can be improved by simply replacing the graph operations with those we defined in this paper.

[1]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[2]  Ruslan Salakhutdinov,et al.  Revisiting Semi-Supervised Learning with Graph Embeddings , 2016, ICML.

[3]  Renjie Liao,et al.  Lorentzian Distance Learning for Hyperbolic Representations , 2019, ICML.

[4]  John M. Lee.  Manifolds and Differential Geometry , 2009 .

[5]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[6]  Linmei Hu,et al.  Heterogeneous Graph Attention Networks for Semi-supervised Short Text Classification , 2019, EMNLP.

[7]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[8]  A. O. Houcine.  On hyperbolic groups , 2006 .

[9]  Mark E. J. Newman,et al.  Power-Law Distributions in Empirical Data , 2007, SIAM Rev..

[10]  Jian Tang,et al.  Session-Based Social Recommendation via Dynamic Graph Attention Networks , 2019, WSDM.

[11]  Christopher De Sa,et al.  Differentiating through the Fréchet Mean , 2020, ICML.

[12]  Ryusuke Takahama,et al.  Hyperbolic Disk Embeddings for Directed Acyclic Graphs , 2019, ICML.

[13]  Timothy M. Hospedales,et al.  Multi-relational Poincaré Graph Embeddings , 2019, NeurIPS.

[14]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[15]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[16]  A. Ungar,et al.  Analytic Hyperbolic Geometry: Mathematical Foundations And Applications , 2005 .

[17]  Frederic Sala,et al.  Learning Mixed-Curvature Representations in Product Spaces , 2018, ICLR.

[18]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[19]  Gary Bécigneul,et al.  Poincaré GloVe: Hyperbolic Word Embeddings , 2018, ICLR.

[20]  Daniel R. Figueiredo,et al.  struc2vec: Learning Node Representations from Structural Identity , 2017, KDD.

[21]  Christopher De Sa,et al.  Representation Tradeoffs for Hyperbolic Embeddings , 2018, ICML.

[22]  Razvan Pascanu,et al.  Hyperbolic Attention Networks , 2018, ICLR.

[23]  Douwe Kiela,et al.  Poincaré Embeddings for Learning Hierarchical Representations , 2017, NIPS.

[24]  Thomas Hofmann,et al.  Hyperbolic Neural Networks , 2018, NeurIPS.

[25]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[26]  Stephan Günnemann,et al.  Pitfalls of Graph Neural Network Evaluation , 2018, ArXiv.

[27]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[28]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[29]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[30]  Jure Leskovec,et al.  Hyperbolic Graph Convolutional Neural Networks , 2019, NeurIPS.

[31]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[32]  Matthias Leimeister,et al.  Gradient descent in hyperbolic space , 2018, 1805.08207.

[33]  Yanfang Ye,et al.  Hyperbolic Graph Attention Network , 2019, IEEE Transactions on Big Data.

[34]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[35]  S. Helgason.  Differential Geometry, Lie Groups, and Symmetric Spaces , 1978 .

[36]  Thomas Hofmann,et al.  Hyperbolic Entailment Cones for Learning Hierarchical Embeddings , 2018, ICML.

[37]  Cao Xiao,et al.  FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling , 2018, ICLR.

[38]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[39]  A. Ungar.  Barycentric calculus in euclidean and hyperbolic geometry: a comparative introduction , 2010 .

[40]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[41]  Silvere Bonnabel,et al.  Stochastic Gradient Descent on Riemannian Manifolds , 2011, IEEE Transactions on Automatic Control.

[42]  Douwe Kiela,et al.  Hyperbolic Graph Neural Networks , 2019, NeurIPS.

[43]  M. Fréchet.  Les éléments aléatoires de nature quelconque dans un espace distancié , 1948 .

[44]  Ben Glocker,et al.  Spectral Graph Convolutions for Population-based Disease Prediction , 2017, MICCAI.

[45]  Seokjun Seo,et al.  Hybrid Approach of Relation Network and Localized Graph Convolutional Filtering for Breast Cancer Subtype Classification , 2017, IJCAI.

[46]  F. Scarselli,et al.  A new model for learning in graph domains , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[47]  H. Karcher.  Riemannian Center of Mass and so called karcher mean , 2014, 1407.2087.

[48]  Douwe Kiela,et al.  Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry , 2018, ICML.

[49]  M. Newman,et al.  Hierarchical structure and the prediction of missing links in networks , 2008, Nature.

[50]  Bhaskar DasGupta,et al.  Topological implications of negative curvature for biological and social networks , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[51]  Jiliang Tang,et al.  Streaming Graph Neural Networks , 2018, SIGIR.

[52]  Octavian-Eugen Ganea,et al.  Constant Curvature Graph Convolutional Networks , 2019, ICML.

[53]  Yuan Luo,et al.  Graph Convolutional Networks for Text Classification , 2018, AAAI.

[54]  Amin Vahdat,et al.  Hyperbolic Geometry of Complex Networks , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[55]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.