Subgraph Neural Networks

Deep learning methods for graphs achieve remarkable performance on many node-level and graph-level prediction tasks. However, despite the proliferation of the methods and their success, prevailing Graph Neural Networks (GNNs) neglect subgraphs, rendering subgraph prediction tasks challenging to tackle in many impactful applications. Further, subgraph prediction tasks present several unique challenges, because subgraphs can have non-trivial internal topology, but also carry a notion of position and external connectivity information relative to the underlying graph in which they exist. Here, we introduce SUB-GNN, a subgraph neural network to learn disentangled subgraph representations. In particular, we propose a novel subgraph routing mechanism that propagates neural messages between the subgraph's components and randomly sampled anchor patches from the underlying graph, yielding highly accurate subgraph representations. SUB-GNN specifies three channels, each designed to capture a distinct aspect of subgraph structure, and we provide empirical evidence that the channels encode their intended properties. We design a series of new synthetic and real-world subgraph datasets. Empirical results for subgraph classification on eight datasets show that SUB-GNN achieves considerable performance gains, outperforming strong baseline methods, including node-level and graph-level GNNs, by 12.4% over the strongest baseline. SUB-GNN performs exceptionally well on challenging biomedical datasets when subgraphs have complex topology and even comprise multiple disconnected components.

[1]  Stephen B. Seidman,et al.  Network structure and minimum degree , 1983 .

[2]  J. N. Boyd,et al.  Discrete Dirichlet Problems, Convex Coordinates, and a Random Walk on a Triangle , 1989 .

[3]  Matthew N. O. Sadiku,et al.  A triangular mesh random walk for Dirichlet problems , 1995 .

[4]  M. Ashburner,et al.  Gene Ontology: tool for the unification of biology , 2000, Nature Genetics.

[5]  Mark Newman,et al.  Detecting community structure in networks , 2004 .

[6]  Pablo Tamayo,et al.  Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[7]  Bernhard Schölkopf,et al.  Learning with Hypergraphs: Clustering, Classification, and Embedding , 2006, NIPS.

[8]  Tatiana A. Tatusova,et al.  Entrez Gene: gene-centered information at NCBI , 2004, Nucleic Acids Res..

[9]  P. Robinson,et al.  The Human Phenotype Ontology: a tool for annotating and analyzing human hereditary disease. , 2008, American journal of human genetics.

[10]  Fabrizio Costa,et al.  Fast Neighborhood Subgraph Pairwise Distance Kernel , 2010, ICML.

[11]  Pan-Jun Kim,et al.  Genetic Co-Occurrence Network across Sequenced Microbes , 2011, PLoS Comput. Biol..

[12]  Marco Rosa,et al.  Arc-Community Detection via Triangular Random Walks , 2012, 2012 Eighth Latin American Web Congress.

[13]  Jure Leskovec,et al.  Defining and evaluating network communities based on ground-truth , 2012, Knowledge and Information Systems.

[14]  Boleslaw K. Szymanski,et al.  Overlapping community detection in networks: The state-of-the-art and comparative study , 2011, CSUR.

[15]  S. Maiella,et al.  Orphanet et son réseau : où trouver une information validée sur les maladies rares , 2013 .

[16]  S. Maiella,et al.  [Orphanet and its consortium: where to find expert-validated information on rare diseases]. , 2013, Revue neurologique.

[17]  Jason Weston,et al.  Question Answering with Subgraph Embeddings , 2014, EMNLP.

[18]  Ümit V. Çatalyürek,et al.  Finding the Hierarchy of Dense Subgraphs using Nucleus Decompositions , 2014, WWW.

[19]  Balaraman Ravindran,et al.  Extended Discriminative Random Walk: A Hypergraph Approach to Multi-View Multi-Relational Transductive Learning , 2015, IJCAI.

[20]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[21]  Eamonn J. Keogh,et al.  Extracting Optimal Performance from Dynamic Time Warping , 2016, KDD.

[22]  Mark E. J. Newman,et al.  Structure and inference in annotated networks , 2015, Nature Communications.

[23]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[24]  Jure Leskovec,et al.  Higher-order organization of complex networks , 2016, Science.

[25]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[26]  Hugh Dawkins,et al.  Medical research: Next decade's goals for rare diseases , 2017, Nature.

[27]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[28]  Núria Queralt-Rosinach,et al.  DisGeNET: a comprehensive platform integrating information on human disease-associated genes and variants , 2016, Nucleic Acids Res..

[29]  Alexander J. Smola,et al.  Deep Sets , 2017, 1703.06114.

[30]  Deng Cai,et al.  Learning Graph-Level Representation for Drug Discovery , 2017, ArXiv.

[31]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[32]  Jure Leskovec,et al.  Modeling polypharmacy side effects with graph convolutional networks , 2018, bioRxiv.

[33]  Razvan Pascanu,et al.  Relational inductive biases, deep learning, and graph networks , 2018, ArXiv.

[34]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[35]  Palash Goyal,et al.  Graph Embedding Techniques, Applications, and Performance: A Survey , 2017, Knowl. Based Syst..

[36]  Yiqun Liu,et al.  Learning on Partial-Order Hypergraphs , 2018, WWW.

[37]  Heike Leitte,et al.  Clique Community Persistence: A Topological Visual Analysis Approach for Complex Networks , 2018, IEEE Transactions on Visualization and Computer Graphics.

[38]  Jure Leskovec,et al.  Learning Structural Node Embeddings via Diffusion Wavelets , 2017, KDD.

[39]  Ken-ichi Kawarabayashi,et al.  Representation Learning on Graphs with Jumping Knowledge Networks , 2018, ICML.

[40]  Yao Zhang,et al.  Sub2Vec: Feature Learning for Subgraphs , 2018, PAKDD.

[41]  Yixin Chen,et al.  Beyond Link Prediction: Predicting Hyperlinks in Adjacency Space , 2018, AAAI.

[42]  Euan A Ashley,et al.  Effect of Genetic Diagnosis on Patients with Previously Undiagnosed Disease , 2018, The New England journal of medicine.

[43]  Yixin Chen,et al.  Link Prediction Based on Graph Neural Networks , 2018, NeurIPS.

[44]  Bruno Ribeiro,et al.  Subgraph Pattern Neural Networks for High-Order Graph Evolution Prediction , 2018, AAAI.

[45]  J. Leskovec,et al.  GNNExplainer: Generating Explanations for Graph Neural Networks , 2019, NeurIPS.

[46]  Samy Bengio,et al.  Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks , 2019, KDD.

[47]  Conor A. Bradley.  A statistical framework for rare disease diagnosis , 2019, Nature Reviews Genetics.

[48]  Hui Xiong,et al.  Adversarial Substructured Representation Learning for Mobile User Profiling , 2019, KDD.

[49]  Julian J. McAuley,et al.  Modeling Heart Rate and Activity Data for Personalized Fitness Recommendation , 2019, WWW.

[50]  Dan Alistarh,et al.  Powerset Convolutional Neural Networks , 2019, NeurIPS.

[51]  Yizhou Sun,et al.  Unsupervised Inductive Graph-Level Representation Learning via Graph-Graph Proximity , 2019, IJCAI.

[52]  Bruno Ribeiro,et al.  HATS: A Hierarchical Sequence-Attention Framework for Inductive Set-of-Sets Embeddings , 2019, KDD.

[53]  Katsuhiko Ishiguro,et al.  Graph Warp Module: an Auxiliary Module for Boosting the Power of Graph Neural Networks , 2019, ArXiv.

[54]  Qing Li,et al.  Graph representation learning with encoding edges , 2019, Neurocomputing.

[55]  Danai Koutra,et al.  Latent Network Summarization: Bridging Network Embedding and Summarization , 2018, KDD.

[56]  The Gene Ontology Consortium,et al.  The Gene Ontology Resource: 20 years and still GOing strong , 2018, Nucleic Acids Res..

[57]  Jure Leskovec,et al.  Pre-training Graph Neural Networks , 2019, ArXiv.

[58]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[59]  Jan Eric Lenssen,et al.  Fast Graph Representation Learning with PyTorch Geometric , 2019, ArXiv.

[60]  Jian Tang,et al.  vGraph: A Generative Model for Joint Community Detection and Node Representation Learning , 2019, NeurIPS.

[61]  Jure Leskovec,et al.  Position-aware Graph Neural Networks , 2019, ICML.

[62]  Vinayak A. Rao,et al.  Relational Pooling for Graph Representations , 2019, ICML.

[63]  Kristina Lerman,et al.  MixHop: Higher-Order Graph Convolutional Architectures via Sparsified Neighborhood Mixing , 2019, ICML.

[64]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[65]  Partha Pratim Talukdar,et al.  HyperGCN: A New Method of Training Graph Convolutional Networks on Hypergraphs , 2018 .

[66]  Ryan L. Murphy,et al.  Janossy Pooling: Learning Deep Permutation-Invariant Functions for Variable-Size Inputs , 2018, ICLR.

[67]  Pietro Liò,et al.  Deep Graph Infomax , 2018, ICLR.

[68]  Liujuan Cao,et al.  Hypergraph Induced Convolutional Manifold Networks , 2019, IJCAI.

[69]  Shan-Hung Wu,et al.  Distributed, Egocentric Representations of Graphs for Detecting Critical Structures , 2019, ICML.

[70]  Yue Gao,et al.  Dynamic Hypergraph Neural Networks , 2019, IJCAI.

[71]  Tudor Groza,et al.  Expansion of the Human Phenotype Ontology (HPO) knowledge base and resources , 2018, Nucleic Acids Res..

[72]  Gang Fu,et al.  edge2vec: Representation learning using edge semantics for biomedical knowledge discovery , 2018, BMC Bioinformatics.

[73]  M. Bronstein,et al.  Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning , 2019, Nature Methods.

[74]  Ichiro Takeuchi,et al.  Learning Interpretable Metric between Graphs: Convex Formulation and Computation with Graph Mining , 2019, KDD.

[75]  Ruochi Zhang,et al.  Hyper-SAGNN: a self-attention based graph neural network for hypergraphs , 2019, ICLR.

[76]  Masashi Sugiyama,et al.  Online Dense Subgraph Discovery via Blurred-Graph Feedback , 2020, ICML.

[77]  Kym M Boycott,et al.  New Diagnostic Approaches for Undiagnosed Rare Genetic Diseases. , 2020, Annual review of genomics and human genetics.

[78]  J. Leskovec,et al.  Strategies for Pre-training Graph Neural Networks , 2019, ICLR.

[79]  William L. Hamilton,et al.  Inductive Relation Prediction by Subgraph Reasoning , 2019, ICML.

[80]  Dylan Mordaunt,et al.  Metabolomics to Improve the Diagnostic Efficiency of Inborn Errors of Metabolism , 2020, International journal of molecular sciences.

[81]  Rajgopal Kannan,et al.  GraphSAINT: Graph Sampling Based Inductive Learning Method , 2019, ICLR.

[82]  Zhiyuan Liu,et al.  Graph Neural Networks: A Review of Methods and Applications , 2018, AI Open.

[83]  Jiawei Zhang,et al.  Ripple Walk Training: A Subgraph-based Training Framework for Large and Deep Graph Neural Network , 2020, 2021 International Joint Conference on Neural Networks (IJCNN).

[84]  Wenwu Zhu,et al.  Deep Learning on Graphs: A Survey , 2018, IEEE Transactions on Knowledge and Data Engineering.