Graph Meta Learning via Local Subgraphs

Prevailing methods for graphs require abundant label and edge information for learning. When data for a new task are scarce, meta-learning can learn from prior experiences and form much-needed inductive biases for fast adaption to new tasks. Here, we introduce G-Meta, a novel meta-learning algorithm for graphs. G-Meta uses local subgraphs to transfer subgraph-specific information and learn transferable knowledge faster via meta gradients. G-Meta learns how to quickly adapt to a new task using only a handful of nodes or edges in the new task and does so by learning from data points in other graphs or related, albeit disjoint label sets. G-Meta is theoretically justified as we show that the evidence for a prediction can be found in the local subgraph surrounding the target node or edge. Experiments on seven datasets and nine baseline methods show that G-Meta outperforms existing methods by up to 16.3%. Unlike previous methods, G-Meta successfully learns in challenging, few-shot learning settings that require generalization to completely new graphs and never-before-seen labels. Finally, G-Meta scales to large graphs, which we demonstrate on a new Tree-of-Life dataset comprising of 1,840 graphs, a two-orders of magnitude increase in the number of graphs used in prior work.

[1]  Xiao-Ming Wu,et al.  Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning , 2018, AAAI.

[2]  Natasa Przulj,et al.  Biological network comparison using graphlet degree distribution , 2007, Bioinform..

[3]  Ji Geng,et al.  Fast Network Alignment via Graph Meta-Learning , 2020, IEEE INFOCOM 2020 - IEEE Conference on Computer Communications.

[4]  Juwen Shen,et al.  Predicting protein–protein interactions based only on sequences information , 2007, Proceedings of the National Academy of Sciences.

[5]  Yixin Chen,et al.  Link Prediction Based on Graph Neural Networks , 2018, NeurIPS.

[6]  Tao Xiang,et al.  Learning to Compare: Relation Network for Few-Shot Learning , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  Mo Yu,et al.  One-Shot Relational Learning for Knowledge Graphs , 2018, EMNLP.

[8]  Richard Socher,et al.  The Natural Language Decathlon: Multitask Learning as Question Answering , 2018, ArXiv.

[9]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[10]  Qiang Chen,et al.  Meta Relational Learning for Few-Shot Link Prediction in Knowledge Graphs , 2019, EMNLP-IJCNLP 2019.

[11]  Jure Leskovec,et al.  Strategies for Pre-training Graph Neural Networks , 2020, ICLR.

[12]  C A Nelson,et al.  Learning to Learn , 2017, Encyclopedia of Machine Learning and Data Mining.

[13]  Danai Koutra,et al.  RolX: structural role extraction & mining in large graphs , 2012, KDD.

[14]  Jing Jiang,et al.  Learning to Propagate for Graph Meta-Learning , 2019, NeurIPS.

[15]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[16]  J. Gough,et al.  The SCOP database in 2020: expanded classification of representative family and superfamily domains of known protein structures , 2019, Nucleic Acids Res..

[17]  Rajgopal Kannan,et al.  GraphSAINT: Graph Sampling Based Inductive Learning Method , 2019, ICLR.

[18]  Pietro Liò,et al.  Deep Graph Infomax , 2018, ICLR.

[19]  Rok Sosic,et al.  Evolution of resilience in protein interactomes across the tree of life , 2018, Proceedings of the National Academy of Sciences.

[20]  Jason Weston,et al.  Memory Networks , 2014, ICLR.

[21]  Yanfang Ye,et al.  Heterogeneous Graph Attention Network , 2019, WWW.

[22]  Jure Leskovec,et al.  Unifying Graph Convolutional Neural Networks and Label Propagation , 2020, ArXiv.

[23]  Samy Bengio,et al.  Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks , 2019, KDD.

[24]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[25]  Yousef Saad,et al.  Dense Subgraph Extraction with Application to Community Detection , 2012, IEEE Transactions on Knowledge and Data Engineering.

[26]  Sahibsingh A. Dudani The Distance-Weighted k-Nearest-Neighbor Rule , 1976, IEEE Transactions on Systems, Man, and Cybernetics.

[27]  Pierre Dupont,et al.  Systems biology Advance Access publication March 12, 2010 , 2009 .

[28]  Jure Leskovec,et al.  Evolution of resilience in protein interactomes across the tree of life , 2018, Proceedings of the National Academy of Sciences.

[29]  Yixin Chen,et al.  Weisfeiler-Lehman Neural Machine for Link Prediction , 2017, KDD.

[30]  Shan-Hung Wu,et al.  Distributed, Egocentric Representations of Graphs for Detecting Critical Structures , 2019, ICML.

[31]  Patrick Aloy,et al.  A reference map of the human binary protein interactome , 2020, Nature.

[32]  Ankit Jain,et al.  Meta-Graph: Few shot Link Prediction via Meta Learning , 2019, ArXiv.

[33]  Jure Leskovec,et al.  Modeling polypharmacy side effects with graph convolutional networks , 2018, bioRxiv.

[34]  Alex Graves,et al.  Neural Turing Machines , 2014, ArXiv.

[35]  Oriol Vinyals,et al.  Matching Networks for One Shot Learning , 2016, NIPS.

[36]  Max Welling,et al.  Variational Graph Auto-Encoders , 2016, ArXiv.

[37]  Hugo Larochelle,et al.  Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples , 2019, ICLR.

[38]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[39]  Jure Leskovec,et al.  Predicting multicellular function through multi-layer tissue networks , 2017, Bioinform..

[40]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[41]  Jure Leskovec,et al.  GNNExplainer: Generating Explanations for Graph Neural Networks , 2019, NeurIPS.

[42]  Roman Garnett,et al.  Graph Kernels for Object Category Prediction in Task-Dependent Robot Grasping , 2013, MLG 2013.

[43]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[44]  Jürgen Schmidhuber,et al.  Shifting Inductive Bias with Success-Story Algorithm, Adaptive Levin Search, and Incremental Self-Improvement , 1997, Machine Learning.

[45]  Ali Farhadi,et al.  Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learning , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  William L. Hamilton,et al.  Inductive Relation Prediction by Subgraph Reasoning , 2020, ICML.

[47]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[48]  Sang Michael Xie,et al.  Combining satellite imagery and machine learning to predict poverty , 2016, Science.

[49]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[50]  Marinka Zitnik,et al.  Subgraph Neural Networks , 2020, NeurIPS.

[51]  Bin Li,et al.  Applications of machine learning in drug discovery and development , 2019, Nature Reviews Drug Discovery.

[52]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[53]  Ricardo Vilalta,et al.  A Perspective View and Survey of Meta-Learning , 2002, Artificial Intelligence Review.

[54]  Jure Leskovec,et al.  Open Graph Benchmark: Datasets for Machine Learning on Graphs , 2020, NeurIPS.

[55]  Hugo Larochelle,et al.  Optimization as a Model for Few-Shot Learning , 2016, ICLR.

[56]  J. Schulman,et al.  Reptile: a Scalable Metalearning Algorithm , 2018 .

[57]  Nitesh V. Chawla,et al.  Graph Few-shot Learning via Knowledge Transfer , 2020, AAAI.

[58]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[59]  Jure Leskovec,et al.  Neural Subgraph Matching , 2020, ArXiv.

[60]  Daan Wierstra,et al.  Meta-Learning with Memory-Augmented Neural Networks , 2016, ICML.

[61]  Manohar Kaul,et al.  Few-Shot Learning on Graphs via Super-Classes based on Graph Spectral Measures , 2020, ICLR.

[62]  Ji Geng,et al.  Meta-GNN: On Few-shot Node Classification in Graph Meta-learning , 2019, CIKM.

[63]  Bridget E. Begg,et al.  A Proteome-Scale Map of the Human Interactome Network , 2014, Cell.

[64]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[65]  Jure Leskovec,et al.  Learning Structural Node Embeddings via Diffusion Wavelets , 2017, KDD.

[66]  Richard S. Zemel,et al.  Prototypical Networks for Few-shot Learning , 2017, NIPS.