Edge Sparsification for Graphs via Meta-Learning

We present a novel edge sparsification approach for semi-supervised learning on undirected and attributed graphs. The main challenge is to retain few edges while minimizing the loss of node classification accuracy. The task can be mathematically formulated as a bi-level optimization problem. We propose to use meta-gradients, which have traditionally been used in meta-learning, to solve the optimization problem, specifically, treating the graph adjacency matrix as hyperparameters to optimize. Experimental results show the effectiveness of the proposed approach. Remarkably, with the resulting sparse and light graph, in many cases the classification accuracy is significantly improved.

[1]  Christos Faloutsos,et al.  Sampling from large graphs , 2006, KDD '06.

[2]  Qing Li,et al.  A Graph Neural Network Framework for Social Recommendations , 2020, IEEE Transactions on Knowledge and Data Engineering.

[3]  Christian Staudt,et al.  NetworKit: A tool suite for large-scale complex network analysis , 2014, Network Science.

[4]  Andrew McCallum,et al.  Automating the Construction of Internet Portals with Machine Learning , 2000, Information Retrieval.

[5]  Xiaowei Xu,et al.  SCAN: a structural clustering algorithm for networks , 2007, KDD '07.

[6]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[7]  Hanghang Tong,et al.  Graph Convolutional Networks: Algorithms, Applications and Open Challenges , 2018, CSoNet.

[8]  Sergey Levine,et al.  Meta-Learning with Implicit Gradients , 2019, NeurIPS.

[9]  Matthew W. Hoffman,et al.  Modular Meta-Learning with Shrinkage , 2020, NeurIPS.

[10]  Stephan Gunnemann,et al.  Adversarial Attacks on Graph Neural Networks via Meta Learning , 2019, ICLR.

[11]  Dorothea Wagner,et al.  Structure-preserving sparsification methods for social networks , 2016, Social Network Analysis and Mining.

[12]  Kangjie Li,et al.  Hierarchical graph attention networks for semi-supervised node classification , 2020, Applied Intelligence.

[13]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[14]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Joan Bruna,et al.  Few-Shot Learning with Graph Neural Networks , 2017, ICLR.

[16]  Stephan Günnemann,et al.  Adversarial Attacks on Graph Neural Networks via Meta Learning , 2019, ICLR.

[17]  Diego Marcheggiani,et al.  Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling , 2017, EMNLP.

[18]  Nikhil Srivastava,et al.  Graph sparsification by effective resistances , 2008, SIAM J. Comput..

[19]  Xiaojin Zhu,et al.  Introduction to Semi-Supervised Learning , 2009, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[20]  Keping Yang,et al.  M2GRL: A Multi-task Multi-view Graph Representation Learning Framework for Web-scale Recommender Systems , 2020, KDD.

[21]  Martial Hebert,et al.  Semi-Supervised Self-Training of Object Detection Models , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[22]  Debmalya Panigrahi,et al.  A general framework for graph sparsification , 2010, STOC '11.

[23]  Mohammad Ali Zare Chahooki,et al.  A Survey on semi-supervised feature selection methods , 2017, Pattern Recognit..

[24]  Srinivasan Parthasarathy,et al.  Local graph sparsification for scalable clustering , 2011, SIGMOD '11.

[25]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[26]  Ulrik Brandes,et al.  Simmelian backbones: Amplifying hidden homophily in Facebook networks , 2013, 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2013).

[27]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[28]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[29]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[30]  Joshua Achiam,et al.  On First-Order Meta-Learning Algorithms , 2018, ArXiv.

[31]  Shang-Hua Teng,et al.  Spectral Sparsification of Graphs , 2008, SIAM J. Comput..

[32]  Yoshua Bengio,et al.  Gradient-Based Optimization of Hyperparameters , 2000, Neural Computation.