Partition-Based Active Learning for Graph Neural Networks

We study the problem of semi-supervised learning with Graph Neural Networks (GNNs) in an active learning setup. We propose GraphPart, a novel partition-based active learning approach for GNNs. GraphPart first splits the graph into disjoint partitions and then selects representative nodes within each partition to query. The proposed method is motivated by a novel analysis of the classification error under realistic smoothness assumptions over the graph and the node features. Extensive experiments on multiple benchmark datasets demonstrate that the proposed method outperforms existing active learning methods for GNNs under a wide range of annotation budget constraints. In addition, the proposed method does not introduce additional hyperparameters, which is crucial for model training, especially in the active learning setting where a labeled validation set may not be available.

[1]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[2]  David A. McAllester,et al.  A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks , 2017, ICLR.

[3]  Yuanzhi Li,et al.  Near-optimal discrete optimization for experimental design: a regret minimization approach , 2017, Mathematical Programming.

[4]  Andrea Montanari,et al.  Contextual Stochastic Block Models , 2018, NeurIPS.

[5]  Kevin Chen-Chuan Chang,et al.  Active Learning for Graph Embedding , 2017, ArXiv.

[6]  Xiaokui Xiao,et al.  Active Learning for Node Classification: The Additional Learning Ability from Unlabelled Nodes , 2020, ArXiv.

[7]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[8]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[9]  Andrew Zisserman,et al.  Advances in Neural Information Processing Systems (NIPS) , 2007 .

[10]  Jiawei Han,et al.  A Variance Minimization Criterion to Active Learning on Graphs , 2012, AISTATS.

[11]  J. Leskovec,et al.  Open Graph Benchmark: Datasets for Machine Learning on Graphs , 2020, NeurIPS.

[12]  Peter J. Rousseeuw,et al.  Finding Groups in Data: An Introduction to Cluster Analysis , 1990 .

[13]  Jie Yin,et al.  SEAL: Semisupervised Adversarial Active Learning on Attributed Graphs , 2021, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Stephan Günnemann,et al.  Predict then Propagate: Graph Neural Networks meet Personalized PageRank , 2018, ICLR.

[15]  Jiawei Han,et al.  Towards Active Learning on Graphs: An Error Bound Minimization Approach , 2012, 2012 IEEE 12th International Conference on Data Mining.

[16]  Aarti Singh,et al.  Active Learning for Graph Neural Networks via Node Feature Propagation , 2019, ArXiv.

[17]  O. William Journal Of The American Statistical Association V-28 , 1932 .

[18]  Marc-Alexandre Côté,et al.  Graph Policy Network for Transferable Active Learning on Graphs , 2020, NeurIPS.

[19]  Jan Eric Lenssen,et al.  Fast Graph Representation Learning with PyTorch Geometric , 2019, ArXiv.

[20]  Silvio Savarese,et al.  Active Learning for Convolutional Neural Networks: A Core-Set Approach , 2017, ICLR.

[21]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[22]  Ludovic Denoyer,et al.  A Meta-Learning Approach to One-Step Active-Learning , 2017, AutoML@PKDD/ECML.

[23]  Xiangliang Zhang,et al.  ActiveHNE: Active Heterogeneous Network Embedding , 2019, IJCAI.

[24]  Stephan Günnemann,et al.  Deep Gaussian Embedding of Graphs: Unsupervised Inductive Learning via Ranking , 2017, ICLR.

[25]  Michael Jackson,et al.  Optimal Design of Experiments , 1994 .

[26]  Robert D. Nowak,et al.  S2: An Efficient Graph Based Active Learning Algorithm with Application to Nonparametric Classification , 2015, COLT.

[27]  Mark Coates,et al.  Active Learning on Attributed Graphs via Graph Cognizant Logistic Regression and Preemptive Query Generation , 2020, ICML.

[28]  Hong Yang,et al.  Active Discriminative Network Representation Learning , 2018, IJCAI.

[29]  T. Yalcinoz,et al.  Implementing soft computing techniques to solve economic dispatch problem in power systems , 2008, Expert Syst. Appl..

[30]  Zhihui Li,et al.  A Survey of Deep Active Learning , 2020, ACM Comput. Surv..