Enhancing Graph Neural Networks via auxiliary training for semi-supervised node classification

Abstract Graph Neural Networks (GNNs) have been successfully applied to graph analysis tasks. As a canonical task in graph analysis, node classification has achieved promising results with GNNs. However, training GNNs remains challenging when training samples are limited. For the node classification task, only the known node labels are used as supervision information to train GNNs, while the implicit information of pairwise relationships between nodes is neglected. The label-to-label relationship of node pairs is explicit in labeled nodes and not utilized. Moreover, it is found that the link relationship in a graph is correlated with the labels between nodes in some graphs, like information network. Based on that, we can extract and establish labels for pairwise nodes. With the supervision of labeled pairwise nodes, it can force the predicted labels to conform to the observed pairwise relationships and provide some useful information to boost performance. In view of this, we utilize pairwise relationships by introducing the auxiliary task, node pair classification or link prediction, and propose a novel training framework for enhancing GNNs, namely EGNN. Via jointly training node classification with the auxiliary task, our framework achieves higher classification accuracy for general GNNs models with only a little extra computational cost. Moreover, the adaptive dynamic weighting strategy is designed to balance the training pace of different tasks automatically. We conduct extensive experiments, and the evaluation results suggest the superiority of our framework.

[1]  Shih-Fu Chang,et al.  Learning with Partially Absorbing Random Walks , 2012, NIPS.

[2]  Wei Gao,et al.  Personalized Microblog Sentiment Classification via Adversarial Cross-lingual Multi-task Learning , 2018, EMNLP.

[3]  Cao Xiao,et al.  FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling , 2018, ICLR.

[4]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[5]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[6]  Xiao-Ming Wu,et al.  Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning , 2018, AAAI.

[7]  Phi Vu Tran,et al.  Multi-Task Graph Autoencoders , 2018, ArXiv.

[8]  Vipin Kumar,et al.  A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs , 1998, SIAM J. Sci. Comput..

[9]  Ben Glocker,et al.  Disease prediction using graph convolutional networks: Application to Autism Spectrum Disorder and Alzheimer's disease , 2018, Medical Image Anal..

[10]  Shuang Yu,et al.  Multi-task Neural Networks with Spatial Activation for Retinal Vessel Segmentation and Artery/Vein Classification , 2019, MICCAI.

[11]  Jie Tang,et al.  ArnetMiner: extraction and mining of academic social networks , 2008, KDD.

[12]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[13]  Eric P. Xing,et al.  AutoLoss: Learning Discrete Schedules for Alternate Optimization , 2018, ICLR 2018.

[14]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[15]  Luís C. Lamb,et al.  Multitask Learning on Graph Neural Networks - Learning Multiple Graph Centrality Measures with a Unified Network , 2018, ICANN.

[16]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[17]  Rich Caruana,et al.  Multitask Learning , 1997, Machine Learning.

[18]  David J. Hand,et al.  Assessing the Performance of Classification Methods , 2012 .

[19]  Le Song,et al.  Stochastic Training of Graph Convolutional Networks with Variance Reduction , 2017, ICML.

[20]  Lutz Prechelt,et al.  Early Stopping - But When? , 2012, Neural Networks: Tricks of the Trade.

[21]  Richard van Noorden Online collaboration: Scientists and the social network. , 2014, Nature.

[22]  Yunming Ye,et al.  Semi-supervised multi-label collective classification ensemble for functional genomics , 2014, BMC Genomics.

[23]  Andrew J. Davison,et al.  End-To-End Multi-Task Learning With Attention , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[25]  A. K. Qin,et al.  A Multi-Task Representation Learning Architecture for Enhanced Graph Classification , 2020, Frontiers in Neuroscience.

[26]  Junzhou Huang,et al.  Adaptive Sampling Towards Fast Graph Representation Learning , 2018, NeurIPS.

[27]  Xing Xie,et al.  Co-Attentive Multi-Task Learning for Explainable Recommendation , 2019, IJCAI.

[28]  Jie Zhou,et al.  Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View , 2020, AAAI.

[29]  Alex Smola,et al.  Deep Graph Library: Towards Efficient and Scalable Deep Learning on Graphs , 2019, ArXiv.

[30]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[31]  Peter A. Flach,et al.  Modifying ROC Curves to Incorporate Predicted Probabilities , 2005 .

[32]  Zhao Chen,et al.  GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks , 2017, ICML.

[33]  Samy Bengio,et al.  Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks , 2019, KDD.

[34]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[35]  Barbara Plank,et al.  When is multitask learning effective? Semantic sequence prediction under varying data conditions , 2016, EACL.

[36]  Tao Mei,et al.  Graph-based semi-supervised learning with multiple labels , 2009, J. Vis. Commun. Image Represent..