Robust and label efficient bi-filtering graph convolutional networks for node classification

Abstract Due to their success at node classification, Graph Convolutional Networks (GCN) have raised a research upsurge of deep learning on graph-structured data. For the semi-supervised classification, graph convolution essentially acts as a low-pass filter on graph spectral domain. According to Graph Signal Processing theory, the low-pass filter in GCN is a finite impulse response (FIR) graph filter. However, compared with FIR graph filters, infinite impulse response (IIR) graph filters exhibit more powerful representation ability and flexibility. Intuitively, it is feasible to replace FIR filter in GCN with IIR graph filter to improve GCN. Therefore, inspired by the direct implementation of IIR graph filters, we propose a Bi-filtering Graph Convolutional Network (BGCN) which can be realized by simply cascading two sub filtering modules. Experimental results demonstrate that BGCN works well in node classification task and achieves comparable performance to GCN and its variants. The improvement of BGCN, however, is at the expense of a time-complexity increase. To simplify the proposed BGCN, we construct a Simple Bi-filtering Graph Convolution framework (SBGC) from the perspective of Graph Signal Processing. Furthermore, for the implementations of BGCN and SBGC, we design a novel low-pass graph filter to capture the low-frequency components that are beneficial to data representation for the task of node classification. Extensive experiments show that SBGC not only outperforms other baseline methods in performance, but also keeps a high level in computational efficiency. Moreover, it is particularly worth noting that both BGCN and SBGC are robust to feature noise and exhibit high label efficiency.

[1]  Zhiyuan Liu,et al.  Graph Neural Networks: A Review of Methods and Applications , 2018, AI Open.

[2]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[3]  Geert Leus,et al.  Autoregressive Moving Average Graph Filtering , 2016, IEEE Transactions on Signal Processing.

[4]  Ah Chung Tsoi,et al.  Computational Capabilities of Graph Neural Networks , 2009, IEEE Transactions on Neural Networks.

[5]  Bernard Ghanem,et al.  Can GCNs Go as Deep as CNNs? , 2019, ArXiv.

[6]  Ljubisa Stankovic,et al.  Introduction to Graph Signal Processing , 2018, Signals and Communication Technology.

[7]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[8]  Razvan Pascanu,et al.  Relational inductive biases, deep learning, and graph networks , 2018, ArXiv.

[9]  Paulo Gonçalves,et al.  Design of graph filters and filterbanks , 2017, ArXiv.

[10]  Yedid Hoshen,et al.  VAIN: Attentional Multi-agent Predictive Modeling , 2017, NIPS.

[11]  Zhuwen Li,et al.  Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search , 2018, NeurIPS.

[12]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[13]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[14]  Ning Feng,et al.  Attention Based Spatial-Temporal Graph Convolutional Networks for Traffic Flow Forecasting , 2019, AAAI.

[15]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[16]  Wenwu Zhu,et al.  Deep Learning on Graphs: A Survey , 2018, IEEE Transactions on Knowledge and Data Engineering.

[17]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs: Frequency Analysis , 2013, IEEE Transactions on Signal Processing.

[18]  Takanori Maehara,et al.  Revisiting Graph Neural Networks: All We Have is Low-Pass Filters , 2019, ArXiv.

[19]  Dahua Lin,et al.  Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition , 2018, AAAI.

[20]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[21]  G. Phillips.  Interpolation and Approximation by Polynomials , 2003 .

[22]  Xiao Wang,et al.  Beyond Low-frequency Information in Graph Convolutional Networks , 2021, AAAI.

[23]  Svetha Venkatesh,et al.  Column Networks for Collective Classification , 2016, AAAI.

[24]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[25]  Martin Simonovsky,et al.  Large-Scale Point Cloud Semantic Segmentation with Superpoint Graphs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Yang Wang,et al.  BiGCN: A Bi-directional Low-Pass Filtering Graph Neural Network , 2021, ArXiv.

[27]  Cao Xiao,et al.  FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling , 2018, ICLR.

[28]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[29]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[30]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[31]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[32]  Qimai Li,et al.  Label Efficient Semi-Supervised Learning via Graph Filtering , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Bo Hu,et al.  Infinite Impulse Response Graph Filters in Wireless Sensor Networks , 2015, IEEE Signal Processing Letters.

[34]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[35]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[36]  Paulo E. Rauber,et al.  Visualizing Time-Dependent Data Using Dynamic t-SNE , 2016, EuroVis.

[37]  Hao Ma,et al.  GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs , 2018, UAI.

[38]  Yuan Luo,et al.  Graph Convolutional Networks for Text Classification , 2018, AAAI.

[39]  Ralph Grishman,et al.  Graph Convolutional Networks With Argument-Aware Pooling for Event Detection , 2018, AAAI.

[40]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[41]  Xavier Bresson,et al.  CayleyNets: Graph Convolutional Neural Networks With Complex Rational Spectral Filters , 2017, IEEE Transactions on Signal Processing.

[42]  David D. Cox,et al.  Hyperopt: A Python Library for Optimizing the Hyperparameters of Machine Learning Algorithms , 2013, SciPy.

[43]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[44]  Pierre Vandergheynst,et al.  Graph Signal Processing: Overview, Challenges, and Applications , 2017, Proceedings of the IEEE.

[45]  Jure Leskovec,et al.  Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation , 2018, NeurIPS.

[46]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.