Two-order graph convolutional networks for semi-supervised classification

Currently, deep learning (DL) algorithms have achieved great success in many applications including computer vision and natural language processing. Many different kinds of DL models have been reported, such as DeepWalk, LINE, diffusionconvolutional neural networks, graph convolutional networks (GCN), and so on. The GCN algorithm is a variant of convolutional neural network and achieves significant superiority by using a one-order localised spectral graph filter. However, only a one-order polynomial in the Laplacian of GCN has been approximated and implemented, which ignores undirect neighbour structure information. The lack of rich structure information reduces the performance of the neural networks in the graph structure data. In this study, the authors deduce and simplify the formula of two-order spectral graph convolutions to preserve rich local information. Furthermore, they build a layerwise GCN based on this two-order approximation, i.e. two-order GCN (TGCN) for semi-supervised classification. With the two-order polynomial in the Laplacian, the proposed TGCN model can assimilate abundant localised structure information of graph data and then boosts the classification significantly. To evaluate the proposed solution, extensive experiments are conducted on several popular datasets including the Citeseer, Cora, and PubMed dataset. Experimental results demonstrate that the proposed TGCN outperforms the state-of-art methods.

[1]  Feiping Nie,et al.  Detecting Coherent Groups in Crowd Scenes by Multiview Clustering , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Qi Wang,et al.  Robust Hierarchical Deep Learning for Vehicular Management , 2019, IEEE Transactions on Vehicular Technology.

[3]  Qi Wang,et al.  A Joint Convolutional Neural Networks and Context Transfer for Street Scenes Labeling , 2018, IEEE Transactions on Intelligent Transportation Systems.

[4]  Junyu Gao,et al.  Embedding structured contour and location prior in siamesed fully convolutional networks for road detection , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[5]  Ben Glocker,et al.  Distance Metric Learning Using Graph Convolutional Networks: Application to Functional Brain Networks , 2017, MICCAI.

[6]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[7]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[8]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[9]  Ruslan Salakhutdinov,et al.  Revisiting Semi-Supervised Learning with Graph Embeddings , 2016, ICML.

[10]  Donald F. Towsley,et al.  Diffusion-Convolutional Neural Networks , 2015, NIPS.

[11]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[12]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[13]  Jinjun Chen,et al.  HireSome-II: Towards Privacy-Aware Cross-Cloud Service Composition for Big Data Applications , 2015, IEEE Transactions on Parallel and Distributed Systems.

[14]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[15]  Jinjun Chen,et al.  KASR: A Keyword-Aware Service Recommendation Method on MapReduce for Big Data Applications , 2014, IEEE Transactions on Parallel and Distributed Systems.

[16]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[17]  Cecile Cabanes,et al.  The CORA dataset: validation and diagnostics of in-situ ocean temperature and salinity measurements , 2013 .

[18]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[19]  Joseph F. Murray,et al.  Convolutional Networks Can Learn to Generate Affinity Graphs for Image Segmentation , 2010, Neural Computation.

[20]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[21]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[22]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[23]  Jason Weston,et al.  Deep learning via semi-supervised embedding , 2008, ICML '08.

[24]  Mikhail Belkin,et al.  Towards a theoretical foundation for Laplacian-based manifold methods , 2005, J. Comput. Syst. Sci..

[25]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[26]  Pedro M. Domingos,et al.  Joint Inference in Information Extraction , 2007, AAAI.

[27]  L. Getoor,et al.  D-Dupe: An Interactive Tool for Entity Resolution in Social Networks , 2006, 2006 IEEE Symposium On Visual Analytics Science And Technology.

[28]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[29]  Jeonghee Yi,et al.  Sentiment analysis: capturing favorability using natural language processing , 2003, K-CAP '03.

[30]  Lise Getoor,et al.  Link-Based Classification , 2003, Encyclopedia of Machine Learning and Data Mining.

[31]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[32]  J. van Leeuwen,et al.  Neural Networks: Tricks of the Trade , 2002, Lecture Notes in Computer Science.

[33]  Yrjö Neuvo,et al.  Detail-preserving median based filters in image processing , 1994, Pattern Recognit. Lett..

[34]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[35]  Alexandru T. Balaban,et al.  Applications of graph theory in chemistry , 1985, J. Chem. Inf. Comput. Sci..

[36]  R. Keys.  Cubic convolution interpolation for digital image processing , 1981 .