Over-Smoothing Algorithm and Its Application to GCN Semi-supervised Classification

The feature information of the local graph structure and the nodes may be over-smoothing due to the large number of encodings, which causes the node characterization to converge to one or several values. In other words, nodes from different clusters become difficult to distinguish, as two different classes of nodes with closer topological distance are more likely to belong to the same class and vice versa. To alleviate this problem, an over-smoothing algorithm is proposed, and a method of reweighted mechanism is applied to make the trade-off of the information representation of nodes and neighborhoods more reasonable. By improving several propagation models, including Chebyshev polynomial kernel model and Laplace linear 1st Chebyshev kernel model, a new model named RWGCN based on different propagation kernels was proposed logically. The experiments show that satisfactory results are achieved on the semi-supervised classification task of graph type data.

[1]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[2]  Ruslan Salakhutdinov,et al.  Revisiting Semi-Supervised Learning with Graph Embeddings , 2016, ICML.

[3]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[4]  Estevam R. Hruschka,et al.  Toward an Architecture for Never-Ending Language Learning , 2010, AAAI.

[5]  Levent Sendur,et al.  Bivariate shrinkage functions for wavelet-based denoising exploiting interscale dependency , 2002, IEEE Trans. Signal Process..

[6]  Hossein Mobahi,et al.  Deep Learning via Semi-supervised Embedding , 2012, Neural Networks: Tricks of the Trade.

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Yizhou Yu,et al.  Multi-evidence Filtering and Fusion for Multi-label Classification, Object Detection and Semantic Segmentation Based on Weakly Supervised Learning , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[9]  M.L. Hilton,et al.  Wavelet and wavelet packet compression of electrocardiograms , 1997, IEEE Transactions on Biomedical Engineering.

[10]  Paul S. Bonsma,et al.  Tight Lower and Upper Bounds for the Complexity of Canonical Colour Refinement , 2013, Theory of Computing Systems.

[11]  Arthur D. Szlam,et al.  Diffusion wavelet packets , 2006 .

[12]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[13]  Xiu-Shen Wei,et al.  Multi-Label Image Recognition With Graph Convolutional Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[15]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[16]  Lise Getoor,et al.  Link-Based Classification , 2003, Encyclopedia of Machine Learning and Data Mining.

[17]  Xin Li,et al.  Multi-label Image Classification with A Probabilistic Label Enhancement Model , 2014, UAI.

[18]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[19]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[20]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[21]  Yu-Chiang Frank Wang,et al.  Multi-label Zero-Shot Learning with Structured Knowledge Graphs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[22]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[23]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[24]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..