Semi-supervised classification on graphs using explicit diffusion dynamics

Classification tasks based on feature vectors can be significantly improved by including within deep learning a graph that summarises pairwise relationships between the samples. Intuitively, the graph acts as a conduit to channel and bias the inference of class labels. Here, we study classification methods that consider the graph as the originator of an explicit graph diffusion. We show that appending graph diffusion to feature-based learning as an \textit{a posteriori} refinement achieves state-of-the-art classification accuracy. This method, which we call Graph Diffusion Reclassification (GDR), uses overshooting events of a diffusive graph dynamics to reclassify individual nodes. The method uses intrinsic measures of node influence, which are distinct for each node, and allows the evaluation of the relationship and importance of features and graph for classification. We also present diff-GCN, a simple extension of Graph Convolutional Neural Network (GCN) architectures that leverages explicit diffusion dynamics, and allows the natural use of directed graphs. To showcase our methods, we use benchmark datasets of documents with associated citation data.

[1]  Jean-Charles Delvenne,et al.  Stability of graph communities across time scales , 2008, Proceedings of the National Academy of Sciences.

[2]  Mauricio Barahona,et al.  Graph-based data clustering via multiscale community detection , 2020, Appl. Netw. Sci..

[3]  Qiang Ma,et al.  Dual Graph Convolutional Networks for Graph-Based Semi-Supervised Classification , 2018, WWW.

[4]  Max Welling,et al.  Semi-supervised Learning with Deep Generative Models , 2014, NIPS.

[5]  Jean-Charles Delvenne,et al.  Multiscale dynamical embeddings of complex networks , 2018, Physical review. E.

[6]  Sariel Har-Peled,et al.  Random Walks , 2021, Encyclopedia of Social Network Analysis and Mining.

[7]  Mauricio Barahona,et al.  Flow-Based Network Analysis of the Caenorhabditis elegans Connectome , 2015, PLoS Comput. Biol..

[8]  F. Chung Laplacians and the Cheeger Inequality for Directed Graphs , 2005 .

[9]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[10]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[11]  Mason A. Porter,et al.  Random walks and diffusion on networks , 2016, ArXiv.

[12]  Jean-Charles Delvenne,et al.  Random Walks, Markov Processes and the Multiscale Modular Organization of Complex Networks , 2014, IEEE Transactions on Network Science and Engineering.

[13]  Mauricio Barahona,et al.  Role-based similarity in directed networks , 2010, ArXiv.

[14]  François Fouss,et al.  Random-Walk Computation of Similarities between Nodes of a Graph with Application to Collaborative Recommendation , 2007, IEEE Transactions on Knowledge and Data Engineering.

[15]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[16]  Xiaolong Li,et al.  GeniePath: Graph Neural Networks with Adaptive Receptive Paths , 2018, AAAI.

[17]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[18]  Zhengyang Wang,et al.  Large-Scale Learnable Graph Convolutional Networks , 2018, KDD.

[19]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[20]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[21]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[22]  Hao Ma,et al.  GaAN: Gated Attention Networks for Learning on Large and Spatiotemporal Graphs , 2018, UAI.

[23]  Ruslan Salakhutdinov,et al.  Revisiting Semi-Supervised Learning with Graph Embeddings , 2016, ICML.

[24]  Mauricio Barahona,et al.  Geometric Multiscale Community Detection: Markov Stability and Vector Partitioning , 2017, J. Complex Networks.

[25]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[26]  Mauricio Barahona,et al.  Interest communities and flow roles in directed networks: the Twitter network of the UK riots , 2013, Journal of The Royal Society Interface.

[27]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[28]  Mauricio Barahona,et al.  Graph centrality is a question of scale , 2019, ArXiv.

[29]  Mauricio Barahona,et al.  Finding role communities in directed networks using Role-Based Similarity, Markov Stability and the Relaxed Minimum Spanning Tree , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[30]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[31]  Xavier Bresson,et al.  CayleyNets: Graph Convolutional Neural Networks With Complex Rational Spectral Filters , 2017, IEEE Transactions on Signal Processing.

[32]  Alexander Zien,et al.  Semi-Supervised Classification by Low Density Separation , 2005, AISTATS.

[33]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[34]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[35]  Stéphane Lafon,et al.  Diffusion maps , 2006 .

[36]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[37]  Jason Weston,et al.  Deep learning via semi-supervised embedding , 2008, ICML '08.

[38]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[39]  Mauricio Barahona,et al.  Quantifying the Alignment of Graph and Features in Deep Learning , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[40]  Le Song,et al.  Stochastic Training of Graph Convolutional Networks with Variance Reduction , 2017, ICML.