Adaptive Diffusions for Scalable Learning Over Graphs

Diffusion-based classifiers such as those relying on the Personalized PageRank and the heat kernel enjoy remarkable classification accuracy at modest computational requirements. Their performance however is affected by the extent to which the chosen diffusion captures a typically unknown label propagation mechanism, which can be specific to the underlying graph, and potentially different for each class. This paper introduces a disciplined, data-efficient approach to learning class-specific diffusion functions adapted to the underlying network topology. The novel learning approach leverages the notion of “landing probabilities” of class-specific random walks, which can be computed efficiently, thereby ensuring scalability to large graphs. This is supported by rigorous analysis of the properties of the model as well as the proposed algorithms. Furthermore, a robust version of the classifier facilitates learning even in noisy environments. Classification tests on real networks demonstrate that adapting the diffusion function to the given graph and observed labels significantly improves the performance over fixed diffusions, reaching—and many times surpassing—the classification accuracy of computationally heavier state-of-the-art competing methods, which rely on node embeddings and deep neural networks.

[1]  Lars Backstrom,et al.  Balanced label propagation for partitioning massive graphs , 2013, WSDM.

[2]  Fan Chung,et al.  The heat kernel as the pagerank of a graph , 2007, Proceedings of the National Academy of Sciences.

[3]  Thorsten Joachims,et al.  Transductive Learning via Spectral Graph Partitioning , 2003, ICML.

[4]  Geert Leus,et al.  Distributed edge-variant graph filters , 2017, 2017 IEEE 7th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[5]  Amir Globerson,et al.  Semi-Supervised Learning with Competitive Infection Models , 2018, AISTATS.

[6]  David F. Gleich,et al.  Heat kernel based community detection , 2014, KDD.

[7]  Shih-Fu Chang,et al.  Learning with Partially Absorbing Random Walks , 2012, NIPS.

[8]  David F. Gleich,et al.  Random Alpha PageRank , 2009, Internet Math..

[9]  Georgios B. Giannakis,et al.  Adaptive Diffusions for Scalable Learning over Graphs (MLG@KDD18) , 2018 .

[10]  Santiago Segarra,et al.  Optimal Graph-Filter Design and Applications to Distributed Linear Network Operators , 2017, IEEE Transactions on Signal Processing.

[11]  Xiaojin Zhu,et al.  Semi-Supervised Learning , 2010, Encyclopedia of Machine Learning.

[12]  Kun He,et al.  Local Spectral Diffusion for Robust Community Detection , 2016 .

[13]  James H. Garrett,et al.  Semi-Supervised Multiresolution Classification Using Adaptive Graph Filtering With Application to Indirect Bridge Structural Health Monitoring , 2014, IEEE Transactions on Signal Processing.

[14]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[15]  Zoubin Ghahramani,et al.  Combining active learning and semi-supervised learning using Gaussian fields and harmonic functions , 2003, ICML 2003.

[16]  Carl D. Meyer,et al.  Deeper Inside PageRank , 2004, Internet Math..

[17]  William W. Cohen,et al.  Semi-Supervised Classification of Network Data Using Very Few Labels , 2010, 2010 International Conference on Advances in Social Networks Analysis and Mining.

[18]  Georgios B. Giannakis,et al.  AdaDIF: Adaptive Diffusions for Efficient Semi-supervised Learning over Graphs , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[19]  Konstantin Avrachenkov,et al.  Generalized Optimization Framework for Graph-based Semi-supervised Learning , 2011, SDM.

[20]  David F. Gleich,et al.  PageRank beyond the Web , 2014, SIAM Rev..

[21]  Jon M. Kleinberg,et al.  Block models and personalized PageRank , 2016, Proceedings of the National Academy of Sciences.

[22]  Alexander Zien,et al.  Label Propagation and Quadratic Criterion , 2006 .

[23]  Ricardo A. Baeza-Yates,et al.  Generic Damping Functions for Propagating Importance in Link-Based Ranking , 2006, Internet Math..

[24]  Paul Lukowicz,et al.  Label Propagation , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[25]  Ruslan Salakhutdinov,et al.  Revisiting Semi-Supervised Learning with Graph Embeddings , 2016, ICML.

[26]  Christopher D. Manning,et al.  Introduction to Information Retrieval , 2010, J. Assoc. Inf. Sci. Technol..

[27]  Edith Cohen,et al.  Bootstrapped Graph Diffusions: Exposing the Power of Nonlinearity , 2018, PERV.

[28]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs , 2012, IEEE Transactions on Signal Processing.

[29]  Wei Liu,et al.  Robust and Scalable Graph-Based Semisupervised Learning , 2012, Proceedings of the IEEE.

[30]  Georgios B. Giannakis,et al.  Random Walks with Restarts for Graph-Based Classification: Teleportation Tuning and Sampling Design , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[31]  John D. Garofalakis,et al.  NCDawareRank: a novel ranking method that exploits the decomposable structure of the web , 2013, WSDM.

[32]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[33]  Kun He,et al.  Detecting Overlapping Communities from Local Spectral Subspaces , 2015, 2015 IEEE International Conference on Data Mining.

[34]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[35]  Alfred O. Hero,et al.  Multidimensional Shrinkage-Thresholding Operator and Group LASSO Penalties , 2011, IEEE Signal Processing Letters.

[36]  John D. Garofalakis,et al.  Random surfing on multipartite graphs , 2016, 2016 IEEE International Conference on Big Data (Big Data).

[37]  David F. Gleich,et al.  AptRank: an adaptive PageRank model for protein function prediction on bi‐relational graphs , 2016, Bioinform..

[38]  Georgios B. Giannakis,et al.  From Sparse Signals to Sparse Residuals for Robust Sensing , 2011, IEEE Transactions on Signal Processing.

[39]  Kathrin Klamroth,et al.  Biconvex sets and optimization with biconvex functions: a survey and extensions , 2007, Math. Methods Oper. Res..

[40]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[41]  Donald F. Towsley,et al.  Diffusion-Convolutional Neural Networks , 2015, NIPS.

[42]  John D. Lafferty,et al.  Diffusion Kernels on Graphs and Other Discrete Input Spaces , 2002, ICML.

[43]  Ling Huang,et al.  Semi-Supervised Learning with Max-Margin Graph Cuts , 2010, AISTATS.

[44]  Mark Herbster,et al.  Combining Graph Laplacians for Semi-Supervised Learning , 2005, NIPS.

[45]  David F. Gleich,et al.  Using Local Spectral Methods to Robustify Graph-Based Learning Algorithms , 2015, KDD.

[46]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[47]  W. Marsden I and J , 2012 .

[48]  Andrea L. Bertozzi,et al.  A Semi-supervised Heat Kernel Pagerank MBO Algorithm for Data Classification , 2018 .

[49]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[50]  Sergey Brin,et al.  Reprint of: The anatomy of a large-scale hypertextual web search engine , 2012, Comput. Networks.

[51]  Koby Crammer,et al.  New Regularized Algorithms for Transductive Learning , 2009, ECML/PKDD.

[52]  V. Climenhaga Markov chains and mixing times , 2013 .

[53]  R. Stephenson A and V , 1962, The British journal of ophthalmology.