Graph Coarsening with Neural Networks

As large-scale graphs become increasingly more prevalent, it poses significant computational challenges to process, extract and analyze large graph data. Graph coarsening is one popular technique to reduce the size of a graph while maintaining essential properties. Despite rich graph coarsening literature, there is only limited exploration of data-driven methods in the field. In this work, we leverage the recent progress of deep learning on graphs for graph coarsening. We first propose a framework for measuring the quality of coarsening algorithm and show that depending on the goal, we need to carefully choose the Laplace operator on the coarse graph and associated projection/lift operators. Motivated by the observation that the current choice of edge weight for the coarse graph may be suboptimal, we parametrize the weight assignment map with graph neural networks and train it to improve the coarsening quality in an unsupervised way. Through extensive experiments on both synthetic and real networks, we demonstrate that our method significantly improves common graph coarsening methods under various metrics, reduction ratios, graph sizes, and graph types. It generalizes to graphs of larger size (25× of training graphs), is adaptive to different losses (differentiable and non-differentiable), and scales to much larger graphs than previous work.

[1]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[3]  Chris Walshaw,et al.  Journal of Graph Algorithms and Applications a Multilevel Algorithm for Force-directed Graph-drawing , 2022 .

[4]  Ann B. Lee,et al.  Diffusion maps and coarse-graining: a unified framework for dimensionality reduction, graph partitioning, and data set parameterization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Jure Leskovec,et al.  Learning to Simulate Complex Physics with Graph Networks , 2020, ICML.

[6]  Jure Leskovec,et al.  GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models , 2018, ICML.

[7]  Fan Chung Graham,et al.  A Combinatorial Laplacian with Vertex Weights , 1996, J. Comb. Theory, Ser. A.

[8]  Tamal K. Dey,et al.  Graph induced complex on point data , 2013, SoCG '13.

[9]  Nikos Komodakis,et al.  GraphVAE: Towards Generation of Small Graphs Using Variational Autoencoders , 2018, ICANN.

[10]  Shang-Hua Teng,et al.  Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems , 2003, STOC '04.

[11]  David Harel,et al.  A fast multi-scale method for drawing large graphs , 2000, AVI '00.

[12]  Razvan Pascanu,et al.  Learning Deep Generative Models of Graphs , 2018, ICLR 2018.

[13]  Yihe Dong,et al.  COPT: Coordinated Optimal Transport on Graphs , 2020, NeurIPS.

[14]  Achi Brandt,et al.  Lean Algebraic Multigrid (LAMG): Fast Graph Laplacian Linear Solver , 2011, SIAM J. Sci. Comput..

[15]  Le Song,et al.  2 Common Formulation for Greedy Algorithms on Graphs , 2018 .

[16]  Ilya Safro,et al.  Algebraic Distance on Graphs , 2011, SIAM J. Sci. Comput..

[17]  Chen Cai,et al.  A simple yet effective baseline for non-attribute graph classification , 2018, ArXiv.

[18]  Daniel Pérez Palomar,et al.  Structured Graph Learning Via Laplacian Spectral Constraints , 2019, NeurIPS.

[19]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[20]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[21]  D. Bartuschat Algebraic Multigrid , 2007 .

[22]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[23]  Zeyuan Allen Zhu,et al.  Spectral Sparsification and Regret Minimization Beyond Matrix Multiplicative Updates , 2015, STOC.

[24]  Nils M. Kriege,et al.  A survey on graph kernels , 2019, Applied Network Science.

[25]  Jeffrey C Grossman,et al.  Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties. , 2017, Physical review letters.

[26]  Inderjit S. Dhillon,et al.  Weighted Graph Cuts without Eigenvectors A Multilevel Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Marc Levoy,et al.  Zippered polygon meshes from range images , 1994, SIGGRAPH.

[28]  Rajgopal Kannan,et al.  GraphSAINT: Graph Sampling Based Inductive Learning Method , 2019, ICLR.

[29]  Achi Brandt,et al.  Fast multiscale clustering and manifold identification , 2006, Pattern Recognit..

[30]  Leslie Hogben,et al.  SPECTRAL GRAPH THEORY AND THE INVERSE EIGENVALUE PROBLEM OF A GRAPH , 2005 .

[31]  Yifan Hu,et al.  Efficient, High-Quality Force-Directed Graph Drawing , 2006 .

[32]  Jaewoo Kang,et al.  Self-Attention Graph Pooling , 2019, ICML.

[33]  Jure Leskovec,et al.  Graph Convolutional Policy Network for Goal-Directed Molecular Graph Generation , 2018, NeurIPS.

[34]  Pietro Liò,et al.  Towards Sparse Hierarchical Graph Classifiers , 2018, ArXiv.

[35]  Ilya Safro,et al.  Relaxation-based coarsening and multiscale graph organization , 2010, Multiscale Model. Simul..

[36]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[37]  Jure Leskovec,et al.  Hierarchical Graph Representation Learning with Differentiable Pooling , 2018, NeurIPS.

[38]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[39]  Shang-Hua Teng,et al.  Spectral Sparsification of Graphs , 2008, SIAM J. Comput..

[40]  Andreas Loukas,et al.  Graph Reduction with Spectral and Cut Guarantees , 2018, J. Mach. Learn. Res..

[41]  Zhuwen Li,et al.  Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search , 2018, NeurIPS.

[42]  Bruce Hendrickson,et al.  A Multi-Level Algorithm For Partitioning Graphs , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[43]  Renjie Liao,et al.  Efficient Graph Generation with Graph Recurrent Attention Networks , 2019, NeurIPS.

[44]  Ronald R. Coifman,et al.  Multiscale Wavelets on Trees, Graphs and High Dimensional Data: Theory and Applications to Semi Supervised Learning , 2010, ICML.

[45]  Jan Eric Lenssen,et al.  Fast Graph Representation Learning with PyTorch Geometric , 2019, ArXiv.

[46]  Pierre Vandergheynst,et al.  A Multiscale Pyramid Transform for Graph Signals , 2013, IEEE Transactions on Signal Processing.

[47]  Lee M. Gunderson,et al.  A Unifying Framework for Spectrum-Preserving Graph Sparsification and Coarsening , 2019, NeurIPS.

[48]  Pierre Vandergheynst,et al.  Spectrally approximating large graphs with smaller graphs , 2018, ICML.

[49]  Nicola De Cao,et al.  MolGAN: An implicit generative model for small molecular graphs , 2018, ArXiv.

[50]  J. Jost,et al.  Spectra of combinatorial Laplace operators on simplicial complexes , 2011, 1105.2712.

[51]  Shaun M. Fallat,et al.  ON TWO CONJECTURES REGARDING AN INVERSE EIGENVALUE PROBLEM FOR ACYCLIC SYMMETRIC MATRICES , 2004 .

[52]  Pasquale Minervini,et al.  Convolutional 2D Knowledge Graph Embeddings , 2017, AAAI.

[53]  Yin Tat Lee,et al.  Constructing Linear-Sized Spectral Sparsification in Almost-Linear Time , 2015, 2015 IEEE 56th Annual Symposium on Foundations of Computer Science.

[54]  Vipin Kumar,et al.  A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs , 1998, SIAM J. Sci. Comput..

[55]  Nikhil Srivastava,et al.  Graph sparsification by effective resistances , 2008, SIAM J. Comput..

[56]  Jingrui He,et al.  Misc-GAN: A Multi-scale Generative Model for Graphs , 2019, Front. Big Data.

[57]  Zhang Yan,et al.  Network Representation Learning , 2015 .

[58]  Ralf Diekmann,et al.  PARTY - A Software Library for Graph Partitioning , 1997 .

[59]  Nikhil Srivastava,et al.  Twice-ramanujan sparsifiers , 2008, STOC '09.

[60]  Yin Tat Lee,et al.  An SDP-based algorithm for linear-sized spectral sparsification , 2017, STOC.

[61]  Giovanni Chierchia,et al.  GOT: An Optimal Transport framework for Graph comparison , 2019, NeurIPS.

[62]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[63]  Jie Chen,et al.  Unsupervised Learning of Graph Hierarchical Abstractions with Differentiable Coarsening and Optimal Transport , 2019, AAAI.

[64]  Tommi Jaakkola,et al.  Solving graph compression via optimal transport , 2019, NeurIPS.

[65]  Cao Xiao,et al.  Constrained Generation of Semantically Valid Graphs via Regularizing Variational Autoencoders , 2018, NeurIPS.

[66]  Srinivasan Parthasarathy,et al.  Network Representation Learning: Consolidation and Renewed Bearing , 2019, ArXiv.