Learning Algebraic Multigrid Using Graph Neural Networks

Efficient numerical solvers for sparse linear systems are crucial in science and engineering. One of the fastest methods for solving large-scale sparse linear systems is algebraic multigrid (AMG). The main challenge in the construction of AMG algorithms is the selection of the prolongation operator -- a problem-dependent sparse matrix which governs the multiscale hierarchy of the solver and is critical to its efficiency. Over many years, numerous methods have been developed for this task, and yet there is no known single right answer except in very special cases. Here we propose a framework for learning AMG prolongation operators for linear systems with sparse symmetric positive (semi-) definite matrices. We train a single graph neural network to learn a mapping from an entire class of such matrices to prolongation operators, using an efficient unsupervised loss function. Experiments on a broad class of problems demonstrate improved convergence rates compared to classical AMG, demonstrating the potential utility of neural networks for developing sparse system solvers.

[1]  Joan Bruna,et al.  Spectral Networks and Locally Connected Networks on Graphs , 2013, ICLR.

[2]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[3]  Joan Bruna,et al.  On the equivalence between graph isomorphism testing and function approximation with GNNs , 2019, NeurIPS.

[4]  D FalgoutRobert An Introduction to Algebraic Multigrid , 2006 .

[5]  Thomas A. Manteuffel,et al.  Algebraic multigrid for directed graph Laplacian linear systems (NS‐LAMG) , 2018, Numer. Linear Algebra Appl..

[6]  K. Stuben,et al.  Algebraic Multigrid (AMG) : An Introduction With Applications , 2000 .

[7]  Mingui Sun,et al.  Solving partial differential equations in real-time using artificial neural network signal processing as an alternative to finite-element analysis , 2003, International Conference on Neural Networks and Signal Processing, 2003. Proceedings of the 2003.

[8]  Thomas A. Manteuffel,et al.  Multilevel Adaptive Aggregation for Markov Chains, with Application to Web Ranking , 2008, SIAM J. Sci. Comput..

[9]  Ronen Basri,et al.  Learning to Optimize Multigrid PDE Solvers , 2019, ICML.

[10]  Elena Virnik,et al.  An Algebraic Multigrid Preconditioner for a Class of Singular M-Matrices , 2007, SIAM J. Sci. Comput..

[11]  Talgat Daulbaev,et al.  Deep Multigrid: learning prolongation and restriction matrices , 2017, 1711.03825.

[12]  Luke N. Olson,et al.  Parallel coarse‐grid selection , 2007, Numer. Linear Algebra Appl..

[13]  Hans De Sterck,et al.  Reducing Complexity in Parallel Algebraic Multigrid Preconditioners , 2004, SIAM J. Matrix Anal. Appl..

[14]  StübenKlaus Algebraic multigrid (AMG) , 1983 .

[15]  J. W. Ruge,et al.  4. Algebraic Multigrid , 1987 .

[16]  Jacob B. Schroder,et al.  A General Interpolation Strategy for Algebraic Multigrid Using Energy Minimization , 2011, SIAM J. Sci. Comput..

[17]  Pascal Fua,et al.  Backpropagation-Friendly Eigendecomposition , 2019, NeurIPS.

[18]  David L. Dill,et al.  Learning a SAT Solver from Single-Bit Supervision , 2018, ICLR.

[19]  Irad Yavneh,et al.  Square and stretch multigrid for stochastic matrix eigenproblems , 2010, Numer. Linear Algebra Appl..

[20]  Jonas Schmitt,et al.  Optimizing Geometric Multigrid Methods with Evolutionary Computation , 2019, ArXiv.

[21]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[22]  D. Brandt,et al.  Multi-level adaptive solutions to boundary-value problems math comptr , 1977 .

[23]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[24]  Thomas A. Manteuffel,et al.  Algebraic multigrid for higher-order finite elements , 2005 .

[25]  Joan Bruna,et al.  Deep Convolutional Networks on Graph-Structured Data , 2015, ArXiv.

[26]  D. Bartuschat Algebraic Multigrid , 2007 .

[27]  K. Stüben Algebraic multigrid (AMG): experiences and comparisons , 1983 .

[28]  Graham Horton,et al.  A multi-level solution algorithm for steady-state Markov chains , 1994, SIGMETRICS.

[29]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[30]  R.D. Falgout,et al.  An Introduction to Algebraic Multigrid Computing , 2006, Computing in Science & Engineering.

[31]  Robert D. Falgout,et al.  Coarse-Grid Selection for Parallel Algebraic Multigrid , 1998, IRREGULAR.

[32]  Yaron Lipman,et al.  Provably Powerful Graph Networks , 2019, NeurIPS.

[33]  F. Scarselli,et al.  A new model for learning in graph domains , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[34]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[35]  Razvan Pascanu,et al.  Relational inductive biases, deep learning, and graph networks , 2018, ArXiv.

[36]  Achi Brandt,et al.  Lean Algebraic Multigrid (LAMG): Fast Graph Laplacian Linear Solver , 2011, SIAM J. Sci. Comput..

[37]  Zhuwen Li,et al.  Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search , 2018, NeurIPS.

[38]  Stefano Ermon,et al.  Learning Neural PDE Solvers with Convergence Guarantees , 2019, ICLR.

[39]  William L. Briggs,et al.  A multigrid tutorial , 1987 .

[40]  Marian Brezina,et al.  Algebraic multigrid by smoothed aggregation for second and fourth order elliptic problems , 2005, Computing.

[41]  Graham Horton,et al.  A multi-level solution algorithm for steady-state Markov chains , 1994, SIGMETRICS.

[42]  Artem Napov,et al.  An Efficient Multigrid Method for Graph Laplacian Systems II: Robust Aggregation , 2017, SIAM J. Sci. Comput..

[43]  Andrew V. Knyazev,et al.  Toward the Optimal Preconditioned Eigensolver: Locally Optimal Block Preconditioned Conjugate Gradient Method , 2001, SIAM J. Sci. Comput..

[44]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[45]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[46]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[47]  Panayot S. Vassilevski,et al.  Element-Free AMGe: General Algorithms for Computing Interpolation Weights in AMG , 2001, SIAM J. Sci. Comput..

[48]  Thomas A. Manteuffel,et al.  Algebraic Multigrid Based on Element Interpolation (AMGe) , 2000, SIAM J. Sci. Comput..

[49]  Thomas A. Manteuffel,et al.  Smoothed Aggregation Multigrid for Markov Chains , 2010, SIAM J. Sci. Comput..

[50]  Jonathan Richard Shewchuk,et al.  Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator , 1996, WACG.

[51]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.