Black-box learning of multigrid parameters

Abstract This paper studies the optimality of the restriction and prolongation operators in the geometric multigrid method (GMG). GMG is used in solving discretized partial differential equation (PDE) and it relies greatly on the restriction and prolongation operators. Many methods to find these operators were proposed, but most of them have limited optimality proofs. To study their optimality we introduce stochastic convergence functional, which estimates the spectral radius of the iteration matrix for given GMG parameters. We implement the GMG method in a modern machine learning framework that can automatically compute the gradients of the introduced convergence functional with respect to restriction and prolongation operators. Therefore, we can minimize the proposed functional starting from some initial parameters and get better ones after some iterations of stochastic gradient descent. To illustrate the performance of the proposed approach, we carry out experiments on the discretized Poisson equation, Helmholtz equation and singularly perturbed convection–diffusion equation and demonstrate that proposed approach gives operators, which lead to faster convergence.

[1]  Maxim A. Olshanskii,et al.  Iterative Methods for Linear Systems - Theory and Applications , 2014 .

[2]  Cornelis W. Oosterlee,et al.  A Genetic Search for Optimal Multigrid Components Within a Fourier Analysis Setting , 2002, SIAM J. Sci. Comput..

[3]  Jacob B. Schroder,et al.  A General Interpolation Strategy for Algebraic Multigrid Using Energy Minimization , 2011, SIAM J. Sci. Comput..

[4]  Sivan Toledo,et al.  Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix , 2011, JACM.

[5]  J. W. Ruge,et al.  4. Algebraic Multigrid , 1987 .

[6]  Daniel Kressner,et al.  Subspace Methods for Computing the Pseudospectral Abscissa and the Stability Radius , 2014, SIAM J. Matrix Anal. Appl..

[7]  Ronen Basri,et al.  Learning to Optimize Multigrid PDE Solvers , 2019, ICML.

[8]  Frédéric Magoulès,et al.  A hybrid multigrid method for convection-diffusion problems , 2014, J. Comput. Appl. Math..

[9]  Emre Mengi,et al.  Numerical Optimization of Eigenvalues of Hermitian Matrix Functions , 2011, SIAM J. Matrix Anal. Appl..

[10]  Achi Brandt,et al.  Bootstrap AMG , 2011, SIAM J. Sci. Comput..

[11]  P. M. De Zeeuw,et al.  Matrix-dependent prolongations and restrictions in a blackbox multigrid solver , 1990 .

[12]  L. Watson,et al.  Modern homotopy methods in optimization , 1989 .

[13]  Thomas A. Manteuffel,et al.  Adaptive Smoothed Aggregation (αSA) , 2004, SIAM J. Sci. Comput..

[14]  L. T. Watson,et al.  Modern homology methods in optimization , 1989 .

[15]  Christiaan C. Stolk,et al.  A Multigrid Method for the Helmholtz Equation with Optimized Coarse Grid Corrections , 2013, SIAM J. Sci. Comput..

[16]  John Salvatier,et al.  Theano: A Python framework for fast computation of mathematical expressions , 2016, ArXiv.

[17]  William L. Briggs,et al.  A multigrid tutorial , 1987 .

[18]  Yurii Nesterov,et al.  Smoothing Technique and its Applications in Semidefinite Optimization , 2004, Math. Program..

[19]  J. Dendy Black box multigrid , 1982 .

[20]  Thomas A. Manteuffel,et al.  Adaptive Algebraic Multigrid , 2005, SIAM J. Sci. Comput..

[21]  Yurii Nesterov,et al.  Optimizing the Spectral Radius , 2013, SIAM J. Matrix Anal. Appl..

[22]  Lu Wang,et al.  A nearly optimal multigrid method for general unstructured grids , 2016, Numerische Mathematik.

[23]  Walter Baur,et al.  The Complexity of Partial Derivatives , 1983, Theor. Comput. Sci..

[24]  V. Kozyakin On accuracy of approximation of the spectral radius by the Gelfand formula , 2008, 0810.2856.

[25]  Piet Hemker,et al.  On the order of prolongations and restrictions in multigrid procedures , 1990 .

[26]  Yin Wang,et al.  Fast and high accuracy multiscale multigrid method with multiple coarse grid updating strategy for the 3D convection-diffusion equation , 2013, Comput. Math. Appl..

[27]  Barak A. Pearlmutter,et al.  Automatic differentiation in machine learning: a survey , 2015, J. Mach. Learn. Res..

[28]  Jinchao Xu,et al.  The auxiliary space method and optimal multigrid preconditioning techniques for unstructured grids , 1996, Computing.

[29]  Irene Livshits,et al.  A scalable multigrid method for solving indefinite Helmholtz equations with constant wave numbers , 2014, Numer. Linear Algebra Appl..

[30]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[31]  M. Hutchinson A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines , 1989 .

[32]  H. Yserentant Old and new convergence proofs for multigrid methods , 1993, Acta Numerica.

[33]  K. Stüben A review of algebraic multigrid , 2001 .