Multigrid with Rough Coefficients and Multiresolution Operator Decomposition from Hierarchical Information Games

We introduce a near-linear complexity (geometric and meshless/algebraic) multigrid/multiresolution method for PDEs with rough ($L^\infty$) coefficients with rigorous a-priori accuracy and performance estimates. The method is discovered through a decision/game theory formulation of the problems of (1) identifying restriction and interpolation operators (2) recovering a signal from incomplete measurements based on norm constraints on its image under a linear operator (3) gambling on the value of the solution of the PDE based on a hierarchy of nested measurements of its solution or source term. The resulting elementary gambles form a hierarchy of (deterministic) basis functions of $H^1_0(\Omega)$ (gamblets) that (1) are orthogonal across subscales/subbands with respect to the scalar product induced by the energy norm of the PDE (2) enable sparse compression of the solution space in $H^1_0(\Omega)$ (3) induce an orthogonal multiresolution operator decomposition. The operating diagram of the multigrid method is that of an inverted pyramid in which gamblets are computed locally (by virtue of their exponential decay), hierarchically (from fine to coarse scales) and the PDE is decomposed into a hierarchy of independent linear systems with uniformly bounded condition numbers. The resulting algorithm is parallelizable both in space (via localization) and in bandwith/subscale (subscales can be computed independently from each other). Although the method is deterministic it has a natural Bayesian interpretation under the measure of probability emerging (as a mixed strategy) from the information game formulation and multiresolution approximations form a martingale with respect to the filtration induced by the hierarchy of nested measurements.

[1]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[2]  B. Engquist,et al.  Wavelet-Based Numerical Homogenization with Applications , 2002 .

[3]  R. Vershynin,et al.  One sketch for all: fast algorithms for compressed sensing , 2007, STOC '07.

[4]  G. Fasshauer Meshfree Methods , 2004 .

[5]  I. Babuska,et al.  Special finite element methods for a class of second order elliptic problems with rough coefficients , 1994 .

[6]  G. Wahba,et al.  A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines , 1970 .

[7]  Pablo A. Parrilo,et al.  Rank-Sparsity Incoherence for Matrix Decomposition , 2009, SIAM J. Optim..

[8]  Thomas Y. Hou,et al.  A Multiscale Finite Element Method for Elliptic Problems in Composite Materials and Porous Media , 1997 .

[9]  Michael A. Osborne,et al.  Probabilistic Integration: A Role for Statisticians in Numerical Analysis? , 2015 .

[10]  Michael A. Osborne,et al.  Probabilistic numerics and uncertainty in computations , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[11]  Ronald R. Coifman,et al.  Multiscale Inversion of Elliptic Operators , 1998 .

[12]  J. Duchon Sur l’erreur d’interpolation des fonctions de plusieurs variables par les $D^m$-splines , 1978 .

[13]  Houman Owhadi,et al.  Anomalous slow diffusion from perpetual homogenization , 2001, math/0105165.

[14]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[15]  Leslie Greengard,et al.  A fast algorithm for particle simulations , 1987 .

[16]  A. Copeland Review: John von Neumann and Oskar Morgenstern, Theory of games and economic behavior , 1945 .

[17]  Joel A. Tropp,et al.  Recovery of short, complex linear combinations via /spl lscr//sub 1/ minimization , 2005, IEEE Transactions on Information Theory.

[18]  E. Novak,et al.  Tractability of Multivariate Problems Volume II: Standard Information for Functionals , 2010 .

[19]  Arkadi S. Nemirovsky,et al.  Information-based complexity of linear operator equations , 1992, J. Complex..

[20]  D. Myers Kriging, cokriging, radial basis functions and the role of positive definiteness , 1992 .

[21]  Marian Brezina,et al.  Energy Optimization of Algebraic Multigrid Bases , 1998, Computing.

[22]  Nathan Halko,et al.  Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions , 2009, SIAM Rev..

[23]  E Weinan,et al.  The heterogeneous multiscale method* , 2012, Acta Numerica.

[24]  Houman Owhadi,et al.  On the Brittleness of Bayesian Inference , 2013, SIAM Rev..

[25]  S. Spagnolo,et al.  Sulla convergenza di soluzioni di equazioni paraboliche ed ellittiche , 1968 .

[26]  H. Ohanian,et al.  The linear approximation , 2013 .

[27]  Sudipto Guha,et al.  Near-optimal sparse fourier representations via sampling , 2002, STOC '02.

[28]  Irad Yavneh,et al.  Why Multigrid Methods Are So Efficient , 2006, Computing in Science & Engineering.

[29]  Antoine Gloria,et al.  An Analytical Framework for the Numerical Homogenization of Monotone Elliptic Operators and Quasiconvex Energies , 2006, Multiscale Model. Simul..

[30]  C. Scovel,et al.  Brittleness of Bayesian inference and new Selberg formulas , 2013, 1304.7046.

[31]  F. M. Larkin Gaussian measure in Hilbert space and applications in numerical analysis , 1972 .

[32]  Panayot S. Vassilevski,et al.  Stabilizing the Hierarchical Basis by Approximate Wavelets, I: Theory , 1997, Numer. Linear Algebra Appl..

[33]  Philipp Hennig,et al.  Probabilistic Interpretation of Linear Solvers , 2014, SIAM J. Optim..

[34]  H. Owhadi,et al.  Polyharmonic homogenization, rough polyharmonic splines and sparse super-localization , 2012, 1212.0812.

[35]  J. Nash NON-COOPERATIVE GAMES , 1951, Classics in Game Theory.

[36]  George Papanicolaou,et al.  A Framework for Adaptive Multiscale Methods for Elliptic Problems , 2008, Multiscale Model. Simul..

[37]  M. Girolami,et al.  Bayesian Solution Uncertainty Quantification for Differential Equations , 2013 .

[38]  Zongmin Wu,et al.  Local error estimates for radial basis function interpolation of scattered data , 1993 .

[39]  B. Engquist,et al.  Convergence of a Multigrid Method for Elliptic Equations with Highly Oscillatory Coefficients , 1997 .

[40]  Yalchin Efendiev,et al.  Multiscale finite element methods for porous media flows and their applications , 2007 .

[41]  C. Chui,et al.  Article in Press Applied and Computational Harmonic Analysis a Randomized Algorithm for the Decomposition of Matrices , 2022 .

[42]  D. Bartuschat Algebraic Multigrid , 2007 .

[43]  J. E. H. Shaw,et al.  A Quasirandom Approach to Integration in Bayesian Statistics , 1988 .

[44]  Houman Owhadi,et al.  Multiscale Homogenization with Bounded Ratios and Anomalous Slow Diffusion , 2001 .

[45]  E. Novak,et al.  Tractability of Multivariate Problems , 2008 .

[46]  David Duvenaud,et al.  Probabilistic ODE Solvers with Runge-Kutta Means , 2014, NIPS.

[47]  J. Skilling Bayesian Solution of Ordinary Differential Equations , 1992 .

[48]  Houman Owhadi,et al.  Gamblets for opening the complexity-bottleneck of implicit schemes for hyperbolic and parabolic ODEs/PDEs with rough coefficients , 2016, J. Comput. Phys..

[49]  Ronald R. Coifman,et al.  Signal and image representation in combined spaces , 1998 .

[50]  K. St A review of algebraic multigrid , 2001 .

[51]  Panagiotis E. Souganidis,et al.  Asymptotic and numerical homogenization , 2008, Acta Numerica.

[52]  Klaus Ritter,et al.  Bayesian numerical analysis , 2000 .

[53]  A. O'Hagan,et al.  Bayes–Hermite quadrature , 1991 .

[54]  S. Kozlov AVERAGING OF RANDOM OPERATORS , 1980 .

[55]  W. Hackbusch,et al.  A fast iterative method for solving poisson’s equation in a general region , 1978 .

[56]  Jean Duchon,et al.  Interpolation des fonctions de deux variables suivant le principe de la flexion des plaques minces , 1976 .

[57]  Xiao-Hui Wu,et al.  Challenges and Technologies in Reservoir Modeling , 2009 .

[58]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[59]  W. Hackbusch,et al.  An introduction to hierarchical matrices , 2001 .

[60]  R. N. Desmarais,et al.  Interpolation using surface splines. , 1972 .

[61]  Douglas Stott Parker,et al.  Using randomization to make recursive matrix algorithms practical , 1999, J. Funct. Program..

[62]  A. Bensoussan,et al.  Asymptotic analysis for periodic structures , 1979 .

[63]  Daniel Peterseim,et al.  Localization of elliptic multiscale problems , 2011, Math. Comput..

[64]  William F. Moss,et al.  Decay rates for inverses of band matrices , 1984 .

[65]  Yalchin Efendiev,et al.  Accurate multiscale finite element methods for two-phase flow simulations , 2006, J. Comput. Phys..

[66]  Yalchin Efendiev,et al.  Spectral Element Agglomerate Algebraic Multigrid Methods for Elliptic Problems with High-Contrast Coefficients , 2011 .

[67]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[68]  F. Otto,et al.  Quantification of ergodicity in stochastic homogenization: optimal bounds via spectral gap on Glauber dynamics , 2013 .

[69]  I. Babuska,et al.  Generalized Finite Element Methods: Their Performance and Their Relation to Mixed Methods , 1983 .

[70]  J. Shewchuk An Introduction to the Conjugate Gradient Method Without the Agonizing Pain , 1994 .

[71]  Erich Novak,et al.  Essays on the Complexity of Continuous Problems , 2009 .

[72]  Klaus Ritter,et al.  Average-case analysis of numerical problems , 2000, Lecture notes in mathematics.

[73]  Houman Owhadi,et al.  Localized Bases for Finite-Dimensional Homogenization Approximations with Nonseparated Scales and High Contrast , 2010, Multiscale Model. Simul..

[74]  G. Beylkin,et al.  A Multiresolution Strategy for Reduction of Elliptic PDEs and Eigenvalue Problems , 1998 .

[75]  Henryk Wozniakowski,et al.  Information-based complexity , 1987, Nature.

[76]  Mario Bebendorf,et al.  Hierarchical Matrices: A Means to Efficiently Solve Elliptic Boundary Value Problems , 2008 .

[77]  H. Weinberger,et al.  An optimal Poincaré inequality for convex domains , 1960 .

[78]  Houman Owhadi,et al.  Optimal Uncertainty Quantification , 2010, SIAM Rev..

[79]  Edward W. Packel,et al.  The algorithm designer versus nature: A game-theoretic approach to information-based complexity , 1987, J. Complex..

[80]  Thomas Y. Hou,et al.  Optimal Local Multi-scale Basis Functions for Linear Elliptic Equations with Rough Coefficient , 2015, 1508.00346.

[81]  Jinchao Xu,et al.  UNIFORM CONVERGENT MULTIGRID METHODS FOR ELLIPTIC PROBLEMS WITH STRONGLY DISCONTINUOUS COEFFICIENTS , 2008 .

[82]  Assyr Abdulle,et al.  Heterogeneous Multiscale FEM for Diffusion Problems on Rough Surfaces , 2005, Multiscale Model. Simul..

[83]  H. Owhadi,et al.  Flux Norm Approach to Finite Dimensional Homogenization Approximations with Non-Separated Scales and High Contrast , 2009, 0901.1463.

[84]  Panayot S. Vassilevski,et al.  Stabilizing the Hierarchical Basis by Approximate Wavelets II: Implementation and Numerical Results , 1998, SIAM J. Sci. Comput..

[85]  Michael A. Osborne,et al.  Probabilistic Integration , 2015, ArXiv.

[86]  Panayot S. Vassilevski,et al.  On Two Ways of Stabilizing the Hierarchical Basis Multilevel Methods , 1997, SIAM Rev..

[87]  Alan M. Frieze,et al.  Fast monte-carlo algorithms for finding low-rank approximations , 2004, JACM.

[88]  Doron Levy,et al.  On Wavelet-Based Numerical Homogenization , 2005, Multiscale Model. Simul..

[89]  N. S. Barnett,et al.  Private communication , 1969 .

[90]  H. Owhadi,et al.  Metric‐based upscaling , 2007 .

[91]  Robert Lipton,et al.  Optimal Local Approximation Spaces for Generalized Finite Element Methods with Application to Multiscale Problems , 2010, Multiscale Model. Simul..

[92]  Panayot S. Vassilevski,et al.  Stabilizing the Hierarchical Basis by Approximate Wavelets, I: Theory , 1997 .

[93]  A. Wald Statistical Decision Functions Which Minimize the Maximum Risk , 1945 .

[94]  E. Rowland Theory of Games and Economic Behavior , 1946, Nature.

[95]  J. Neumann Zur Theorie der Gesellschaftsspiele , 1928 .

[96]  Stefan A. Sauter,et al.  The AL Basis for the Solution of Elliptic Problems in Heterogeneous Media , 2012, Multiscale Model. Simul..

[97]  Houman Owhadi,et al.  Toward Machine Wald , 2015, 1508.02449.

[98]  A. Brandt Multi-level adaptive technique (MLAT) for fast numerical solution to boundary value problems , 1973 .

[99]  Xin Wang,et al.  Transfer-of-approximation Approaches for Subgrid Modeling , 2012 .

[100]  Henryk Wozniakowski Probabilistic setting of information-based complexity , 1986, J. Complex..

[101]  Edmond Chow,et al.  Multilevel block factorizations in generalized hierarchical bases , 2003, Numer. Linear Algebra Appl..

[102]  Panayot S. Vassilevski,et al.  General Constrained Energy Minimization Interpolation Mappings for AMG , 2010, SIAM J. Sci. Comput..

[103]  Guillaume Bal,et al.  Corrector Theory for MsFEM and HMM in Random Media , 2010, Multiscale Model. Simul..

[104]  Houman Owhadi,et al.  Bayesian Numerical Homogenization , 2014, Multiscale Model. Simul..

[105]  Panayot S. Vassilevski,et al.  Multilevel Preconditioning Matrices and Multigrid V-Cycle Methods , 1989 .

[106]  H. Yserentant On the multi-level splitting of finite element spaces , 1986 .

[107]  Patrick R. Conrad,et al.  Probability Measures for Numerical Solutions of Differential Equations , 2015, 1506.04592.

[108]  G. Beylkin,et al.  A Multiresolution Strategy for Numerical Homogenization , 1995 .

[109]  R. Bank,et al.  The hierarchical basis multigrid method , 1988 .

[110]  H. Poincaré Calcul des Probabilités , 1912 .

[111]  John E. Osborn,et al.  Can a finite element method perform arbitrarily badly? , 2000, Math. Comput..

[112]  M. Urner Scattered Data Approximation , 2016 .

[113]  Jean Duchon,et al.  Splines minimizing rotation-invariant semi-norms in Sobolev spaces , 1976, Constructive Theory of Functions of Several Variables.

[114]  C. Scovel,et al.  Brittleness of Bayesian Inference Under Finite Information in a Continuous World , 2013, 1304.6772.

[115]  Tony F. Chan,et al.  An Energy-minimizing Interpolation for Robust Multigrid Methods , 1999, SIAM J. Sci. Comput..

[116]  Mario Bebendorf,et al.  Efficient inversion of the Galerkin matrix of general second-order elliptic operators with nonsmooth coefficients , 2004, Math. Comput..

[117]  J. Mandel,et al.  Energy optimization of algebraic multigrid bases , 1999 .

[118]  William W. Symes Transfer of approximation and numerical homogenization of hyperbolic boundary value problems with a continuum of scales , 2010 .

[119]  Grégoire Allaire,et al.  A Multiscale Finite Element Method for Numerical Homogenization , 2005, Multiscale Model. Simul..

[120]  R. Horne,et al.  Computing Absolute Transmissibility in the Presence of Fine-Scale Heterogeneity , 1987 .

[121]  F. Murat,et al.  Compacité par compensation , 1978 .

[122]  Jinchao Xu,et al.  On an energy minimizing basis for algebraic multigrid methods , 2004 .