A reduced order Schwarz method for nonlinear multiscale elliptic equations based on two-layer neural networks

Neural networks are powerful tools for approximating high dimensional data that have been used in many contexts, including solution of partial differential equations (PDEs). We describe a solver for multiscale fully nonlinear elliptic equations that makes use of domain decomposition, an accelerated Schwarz framework, and two-layer neural networks to approximate the boundary-to-boundary map for the subdomains, which is the key step in the Schwarz procedure. Conventionally, the boundary-to-boundary map requires solution of boundary-value elliptic problems on each subdomain. By leveraging the compressibility of multiscale problems, our approach trains the neural network offline to serve as a surrogate for the usual implementation of the boundary-to-boundary map. Our method is applied to a multiscale semilinear elliptic equation and a multiscale p-Laplace equation. In both cases we demonstrate significant improvement in efficiency as well as good accuracy and generalization performance. 2020 Mathematics subject classification: 65N55, 35J66, 41A46, 68T07.

[1]  D. Joseph,et al.  Principles of non-Newtonian fluid mechanics , 1974 .

[2]  R. Ghanem,et al.  Stochastic Finite Elements: A Spectral Approach , 1990 .

[3]  P. Lions,et al.  Viscosity solutions of fully nonlinear second-order elliptic partial differential equations , 1990 .

[4]  L. Evans.  Periodic homogenisation of certain fully nonlinear partial differential equations , 1992, Proceedings of the Royal Society of Edinburgh: Section A Mathematics.

[5]  G. Allaire.  Homogenization and two-scale convergence , 1992 .

[6]  Andrew R. Barron,et al.  Universal approximation bounds for superpositions of a sigmoidal function , 1993, IEEE Trans. Inf. Theory.

[7]  L. Caffarelli,et al.  Fully Nonlinear Elliptic Equations , 1995 .

[8]  I. Babuska,et al.  The Partition of Unity Method , 1997 .

[9]  Thomas Y. Hou,et al.  A Multiscale Finite Element Method for Elliptic Problems in Composite Materials and Porous Media , 1997 .

[10]  Zi-Cai Li,et al.  Schwarz Alternating Method , 1998 .

[11]  Thomas Y. Hou,et al.  Convergence of a multiscale finite element method for elliptic problems with rapidly oscillating coefficients , 1999, Math. Comput..

[12]  Thomas Y. Hou,et al.  Convergence of a Nonconforming Multiscale Finite Element Method , 2000, SIAM J. Numer. Anal..

[13]  Dongbin Xiu,et al.  The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations , 2002, SIAM J. Sci. Comput..

[14]  E. Weinan,et al.  Analysis of the heterogeneous multiscale method for elliptic homogenization problems , 2004 .

[15]  Raúl Tempone,et al.  Galerkin Finite Element Approximations of Stochastic Elliptic Partial Differential Equations , 2004, SIAM J. Numer. Anal..

[16]  T. Hou,et al.  Multiscale Finite Element Methods for Nonlinear Problems and Their Applications , 2004 .

[17]  P. Frauenfelder,et al.  Finite elements for elliptic problems with stochastic coefficients , 2005 .

[18]  Assyr Abdulle,et al.  Heterogeneous Multiscale FEM for Diffusion Problems on Rough Surfaces , 2005, Multiscale Model. Simul..

[19]  ZHANGXIN CHEN,et al.  Analysis of the Multiscale Finite Element Method for Nonlinear and Random Homogenization Problems , 2007, SIAM J. Numer. Anal..

[20]  H. Owhadi,et al.  Metric‐based upscaling , 2007 .

[21]  Mario Bebendorf,et al.  Why Finite Element Discretizations Can Be Factored by Triangular Hierarchical Matrices , 2007, SIAM J. Numer. Anal..

[22]  Ruo Li,et al.  Preconditioned Descent Algorithms for p-Laplacian , 2007, J. Sci. Comput..

[23]  Martin J. Gander,et al.  Schwarz Methods over the Course of Time , 2008 .

[24]  Yalchin Efendiev,et al.  Multiscale Finite Element Methods: Theory and Applications , 2009 .

[25]  C. Schwab,et al.  Sparse high order FEM for elliptic sPDEs , 2009 .

[26]  R. DeVore,et al.  ANALYTIC REGULARITY AND POLYNOMIAL APPROXIMATION OF PARAMETRIC AND STOCHASTIC ELLIPTIC PDE'S , 2011 .

[27]  Andrea Barth,et al.  Multi-level Monte Carlo Finite Element method for elliptic PDEs with stochastic coefficients , 2011, Numerische Mathematik.

[28]  Robert Lipton,et al.  Optimal Local Approximation Spaces for Generalized Finite Element Methods with Application to Multiscale Problems , 2010, Multiscale Model. Simul..

[29]  Assyr Abdulle,et al.  Reduced basis finite element heterogeneous multiscale method for high-order discretizations of elliptic homogenization problems , 2012, J. Comput. Phys..

[30]  P. Henning,et al.  A localized orthogonal decomposition method for semi-linear elliptic problems , 2012, 1211.3551.

[31]  Assyr Abdulle,et al.  Analysis of the finite element heterogeneous multiscale method for quasilinear elliptic homogenization problems , 2013, Math. Comput..

[32]  Daniel Peterseim,et al.  Localization of elliptic multiscale problems , 2011, Math. Comput..

[33]  H. Owhadi,et al.  Polyharmonic homogenization, rough polyharmonic splines and sparse super-localization , 2012, 1212.0812.

[34]  Y. Efendiev,et al.  Generalized Multiscale Finite Element Methods. Nonlinear Elliptic Equations , 2013, 1304.5188.

[35]  A. Abdulle,et al.  Reduced basis finite element heterogeneous multiscale method for quasilinear elliptic homogenization problems , 2014 .

[36]  W. Hackbusch,et al.  Hierarchical Matrices: Algorithms and Analysis , 2015 .

[37]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[38]  M. Presho,et al.  Reduced-order multiscale modeling of nonlinear p-Laplacian flows in high-contrast media , 2015, Computational Geosciences.

[39]  Jason M. Klusowski,et al.  Risk Bounds for High-dimensional Ridge Function Combinations Including Neural Networks , 2016, 1607.01434.

[40]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[41]  E Weinan,et al.  The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems , 2017, Communications in Mathematics and Statistics.

[42]  Julien Yvonnet,et al.  Homogenization methods and multiscale modeling : Nonlinear problems , 2017 .

[43]  E Weinan,et al.  Deep Learning-Based Numerical Methods for High-Dimensional Parabolic Partial Differential Equations and Backward Stochastic Differential Equations , 2017, Communications in Mathematics and Statistics.

[44]  Houman Owhadi,et al.  Multigrid with Rough Coefficients and Multiresolution Operator Decomposition from Hierarchical Information Games , 2015, SIAM Rev..

[45]  Yalchin Efendiev,et al.  A multiscale model reduction method for nonlinear monotone elliptic equations in heterogeneous media , 2017, Networks Heterog. Media.

[46]  Qin Li,et al.  Exploring the Locally Low Dimensional Structure in Solving Random Elliptic PDEs , 2016, Multiscale Model. Simul..

[47]  Bin Dong,et al.  PDE-Net: Learning PDEs from Data , 2017, ICML.

[48]  Lexing Ying,et al.  Solving for high-dimensional committor functions using artificial neural networks , 2018, Research in the Mathematical Sciences.

[49]  Justin A. Sirignano,et al.  DGM: A deep learning algorithm for solving partial differential equations , 2017, J. Comput. Phys..

[50]  Stephen J. Wright,et al.  Schwarz iteration method for elliptic equation with rough media based on random sampling , 2019, ArXiv.

[51]  Lei Wu,et al.  Barron Spaces and the Compositional Function Spaces for Neural Network Models , 2019, ArXiv.

[52]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[53]  W. Hager,et al.  and s , 2019, Shallow Water Hydraulics.

[54]  Lexing Ying,et al.  BCR-Net: a neural network based on the nonstandard wavelet form , 2018, J. Comput. Phys..

[55]  Stephan Hoyer,et al.  Learning data-driven discretizations for partial differential equations , 2018, Proceedings of the National Academy of Sciences.

[56]  Yalchin Efendiev,et al.  Reduced-order deep learning for flow dynamics. The interplay between deep learning and model reduction , 2019, J. Comput. Phys..

[57]  George Em Karniadakis,et al.  DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators , 2019, ArXiv.

[58]  Lexing Ying,et al.  A Multiscale Neural Network Based on Hierarchical Matrices , 2018, Multiscale Model. Simul..

[59]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[60]  Lexing Ying,et al.  SwitchNet: a neural network model for forward and inverse scattering problems , 2018, SIAM J. Sci. Comput..

[61]  E Weinan,et al.  Towards a Mathematical Understanding of Neural Network-Based Machine Learning: what we know and what we don't , 2020, CSIAM Transactions on Applied Mathematics.

[62]  Yingzhou Li,et al.  Butterfly-Net2: Simplified Butterfly-Net and Fourier Transform Initialization , 2019, MSML.

[63]  Julius Berner,et al.  Numerically Solving Parametric Families of High-Dimensional Kolmogorov Partial Differential Equations via Deep Learning , 2020, NeurIPS.

[64]  Lexing Ying,et al.  Meta-learning Pseudo-differential Operators with Deep Neural Networks , 2019, J. Comput. Phys..

[65]  Lei Zhang,et al.  A Multi-Scale DNN Algorithm for Nonlinear Elliptic Equations with Multiple Scales , 2020, Communications in Computational Physics.

[66]  Lexing Ying,et al.  Solving parametric PDE problems with artificial neural networks , 2017, European Journal of Applied Mathematics.

[67]  Yingzhou Li,et al.  Butterfly-Net: Optimal Function Representation Based on Convolutional Neural Networks , 2018, Communications in Computational Physics.

[68]  Stephen J. Wright,et al.  Manifold Learning and Nonlinear Homogenization , 2020, Multiscale Model. Simul..

[69]  Ziqi Liu,et al.  Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domains , 2020, Communications in Computational Physics.

[70]  Yingzhou Li,et al.  Variational training of neural network approximations of solution maps for physical models , 2019, J. Comput. Phys..

[71]  Yalchin Efendiev,et al.  Reduced-order deep learning for flow dynamics. The interplay between deep learning and model reduction , 2020, J. Comput. Phys..

[72]  Gang Bao,et al.  Weak Adversarial Networks for High-dimensional Partial Differential Equations , 2019, J. Comput. Phys..

[73]  Stephen J. Wright,et al.  Random Sampling and Efficient Algorithms for Multiscale PDEs , 2018, SIAM J. Sci. Comput..

[74]  D. Xiu,et al.  Data-Driven Deep Learning of Partial Differential Equations in Modal Space , 2019, J. Comput. Phys..

[75]  Deep Ray,et al.  Controlling oscillations in high-order Discontinuous Galerkin schemes using artificial viscosity tuned by neural networks , 2020, J. Comput. Phys..

[76]  Stephen J. Wright,et al.  Randomized Sampling for Basis Function Construction in Generalized Finite Element Methods , 2018, Multiscale Model. Simul..

[77]  Nikola B. Kovachki,et al.  Fourier Neural Operator for Parametric Partial Differential Equations , 2020, ICLR.

[78]  Lei Zhang,et al.  Iterated numerical homogenization for multi-scale elliptic equations with monotone nonlinearity , 2021, Multiscale Model. Simul..

[79]  Qin Li,et al.  A low-rank Schwarz method for radiative transport equation with heterogeneous scattering coefficient , 2019, Multiscale Model. Simul..

[80]  P. Alam.  ‘L’ , 2021, Composites Engineering: An A–Z Guide.