Non-intrusive reduced order modeling of nonlinear problems using neural networks

We develop a non-intrusive reduced basis (RB) method for parametrized steady-state partial differential equations (PDEs). The method extracts a reduced basis from a collection of high-fidelity solutions via a proper orthogonal decomposition (POD) and employs artificial neural networks (ANNs), particularly multi-layer perceptrons (MLPs), to accurately approximate the coefficients of the reduced model. The search for the optimal number of neurons and the minimum amount of training samples to avoid overfitting is carried out in the offline phase through an automatic routine, relying upon a joint use of the latin hypercube sampling (LHS) and the Levenberg-Marquardt training algorithm. This guarantees a complete offline-online decoupling, leading to an efficient RB method referred to as POD-NN suitable also for general nonlinear problems with a non-affine parametric dependence. Numerical studies are presented for the nonlinear Poisson equation and for driven cavity viscous flows, modeled through the steady incompressible Navier-Stokes equations. Both physical and geometrical parametrizations are considered. Several results confirm the accuracy of the POD-NN method and show the substantial speed-up enabled at the online stage as compared to a traditional RB strategy.

[1]  Benjamin Stamm,et al.  EFFICIENT GREEDY ALGORITHMS FOR HIGH-DIMENSIONAL PARAMETER SPACES WITH APPLICATIONS TO EMPIRICAL INTERPOLATION AND REDUCED BASIS METHODS ∗ , 2014 .

[2]  A. Patera,et al.  A PRIORI CONVERGENCE OF THE GREEDY ALGORITHM FOR THE PARAMETRIZED REDUCED BASIS METHOD , 2012 .

[3]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[4]  Simone Deparis,et al.  Reduced Basis Error Bound Computation of Parameter-Dependent Navier-Stokes Equations by the Natural Norm Approach , 2008, SIAM J. Numer. Anal..

[5]  David Amsallem,et al.  Efficient model reduction of parametrized systems by matrix discrete empirical interpolation , 2015, J. Comput. Phys..

[6]  Carl de Boor,et al.  A Practical Guide to Splines , 1978, Applied Mathematical Sciences.

[7]  Charu C. Aggarwal,et al.  Neural Networks and Deep Learning , 2018, Springer International Publishing.

[8]  H. P. Lee,et al.  PROPER ORTHOGONAL DECOMPOSITION AND ITS APPLICATIONS—PART I: THEORY , 2002 .

[9]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[10]  Gianluigi Rozza,et al.  Supremizer stabilization of POD-Galerkin approximation of parametrized Navier-Stokes equations , 2015 .

[11]  William F. Mitchell,et al.  A Collection of 2D Elliptic Problems for Testing Adaptive Algorithms , 2010 .

[12]  S. Volkwein,et al.  MODEL REDUCTION USING PROPER ORTHOGONAL DECOMPOSITION , 2008 .

[13]  D. Rovas,et al.  Reduced--Basis Output Bound Methods for Parametrized Partial Differential Equations , 2002 .

[14]  J. Hesthaven,et al.  Greedy Nonintrusive Reduced Order Model for Fluid Dynamics , 2018, AIAA Journal.

[15]  Fabien Casenave,et al.  A nonintrusive reduced basis method applied to aeroacoustic simulations , 2014, Adv. Comput. Math..

[16]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[17]  Andrea Manzoni,et al.  Automatic reduction of PDEs defined on domains with variable shape , 2016 .

[18]  O. L. Maître,et al.  Spectral Methods for Uncertainty Quantification: With Applications to Computational Fluid Dynamics , 2010 .

[19]  Implementation of Finite Element-Based Navier-Stokes Solver 2 . 094 – Project , 2004 .

[20]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[21]  Rolf Rannacher,et al.  Finite Element Methods for the Incompressible Navier-Stokes Equations , 2000 .

[22]  Max Gunzburger,et al.  POD and CVT-based reduced-order modeling of Navier-Stokes flows , 2006 .

[23]  Jens L. Eftang,et al.  Reduced basis methods for parametrized partial differential equations , 2011 .

[24]  Yvon Maday,et al.  Reduced basis method for the rapid and reliable solution of partial differential equations , 2006 .

[25]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[26]  Jian Cheng,et al.  Latin Hypercube Sampling in Bayesian Networks , 2000, FLAIRS.

[27]  N. Nguyen,et al.  An ‘empirical interpolation’ method: application to efficient reduced-basis discretization of partial differential equations , 2004 .

[28]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[29]  Robert L. Mercer,et al.  The Mathematics of Statistical Machine Translation: Parameter Estimation , 1993, CL.

[30]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[31]  Gianluigi Rozza,et al.  An improvement on geometrical parameterizations by transfinite maps , 2014 .

[32]  A. Quarteroni,et al.  Reduced Basis Methods for Partial Differential Equations: An Introduction , 2015 .

[33]  E B Lee,et al.  Foundations of optimal control theory , 1967 .

[34]  M. Bendsøe,et al.  Topology Optimization: "Theory, Methods, And Applications" , 2011 .

[35]  Danny C. Sorensen,et al.  Nonlinear Model Reduction via Discrete Empirical Interpolation , 2010, SIAM J. Sci. Comput..

[36]  A. Quarteroni.  Numerical Models for Differential Problems , 2009 .

[37]  Erich Novak,et al.  High dimensional polynomial interpolation on sparse grids , 2000, Adv. Comput. Math..

[38]  J. Hesthaven,et al.  Certified Reduced Basis Methods for Parametrized Partial Differential Equations , 2015 .