Surrogate and reduced-order modeling: a comparison of approaches for large-scale statistical inverse problems [Chapter 7]

Solution of statistical inverse problems via the frequentist or Bayesian approaches described in earlier chapters can be a computationally intensive endeavor, particularly when faced with large-scale forward models characteristic of many engineering and science applications. High computational cost arises in several ways. First, thousands or millions of forward simulations may be required to evaluate estimators of interest or to characterize a posterior distribution. In the large-scale setting, performing so many forward simulations is often computationally intractable. Second, sampling may be complicated by the large dimensionality of the input space—as when the inputs are fields represented with spatial discretizations of high dimension—and

[1]  Danny C. Sorensen,et al.  The Sylvester equation and approximate balanced reduction , 2002 .

[2]  N. Nguyen,et al.  EFFICIENT REDUCED-BASIS TREATMENT OF NONAFFINE AND NONLINEAR PARTIAL DIFFERENTIAL EQUATIONS , 2007 .

[3]  C. Geyer Markov Chain Monte Carlo Maximum Likelihood , 1991 .

[4]  Habib N. Najm,et al.  Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems , 2008, J. Comput. Phys..

[5]  Christoph Schwab,et al.  Sparse Tensor Discretization of Elliptic sPDEs , 2009, SIAM J. Sci. Comput..

[6]  Fabio Nobile,et al.  An Anisotropic Sparse Grid Stochastic Collocation Method for Partial Differential Equations with Random Input Data , 2008, SIAM J. Numer. Anal..

[7]  Stefan M. Wild,et al.  Bayesian Calibration of Computationally Expensive Models Using Optimization and Radial Basis Function Approximation , 2006 .

[8]  Y. Marzouk,et al.  A stochastic collocation approach to Bayesian inference in inverse problems , 2009 .

[9]  G. Rozza,et al.  On the stability of the reduced basis method for Stokes equations in parametrized domains , 2007 .

[10]  E. Somersalo,et al.  Approximation errors and model reduction with an application in optical diffusion tomography , 2006 .

[11]  Habib N. Najm,et al.  Stochastic spectral methods for efficient Bayesian solution of inverse problems , 2005, J. Comput. Phys..

[12]  N. Cutland,et al.  On homogeneous chaos , 1991, Mathematical Proceedings of the Cambridge Philosophical Society.

[13]  W. T. Martin,et al.  The Orthogonal Development of Non-Linear Functionals in Series of Fourier-Hermite Functionals , 1947 .

[14]  I. Kevrekidis,et al.  Low‐dimensional models for complex geometry flows: Application to grooved channels and circular cylinders , 1991 .

[15]  A. Antoulas,et al.  A Survey of Model Reduction by Balanced Truncation and Some New Results , 2004 .

[16]  Karen Willcox,et al.  Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems , 2010, SIAM J. Sci. Comput..

[17]  David Galbally,et al.  Non‐linear model reduction for uncertainty quantification in large‐scale inverse problems , 2009 .

[18]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[19]  BabuskaIvo,et al.  A Stochastic Collocation Method for Elliptic Partial Differential Equations with Random Input Data , 2007 .

[20]  Habib N. Najm,et al.  Numerical Challenges in the Use of Polynomial Chaos Representations for Stochastic Processes , 2005, SIAM J. Sci. Comput..

[21]  James O. Berger,et al.  Markov chain Monte Carlo-based approaches for inference in computationally intensive inverse problems , 2003 .

[22]  Baskar Ganapathysubramanian,et al.  Sparse grid collocation schemes for stochastic natural convection problems , 2007, J. Comput. Phys..

[23]  K. Afanasiev,et al.  Adaptive Control Of A Wake Flow Using Proper Orthogonal Decomposition1 , 2001 .

[24]  N. Nguyen,et al.  An ‘empirical interpolation’ method: application to efficient reduced-basis discretization of partial differential equations , 2004 .

[25]  C. Fox,et al.  Markov chain Monte Carlo Using an Approximation , 2005 .

[26]  P. Holmes,et al.  Turbulence, Coherent Structures, Dynamical Systems and Symmetry , 1996 .

[27]  Jacob K. White,et al.  Low-Rank Solution of Lyapunov Equations , 2004, SIAM Rev..

[28]  Ahmed K. Noor,et al.  Reduced Basis Technique for Nonlinear Analysis of Structures , 1979 .

[29]  D. Rovas,et al.  A blackbox reduced-basis output bound method for noncoercive linear problems , 2002 .

[30]  Michael S. Eldred,et al.  Second-Order Corrections for Surrogate-Based Optimization with Model Hierarchies , 2004 .

[31]  R. Ghanem Probabilistic characterization of transport in heterogeneous media , 1998 .

[32]  Eric James Grimme,et al.  Krylov Projection Methods for Model Reduction , 1997 .

[33]  P. Beran,et al.  Uncertainty quantification of limit-cycle oscillations , 2006, J. Comput. Phys..

[34]  Yalchin Efendiev,et al.  Preconditioning Markov Chain Monte Carlo Simulations Using Coarse-Scale Models , 2006, SIAM J. Sci. Comput..

[35]  D. Xiu Fast numerical methods for stochastic computations: A review , 2009 .

[36]  Xiang Ma,et al.  An adaptive hierarchical sparse grid collocation algorithm for the solution of stochastic differential equations , 2009, J. Comput. Phys..

[37]  K. Kunisch,et al.  Control of the Burgers Equation by a Reduced-Order Approach Using Proper Orthogonal Decomposition , 1999 .

[38]  C. Crainiceanu,et al.  Fast Adaptive Penalized Splines , 2008 .

[39]  Fabio Nobile,et al.  A Stochastic Collocation Method for Elliptic Partial Differential Equations with Random Input Data , 2007, SIAM Rev..

[40]  R. Ghanem,et al.  Stochastic Finite Elements: A Spectral Approach , 1990 .

[41]  Wei Li,et al.  Efficient geostatistical inverse methods for structured and unstructured grids , 2006 .

[42]  H. Miura,et al.  An approximate analysis technique for design calculations , 1971 .

[43]  Kurt Maute,et al.  Application of Reduced Order Models for the Stochastic Design Optimization of Dynamic Systems , 2004 .

[44]  D. Rovas,et al.  A Posteriori Error Bounds for Reduced-Basis Approximation of Parametrized Noncoercive and Nonlinear Elliptic Partial Differential Equations , 2003 .

[45]  Stefan M. Wild,et al.  Bayesian Calibration and Uncertainty Analysis for Computationally Expensive Models Using Optimization and Radial Basis Function Approximation , 2008 .

[46]  Siep Weiland,et al.  Missing Point Estimation in Models Described by Proper Orthogonal Decomposition , 2004, IEEE Transactions on Automatic Control.

[47]  B. Moore Principal component analysis in linear systems: Controllability, observability, and model reduction , 1981 .

[48]  H. Tran,et al.  Modeling and control of physical processes using proper orthogonal decomposition , 2001 .

[49]  M. Grepl Reduced-basis approximation a posteriori error estimation for parabolic partial differential equations , 2005 .

[50]  J. Peraire,et al.  Balanced Model Reduction via the Proper Orthogonal Decomposition , 2002 .

[51]  D. Rovas,et al.  Reduced--Basis Output Bound Methods for Parametrized Partial Differential Equations , 2002 .

[52]  L. Sirovich Turbulence and the dynamics of coherent structures. II. Symmetries and transformations , 1987 .

[53]  Luca Daniel,et al.  Parameterized model order reduction of nonlinear dynamical systems , 2005, ICCAD-2005. IEEE/ACM International Conference on Computer-Aided Design, 2005..

[54]  Ngoc Cuong Nguyen,et al.  A posteriori error estimation and basis adaptivity for reduced-basis approximation of nonaffine-parametrized linear elliptic partial differential equations , 2007, J. Comput. Phys..

[55]  R. Ghanem,et al.  A stochastic projection method for fluid flow. I: basic formulation , 2001 .

[56]  Y. Marzouk,et al.  Uncertainty quantification in chemical systems , 2009 .

[57]  H. Najm,et al.  A stochastic projection method for fluid flow II.: random process , 2002 .

[58]  Jacob K. White,et al.  A multiparameter moment-matching model-reduction approach for generating geometrically parameterized interconnect performance models , 2004, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[59]  O. L. Maître,et al.  Protein labeling reactions in electrochemical microchannel flow: Numerical simulation and uncertainty propagation , 2003 .

[60]  Ahmed K. Noor,et al.  Reduced Basis Technique for Nonlinear Analysis of Structures , 1980 .

[61]  George Em Karniadakis,et al.  Error Control in Multi-Element Generalized Polynomial Chaos Methodfor EllipticProblemswith Random Coefficients , 2009 .

[62]  Dongbin Xiu,et al.  The Wiener-Askey Polynomial Chaos for Stochastic Differential Equations , 2002, SIAM J. Sci. Comput..

[63]  M. Hinze,et al.  Proper Orthogonal Decomposition Surrogate Models for Nonlinear Dynamical Systems: Error Estimates and Suboptimal Control , 2005 .

[64]  A. Patera,et al.  Certified real‐time solution of the parametrized steady incompressible Navier–Stokes equations: rigorous reduced‐basis a posteriori error bounds , 2005 .

[65]  J. Marsden,et al.  A subspace approach to balanced truncation for model reduction of nonlinear control systems , 2002 .

[66]  E. Grimme,et al.  Pade approximation of large-scale dynamic systems with Lanczos methods , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[67]  A. Patera,et al.  A posteriori error bounds for reduced-basis approximations of parametrized parabolic partial differential equations , 2005 .

[68]  Dongbin Xiu,et al.  High-Order Collocation Methods for Differential Equations with Random Inputs , 2005, SIAM J. Sci. Comput..

[69]  Charbel Farhat,et al.  Aeroelastic Analysis of F-16 and F-18/A Configurations Using Adapted CFD-Based Reduced-Order Models , 2007 .

[70]  Jie Shen,et al.  Efficient stochastic Galerkin methods for random diffusion equations , 2009, J. Comput. Phys..

[71]  Alexandre Megretski,et al.  A Quasi-Convex Optimization Approach to Parameterized Model Order Reduction , 2005, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[72]  Alison L Gibbs,et al.  On Choosing and Bounding Probability Metrics , 2002, math/0209021.

[73]  Fabio Nobile,et al.  A Stochastic Collocation Method for Elliptic Partial Differential Equations with Random Input Data , 2010, SIAM Rev..

[74]  Panos G. Georgopoulos,et al.  Uncertainty reduction and characterization for complex environmental fate and transport models: An empirical Bayesian framework incorporating the stochastic response surface method , 2003 .

[75]  J. Delville,et al.  Proper Orthogonal Decomposition , 2003 .

[76]  L. Sirovich Turbulence and the dynamics of coherent structures. I. Coherent structures , 1987 .

[77]  M. Gunzburger,et al.  Reduced-order modeling of time-dependent PDEs with multiple parameters in the boundary data , 2007 .

[78]  Habib N. Najm,et al.  Uncertainty Quantification and Polynomial Chaos Techniques in Computational Fluid Dynamics , 2009 .

[79]  O P Le Maître,et al.  Spectral stochastic uncertainty quantification in chemical systems , 2004 .

[80]  Karen Willcox,et al.  Model Reduction for Large-Scale Systems with High-Dimensional Parametric Input Space , 2008, SIAM J. Sci. Comput..

[81]  Marco Fahl,et al.  Reduced Order Modelling Approaches to PDE-Constrained Optimization Based on Proper Orthogonal Decomposition , 2003 .

[82]  G. Karniadakis,et al.  An adaptive multi-element generalized polynomial chaos method for stochastic differential equations , 2005 .

[83]  Hermann G. Matthies,et al.  Galerkin methods for linear and nonlinear elliptic stochastic partial differential equations , 2005 .

[84]  R. Ghanem,et al.  Multi-resolution analysis of wiener-type uncertainty propagation schemes , 2004 .

[85]  Roland W. Freund,et al.  Efficient linear circuit analysis by Pade´ approximation via the Lanczos process , 1994, EURO-DAC '94.

[86]  Nicholas Zabaras,et al.  Using Bayesian statistics in the estimation of heat source in radiation , 2005 .