Efficient approximation of Sparse Jacobians for time‐implicit reduced order models

This paper introduces a sparse matrix discrete interpolation method to effectively compute matrix approximations in the reduced order modeling framework. The sparse algorithm developed herein relies on the discrete empirical interpolation method and uses only samples of the nonzero entries of the matrix series. The proposed approach can approximate very large matrices, unlike the current matrix discrete empirical interpolation method which is limited by its large computational memory requirements. The empirical interpolation indexes obtained by the sparse algorithm slightly differ from the ones computed by the matrix discrete empirical interpolation method as a consequence of the singular vectors round-off errors introduced by the economy or full singular value decomposition (SVD) algorithms when applied to the full matrix snapshots. When appropriately padded with zeros the economy SVD factorization of the nonzero elements of the snapshots matrix is a valid economy SVD for the full snapshots matrix. Numerical experiments are performed with the 1D Burgers and 2D Shallow Water Equations test problems where the quadratic reduced nonlinearities are computed via tensorial calculus. The sparse matrix approximation strategy is compared against five existing methods for computing reduced Jacobians: a) matrix discrete empirical interpolation method, b) discrete empirical interpolation method, c) tensorial calculus, d) full Jacobian projection onto the reduced basis subspace, and e) directional derivatives of the model along the reduced basis functions. The sparse matrix method outperforms all other algorithms. The use of traditional matrix discrete empirical interpolation method is not possible for very large instances due to its excessive memory requirements.

[1]  Bernard Haasdonk,et al.  Certified PDE-constrained parameter optimization using reduced basis surrogate models for evolution problems , 2015, Comput. Optim. Appl..

[2]  Gilead Tadmor,et al.  System reduction strategy for Galerkin models of fluid flows , 2009 .

[3]  Sivasankaran Rajamanickam,et al.  EFFICIENT ALGORITHMS FOR SPARSE SINGULAR VALUE DECOMPOSITION , 2009 .

[4]  Karen Willcox,et al.  Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems , 2010, SIAM J. Sci. Comput..

[5]  Charbel Farhat,et al.  Nonlinear model order reduction based on local reduced‐order bases , 2012 .

[6]  D. Mingori,et al.  Approximate Subspace Iteration for constructing internally balanced reduced order models of unsteady aerodynamic systems , 1996 .

[7]  H. Hotelling Analysis of a complex of statistical variables into principal components. , 1933 .

[8]  Timo Tonn Reduced-basis method (RBM) for non-affine elliptic parametrized PDEs - (motivated by optimization in hydromechanics) , 2012 .

[10]  Mihai Anitescu,et al.  Proper orthogonal decompositions in multifidelity uncertainty quantification of complex simulation models , 2014, Int. J. Comput. Math..

[11]  Athanasios C. Antoulas,et al.  Approximation of Large-Scale Dynamical Systems , 2005, Advances in Design and Control.

[12]  A. Patera,et al.  Reduced basis approximation and a posteriori error estimation for affinely parametrized elliptic coercive partial differential equations , 2007 .

[13]  Razvan Stefanescu,et al.  Application of POD-DEIM Approach for Dimension Reduction of a Diffusive Predator-Prey System with Allee Effect , 2013, LSSC.

[14]  Siep Weiland,et al.  Missing Point Estimation in Models Described by Proper Orthogonal Decomposition , 2004, IEEE Transactions on Automatic Control.

[15]  Lei Xie,et al.  HJB-POD-Based Feedback Design for the Optimal Control of Evolution Problems , 2004, SIAM J. Appl. Dyn. Syst..

[16]  Jan Vierendeels,et al.  Implicit coupling of partitioned fluid-structure interaction problems with reduced order models , 2007 .

[17]  Clarence W. Rowley,et al.  Model Reduction for fluids, Using Balanced Proper Orthogonal Decomposition , 2005, Int. J. Bifurc. Chaos.

[18]  José M. Vega,et al.  Reduced order models based on local POD plus Galerkin projection , 2010, J. Comput. Phys..

[19]  Stefan Volkwein,et al.  Galerkin proper orthogonal decomposition methods for parabolic problems , 2001, Numerische Mathematik.

[20]  R. Murray,et al.  Model reduction for compressible flows using POD and Galerkin projection , 2004 .

[21]  R. Freund Model reduction methods based on Krylov subspaces , 2003, Acta Numerica.

[22]  Paul Van Dooren,et al.  The Lanczos algorithm and Pad(cid:19)e approximations , 1995 .

[23]  L. Sirovich Turbulence and the dynamics of coherent structures. II. Symmetries and transformations , 1987 .

[24]  Arlindo L. Oliveira,et al.  Biclustering algorithms for biological data analysis: a survey , 2004, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[25]  C. Kelley Iterative Methods for Linear and Nonlinear Equations , 1987 .

[26]  A. Laub,et al.  Approximate solution of large sparse Lyapunov equations , 1994, IEEE Trans. Autom. Control..

[27]  W. Gragg,et al.  On the partial realization problem , 1983 .

[28]  J. Peraire,et al.  Balanced Model Reduction via the Proper Orthogonal Decomposition , 2002 .

[29]  W. Gragg,et al.  The Padé Table and Its Relation to Certain Algorithms of Numerical Analysis , 1972 .

[30]  Bertil Gustafsson,et al.  An alternating direction implicit method for solving the shallow water equations , 1971 .

[31]  Zhu Wang,et al.  Are the Snapshot Difference Quotients Needed in the Proper Orthogonal Decomposition? , 2013, SIAM J. Sci. Comput..

[32]  E. Grimme,et al.  Pade approximation of large-scale dynamic systems with Lanczos methods , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[33]  Peter Benner,et al.  Two-Sided Projection Methods for Nonlinear Model Order Reduction , 2015, SIAM J. Sci. Comput..

[34]  Yuxiang Beckett Zhou,et al.  Model reduction for nonlinear dynamical systems with parametric uncertainties , 2012 .

[35]  Paul T. Boggs,et al.  Efficient structure-preserving model reduction for nonlinear mechanical systems with application to structural dynamics. , 2012 .

[36]  A. Grammeltvedt A SURVEY OF FINITE-DIFFERENCE SCHEMES FOR THE PRIMITIVE EQUATIONS FOR A BAROTROPIC FLUID , 1969 .

[37]  Kevin Carlberg,et al.  Decreasing the temporal complexity for nonlinear, implicit reduced-order models by forecasting , 2012, 1209.5455.

[38]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .

[39]  Adrian Sandu,et al.  A time-parallel approach to strong-constraint four-dimensional variational data assimilation , 2015, J. Comput. Phys..

[40]  K. Karhunen Zur Spektraltheorie stochastischer prozesse , 1946 .

[41]  Danny C. Sorensen,et al.  Nonlinear Model Reduction via Discrete Empirical Interpolation , 2010, SIAM J. Sci. Comput..

[42]  Kevin Carlberg,et al.  The ROMES method for statistical modeling of reduced-order-model error , 2014, SIAM/ASA J. Uncertain. Quantification.

[43]  P. Schmid,et al.  Dynamic mode decomposition of numerical and experimental data , 2008, Journal of Fluid Mechanics.

[44]  A. Patera,et al.  A posteriori error bounds for reduced-basis approximations of parametrized parabolic partial differential equations , 2005 .

[45]  Bernd R. Noack,et al.  Model reduction using Dynamic Mode Decomposition , 2014 .

[46]  Saifon Chaturantabut Dimension reduction for unsteady nonlinear partial differential equations via empirical interpolation methods , 2009 .

[47]  Eric James Grimme,et al.  Krylov Projection Methods for Model Reduction , 1997 .

[48]  R. Dedden Model Order Reduction using the Discrete Empirical Interpolation Method , 2012 .

[49]  A. Hodel,et al.  Least squares approximate solution of the Lyapunov equation , 1991, [1991] Proceedings of the 30th IEEE Conference on Decision and Control.

[50]  C. Farhat,et al.  Efficient non‐linear model reduction via a least‐squares Petrov–Galerkin projection and compressive tensor approximations , 2011 .

[51]  Youcef Saad,et al.  A Basic Tool Kit for Sparse Matrix Computations , 1990 .

[52]  Ionel Michael Navon,et al.  GUSTAF: a quasi-Newton nonlinear ADI FORTRAN IV program for solving the shallow-water equations with augmented Lagrangians , 1984 .

[53]  Philipp Birken,et al.  Numerical Linear Algebra , 2011, Encyclopedia of Parallel Computing.

[54]  Timothy A. Davis,et al.  Algorithm 836: COLAMD, a column approximate minimum degree ordering algorithm , 2004, TOMS.

[55]  Zlatko Drmac,et al.  A New Selection Operator for the Discrete Empirical Interpolation Method - Improved A Priori Error Bound and Extensions , 2015, SIAM J. Sci. Comput..

[56]  Juan Du,et al.  Non-linear model reduction for the Navier-Stokes equations using residual DEIM method , 2014, J. Comput. Phys..

[57]  Zhi Wang,et al.  A truncated Newton optimization algorithm in meteorology applications with analytic Hessian/vector products , 1995, Comput. Optim. Appl..

[58]  L. Sirovich Turbulence and the dynamics of coherent structures. I. Coherent structures , 1987 .

[59]  Stefan Volkwein,et al.  Galerkin Proper Orthogonal Decomposition Methods for a General Equation in Fluid Dynamics , 2002, SIAM J. Numer. Anal..

[60]  I. Jaimoukha,et al.  Krylov subspace methods for solving large Lyapunov equations , 1994 .

[61]  Adrian Sandu,et al.  A Posteriori Error Estimates for DDDAS Inference Problems , 2014, ICCS.

[62]  Benjamin Peherstorfer,et al.  Localized Discrete Empirical Interpolation Method , 2014, SIAM J. Sci. Comput..

[63]  D. Keyes,et al.  Jacobian-free Newton-Krylov methods: a survey of approaches and applications , 2004 .

[64]  Adrian Sandu,et al.  Comparison of POD reduced order strategies for the nonlinear 2D shallow water equations , 2014, International Journal for Numerical Methods in Fluids.

[65]  Lawrence Sirovich,et al.  Karhunen–Loève procedure for gappy data , 1995 .

[66]  Danny C. Sorensen,et al.  The Sylvester equation and approximate balanced reduction , 2002 .

[67]  Michael Hinze,et al.  Discrete Empirical Interpolation in POD Model Order Reduction of Drift-Diffusion Equations in Electrical Networks , 2012 .

[68]  Roland W. Freund,et al.  Efficient linear circuit analysis by Pade´ approximation via the Lanczos process , 1994, EURO-DAC '94.

[69]  N. Nguyen,et al.  An ‘empirical interpolation’ method: application to efficient reduced-basis discretization of partial differential equations , 2004 .

[70]  D. A. Bistrian,et al.  Comparison of optimized Dynamic Mode Decomposition vs POD for the shallow water equations model reduction with large-timestep observations , 2014 .

[71]  I. Mezić,et al.  Spectral analysis of nonlinear flows , 2009, Journal of Fluid Mechanics.

[72]  K. Willcox,et al.  Aerodynamic Data Reconstruction and Inverse Design Using Proper Orthogonal Decomposition , 2004 .

[73]  Bernd R. Noack,et al.  Cluster-based reduced-order modelling of a mixing layer , 2013, Journal of Fluid Mechanics.

[74]  Gianluigi Rozza,et al.  Reduced Basis Method for Parametrized Elliptic Optimal Control Problems , 2013, SIAM J. Sci. Comput..

[75]  Adhemar Bultheel,et al.  Rational approximation in linear systems and control , 2000 .

[76]  Timothy A. Davis,et al.  A column approximate minimum degree ordering algorithm , 2000, TOMS.

[77]  K. Kunisch,et al.  Control of the Burgers Equation by a Reduced-Order Approach Using Proper Orthogonal Decomposition , 1999 .

[78]  J. MacQueen Some methods for classification and analysis of multivariate observations , 1967 .

[79]  Gene H. Golub,et al.  Singular value decomposition and least squares solutions , 1970, Milestones in Matrix Computation.

[80]  B. Moore Principal component analysis in linear systems: Controllability, observability, and model reduction , 1981 .

[81]  L. Sirovich Turbulence and the dynamics of coherent structures. III. Dynamics and scaling , 1987 .

[82]  Razvan Stefanescu,et al.  POD/DEIM nonlinear model order reduction of an ADI implicit shallow water equations model , 2012, J. Comput. Phys..

[83]  Adrian Sandu,et al.  POD/DEIM reduced-order strategies for efficient four dimensional variational data assimilation , 2014, J. Comput. Phys..

[84]  Peter Benner,et al.  Partial realization of descriptor systems , 2006, Syst. Control. Lett..

[85]  Youcef Saad,et al.  A Basic Tool Kit for Sparse Matrix Computations , 1990 .

[86]  Arnold W. Heemink,et al.  Model-Reduced Variational Data Assimilation , 2006 .

[87]  D. Wirtz Model reduction for nonlinear systems : kernel methods and error estimation (Modellreduktion für nichtlineare Systeme : Kernmethoden und Fehlerschätzung) , 2014 .

[88]  William W. Hager,et al.  Minimizing the Profile of a Symmetric Matrix , 2001, SIAM J. Sci. Comput..

[89]  Graeme Fairweather,et al.  A Linear ADI Method for the Shallow-Water Equations , 1980 .

[90]  D. Rixen,et al.  Discrete Empirical Interpolation Method for Finite Element Structural Dynamics , 2013 .

[91]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[92]  Clifford T. Mullis,et al.  Synthesis of minimum roundoff noise fixed point digital filters , 1976 .

[93]  Danny C. Sorensen,et al.  A Posteriori Error Estimation for DEIM Reduced Nonlinear Dynamical Systems , 2014, SIAM J. Sci. Comput..

[94]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[95]  Traian Iliescu,et al.  Proper orthogonal decomposition closure models for fluid flows: Burgers equation , 2013, 1308.3276.

[96]  Richard Barrett,et al.  Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods , 1994, Other Titles in Applied Mathematics.

[97]  Harbir Antil,et al.  Application of the Discrete Empirical Interpolation Method to Reduced Order Modeling of Nonlinear and Parametric Systems , 2014 .

[98]  Danny C. Sorensen,et al.  A State Space Error Estimate for POD-DEIM Nonlinear Model Reduction , 2012, SIAM J. Numer. Anal..

[99]  J. Hesthaven,et al.  Reduced Basis Approximation and A Posteriori Error Estimation for Parametrized Partial Differential Equations , 2007 .

[100]  J. Peraire,et al.  A ‘best points’ interpolation method for efficient approximation of parametrized functions , 2008 .

[101]  B. R. Noack,et al.  A hierarchy of low-dimensional models for the transient and post-transient cylinder wake , 2003, Journal of Fluid Mechanics.

[102]  L. Sirovich TURBULENCE AND THE DYNAMICS OF COHERENT STRUCTURES PART I : COHERENT STRUCTURES , 2016 .