Multifidelity Dimension Reduction via Active Subspaces

We propose a multifidelity dimension reduction method to identify a low-dimensional structure present in many engineering models. The structure of interest arises when functions vary primarily on a low-dimensional subspace of the high-dimensional input space, while varying little along the complementary directions. Our approach builds on the gradient-based methodology of active subspaces, and exploits models of different fidelities to reduce the cost of performing dimension reduction through the computation of the active subspace matrix. We provide a non-asymptotic analysis of the number of gradient evaluations sufficient to achieve a prescribed error in the active subspace matrix, both in expectation and with high probability. We show that the sample complexity depends on a notion of intrinsic dimension of the problem, which can be much smaller than the dimension of the input space. We illustrate the benefits of such a multifidelity dimension reduction approach using numerical experiments with input spaces of up to three thousand dimensions.

[1]  Michael B. Giles,et al.  Multilevel Monte Carlo Path Simulation , 2008, Oper. Res..

[2]  Tiangang Cui,et al.  Likelihood-informed dimension reduction for nonlinear inverse problems , 2014, 1403.4680.

[3]  Karen Willcox,et al.  Parameter and State Model Reduction for Large-Scale Statistical Inverse Problems , 2010, SIAM J. Sci. Comput..

[4]  Paul G. Constantine,et al.  Active Subspaces - Emerging Ideas for Dimension Reduction in Parameter Studies , 2015, SIAM spotlights.

[5]  Karen Willcox,et al.  Provably Convergent Multifidelity Optimization Algorithm Not Requiring High-Fidelity Derivatives , 2012 .

[6]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.

[7]  Vladimir Koltchinskii,et al.  Asymptotics and Concentration Bounds for Bilinear Forms of Spectral Projectors of Sample Covariance , 2014, 1408.4643.

[8]  Remi Roger Alain Paul Lam,et al.  Scaling Bayesian optimization for engineering design : lookahead approaches and multifidelity dimension reduction , 2018 .

[9]  Luca Heltai,et al.  Isogeometric analysis-based reduced order modelling for incompressible linear viscous flows in parametrized shapes , 2016, Advanced Modeling and Simulation in Engineering Sciences.

[10]  Roger G. Ghanem,et al.  Basis adaptation in homogeneous chaos spaces , 2014, J. Comput. Phys..

[11]  Ker-Chau Li,et al.  Sliced Inverse Regression for Dimension Reduction , 1991 .

[12]  William L. Briggs,et al.  A multigrid tutorial , 1987 .

[13]  Saltelli Andrea,et al.  Global Sensitivity Analysis: The Primer , 2008 .

[14]  Karen Willcox,et al.  Multifidelity Optimization using Statistical Surrogate Modeling for Non-Hierarchical Information Sources , 2015 .

[15]  Benjamin Peherstorfer,et al.  Optimal Model Management for Multifidelity Monte Carlo Estimation , 2016, SIAM J. Sci. Comput..

[16]  Wolfgang Hörmann,et al.  Quasi Importance Sampling , 2005 .

[17]  Jing Li,et al.  An efficient surrogate-based method for computing rare failure probability , 2011, J. Comput. Phys..

[18]  Ilse C. F. Ipsen,et al.  A Probabilistic Subspace Bound with Application to Active Subspaces , 2018, SIAM J. Matrix Anal. Appl..

[19]  Max Gunzburger,et al.  A Multilevel Stochastic Collocation Method for Partial Differential Equations with Random Input Data , 2014, SIAM/ASA J. Uncertain. Quantification.

[20]  A. Samarov.  Exploring Regression Structure Using Nonparametric Functional Estimation , 1993 .

[21]  Kirthevasan Kandasamy,et al.  Multi-fidelity Bayesian Optimisation with Continuous Approximations , 2017, ICML.

[22]  P. A. Newman,et al.  Approximation and Model Management in Aerodynamic Optimization with Variable-Fidelity Models , 2001 .

[23]  Tiangang Cui,et al.  Certified dimension reduction in nonlinear Bayesian inverse problems , 2018, Math. Comput..

[24]  Paul G. Constantine,et al.  Time‐dependent global sensitivity analysis with active subspaces for a lithium ion battery model , 2016, Stat. Anal. Data Min..

[25]  Matthias Poloczek,et al.  Multi-Information Source Optimization , 2016, NIPS.

[26]  V. Cevher,et al.  Learning Non-Parametric Basis Independent Models from Point Queries via Low-Rank Methods , 2013, 1310.1826.

[27]  Joel A. Tropp,et al.  An Introduction to Matrix Concentration Inequalities , 2015, Found. Trends Mach. Learn..

[28]  G. Rozza,et al.  Combined Parameter and Model Reduction of Cardiovascular Problems by Means of Active Subspaces and POD-Galerkin Methods , 2017, 1711.10884.

[29]  Jing Li,et al.  Evaluation of failure probability via surrogate models , 2010, J. Comput. Phys..

[30]  Wolfgang Hackbusch,et al.  Multi-grid methods and applications , 1985, Springer series in computational mathematics.

[31]  Jasper Snoek,et al.  Multi-Task Bayesian Optimization , 2013, NIPS.

[32]  Frances Y. Kuo,et al.  Multilevel Quasi-Monte Carlo methods for lognormal diffusion problems , 2015, Math. Comput..

[33]  Paul G. Constantine,et al.  Exploring the Sensitivity of Photosynthesis and Stomatal Resistance Parameters in a Land Surface Model , 2017 .

[34]  Assad A. Oberai,et al.  INVERSE PROBLEMS PII: S0266-5611(03)54272-1 Solution of inverse problems in elasticity imaging using the adjoint method , 2003 .

[35]  D. Brandt,et al.  Multi-level adaptive solutions to boundary-value problems math comptr , 1977 .

[36]  Simona Perotto,et al.  A POD‐selective inverse distance weighting method for fast parametrized shape morphing , 2017, International Journal for Numerical Methods in Engineering.

[37]  Juan J. Alonso,et al.  Active Subspaces for Shape Optimization , 2014 .

[38]  Mario Ohlberger,et al.  Data-driven combined state and parameter reduction for inverse problems , 2014, Advances in Computational Mathematics.

[39]  Youssef Marzouk,et al.  Gradient-Based Dimension Reduction of Multivariate Vector-Valued Functions , 2020, SIAM J. Sci. Comput..

[40]  Benjamin Peherstorfer,et al.  Survey of multifidelity methods in uncertainty propagation, inference, and optimization , 2018, SIAM Rev..

[41]  Trent Michael Russi,et al.  Uncertainty Quantification with Experimental Data and Complex System Models , 2010 .

[42]  Zhuyin Ren,et al.  Shared low-dimensional subspaces for propagating kinetic uncertainty to multiple outputs , 2018 .

[43]  Tiangang Cui,et al.  Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction , 2015, J. Comput. Phys..

[44]  D. Gleich,et al.  Computing active subspaces with Monte Carlo , 2014, 1408.0545.

[45]  Qiqi Wang,et al.  Erratum: Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces , 2013, SIAM J. Sci. Comput..

[46]  Yan Zhou,et al.  Multilevel Sequential Monte Carlo with Dimension-Independent Likelihood-Informed Proposals , 2017, SIAM/ASA J. Uncertain. Quantification.

[47]  Paul G. Constantine,et al.  Accelerating Markov Chain Monte Carlo with Active Subspaces , 2016, SIAM J. Sci. Comput..

[48]  Iason Papaioannou,et al.  Multilevel Estimation of Rare Events , 2015, SIAM/ASA J. Uncertain. Quantification.

[49]  Tiangang Cui,et al.  Multifidelity importance sampling , 2016 .

[50]  Benjamin Peherstorfer,et al.  Combining multiple surrogate models to accelerate failure probability estimation with expensive high-fidelity models , 2017, J. Comput. Phys..

[51]  Alexander I. J. Forrester,et al.  Multi-fidelity optimization via surrogate modelling , 2007, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.