Multi‐fidelity data fusion for the approximation of scalar functions with low intrinsic dimensionality through active subspaces

Gaussian processes are employed for non-parametric regression in a Bayesian setting. They generalize linear regression, embedding the inputs in a latent manifold inside an infinite-dimensional reproducing kernel Hilbert space. We can augment the inputs with the observations of low-fidelity models in order to learn a more expressive latent manifold and thus increment the model's accuracy. This can be realized recursively with a chain of Gaussian processes with incrementally higher fidelity. We would like to extend these multi-fidelity model realizations to case studies affected by a high-dimensional input space but with low intrinsic dimensionality. In this cases physical supported or purely numerical low-order models are still affected by the curse of dimensionality when queried for responses. When the model's gradient information is provided, the presence of an active subspace can be exploited to design low-fidelity response surfaces and thus enable Gaussian process multi-fidelity regression, without the need to perform new simulations. This is particularly useful in the case of data scarcity. In this work we present a multi-fidelity approach involving active subspaces and we test it on two different high-dimensional benchmarks.

[1]  G. Rozza,et al.  An integrated data-driven computational pipeline with model order reduction for industrial and applied mathematics , 2018, 1810.12364.

[2]  George Em Karniadakis,et al.  Improving SWATH Seakeeping Performance using Multi-Fidelity Gaussian Process and Bayesian Optimization , 2018, Journal of Ship Research.

[3]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[4]  Alexander I. J. Forrester,et al.  Multi-fidelity optimization via surrogate modelling , 2007, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[5]  Gianluigi Rozza,et al.  Kernel‐based active subspaces with application to computational fluid dynamics parametric problems using the discontinuous Galerkin method , 2020, International journal for numerical methods in engineering.

[6]  Dino Sejdinovic,et al.  Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences , 2018, ArXiv.

[7]  Youssef Marzouk,et al.  Gradient-Based Dimension Reduction of Multivariate Vector-Valued Functions , 2020, SIAM J. Sci. Comput..

[8]  Loic Le Gratiet,et al.  RECURSIVE CO-KRIGING MODEL FOR DESIGN OF COMPUTER EXPERIMENTS WITH MULTIPLE LEVELS OF FIDELITY , 2012, 1210.0686.

[9]  George Em Karniadakis,et al.  Multi-fidelity optimization of super-cavitating hydrofoils , 2018 .

[10]  Gianluigi Rozza,et al.  A non-intrusive approach for the reconstruction of POD modal coefficients through active subspaces , 2019 .

[11]  David M. Steinberg,et al.  Modeling Data from Computer Experiments: An Empirical Comparison of Kriging with MARS and Projection Pursuit Regression , 2007 .

[12]  Gianluigi Rozza,et al.  A supervised learning approach involving active subspaces for an efficient genetic algorithm in high-dimensional optimization problems , 2020, ArXiv.

[13]  J. Weston,et al.  Approximation Methods for Gaussian Process Regression , 2007 .

[14]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[15]  Haitao Liu,et al.  When Gaussian Process Meets Big Data: A Review of Scalable GPs , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[16]  G. Rozza,et al.  Combined Parameter and Model Reduction of Cardiovascular Problems by Means of Active Subspaces and POD-Galerkin Methods , 2017, 1711.10884.

[17]  Paul G. Constantine,et al.  Active subspaces for sensitivity analysis and dimension reduction of an integrated hydrologic model , 2015, Comput. Geosci..

[18]  Paul G. Constantine,et al.  Active Subspaces - Emerging Ideas for Dimension Reduction in Parameter Studies , 2015, SIAM spotlights.

[19]  Paul G. Constantine,et al.  A modified SEIR model for the spread of Ebola in Western Africa and metrics for resource allocation , 2016, Appl. Math. Comput..

[20]  Juan J. Alonso,et al.  Active Subspaces for Shape Optimization , 2014 .

[21]  Gianluigi Rozza,et al.  Advances in geometrical parametrization and reduced order models and methods for computational fluid dynamics problems in applied sciences and engineering: overview and perspectives , 2016 .

[22]  Martin W. Hess,et al.  Basic Ideas and Tools for Projection-Based Model Reduction of Parametric Partial Differential Equations , 2019, Snapshot-Based Methods and Algorithms.

[23]  Carl E. Rasmussen,et al.  Warped Gaussian Processes , 2003, NIPS.

[24]  George Em Karniadakis,et al.  A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems , 2019, J. Comput. Phys..

[25]  Gianluigi Rozza,et al.  An efficient shape parametrisation by free-form deformation enhanced by active subspace for hull hydrodynamic ship design problems in open source environment , 2018, 1801.06369.

[26]  Paris Perdikaris,et al.  Inferring solutions of differential equations using noisy multi-fidelity data , 2016, J. Comput. Phys..

[27]  Gianluigi Rozza,et al.  Advances in reduced order methods for parametric industrial problems in computational fluid dynamics , 2018, 1811.08319.

[28]  Gianluigi Rozza,et al.  Model Order Reduction by means of Active Subspaces and Dynamic Mode Decomposition for Parametric Hull Shape Design Hydrodynamics , 2018, 1803.07377.

[29]  Gianluigi Rozza,et al.  Dimension reduction in heterogeneous parametric spaces with application to naval engineering shape design problems , 2017, Advanced Modeling and Simulation in Engineering Sciences.

[30]  Paul G. Constantine,et al.  Global sensitivity metrics from active subspaces , 2015, Reliab. Eng. Syst. Saf..

[31]  A. O'Hagan,et al.  Predicting the output from a complex computer code when fast approximations are available , 2000 .

[32]  Karen Willcox,et al.  Multifidelity Dimension Reduction via Active Subspaces , 2018, SIAM J. Sci. Comput..

[33]  Mark Pullin,et al.  Emulation of physical processes with Emukit , 2021, ArXiv.

[34]  Benjamin Peherstorfer,et al.  Multifidelity probability estimation via fusion of estimators , 2019, J. Comput. Phys..

[35]  Gianluigi Rozza,et al.  Shape optimization through proper orthogonal decomposition with interpolation and dynamic mode decomposition enhanced by active subspaces , 2019, 1905.05483.

[36]  Andreas C. Damianou,et al.  Nonlinear information fusion algorithms for data-efficient multi-fidelity modelling , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[37]  Carl E. Rasmussen,et al.  Sparse Spectrum Gaussian Process Regression , 2010, J. Mach. Learn. Res..

[38]  Gianluigi Rozza,et al.  Enhancing CFD predictions in shape design problems by model and parameter space reduction , 2020, Adv. Model. Simul. Eng. Sci..

[39]  Gianluigi Rozza,et al.  Efficient Reduction in Shape Parameter Space Dimension for Ship Propeller Blade Design , 2019, ArXiv.

[40]  Neil D. Lawrence,et al.  Deep Gaussian Processes , 2012, AISTATS.

[41]  Miguel Lázaro-Gredilla,et al.  Variational Heteroscedastic Gaussian Process Regression , 2011, ICML.