Multi-fidelity data fusion through parameter space reduction with applications to automotive engineering

Multi-fidelity models are of great importance due to their capability of fusing information coming from different simulations and sensors. Gaussian processes are employed for nonparametric regression in a Bayesian setting. They generalize linear regression embedding the inputs in a latent manifold inside an infinite-dimensional reproducing kernel Hilbert space. We can augment the inputs with the observations of low-fidelity models in order to learn a more expressive latent manifold and thus increment the model’s accuracy. This can be realized recursively with a chain of Gaussian processes with incrementally higher fidelity. We would like to extend these multi-fidelity model realizations to case studies affected by a high-dimensional input space but with low intrinsic dimensionality. In these cases physical supported or purely numerical low-order models are still affected by the curse of dimensionality when queried for responses. When the model’s gradient information is provided, the existence of an active subspace, or a nonlinear transformation of the input parameter space, can be exploited to design low-fidelity response surfaces and thus enable Gaussian process multi-fidelity regression, without the need to perform new simulations. This is particularly useful in the case of data scarcity. In this work we present a new multi-fidelity approach involving active subspaces and nonlinear level-set learning method. We test the proposed numerical method on two different high-dimensional benchmarks, and on a more complex car aerodynamics problem. We show how a low intrinsic dimensionality bias can increase the accuracy of GP response surfaces.

[1]  Paul G. Constantine,et al.  A modified SEIR model for the spread of Ebola in Western Africa and metrics for resource allocation , 2016, Appl. Math. Comput..

[2]  Andreas C. Damianou,et al.  Nonlinear information fusion algorithms for data-efficient multi-fidelity modelling , 2017, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[3]  Paul G. Constantine,et al.  Active subspaces for sensitivity analysis and dimension reduction of an integrated hydrologic model , 2015, Comput. Geosci..

[4]  Martin W. Hess,et al.  Basic Ideas and Tools for Projection-Based Model Reduction of Parametric Partial Differential Equations , 2019, Snapshot-Based Methods and Algorithms.

[5]  J. Weston,et al.  Approximation Methods for Gaussian Process Regression , 2007 .

[6]  Gianluigi Rozza,et al.  A supervised learning approach involving active subspaces for an efficient genetic algorithm in high-dimensional optimization problems , 2020, SIAM J. Sci. Comput..

[7]  George Em Karniadakis,et al.  Improving SWATH Seakeeping Performance using Multi-Fidelity Gaussian Process and Bayesian Optimization , 2018, Journal of Ship Research.

[8]  Juan J. Alonso,et al.  On Active Subspaces in Car Aerodynamics , 2016 .

[9]  Miguel Lázaro-Gredilla,et al.  Variational Heteroscedastic Gaussian Process Regression , 2011, ICML.

[10]  George Em Karniadakis,et al.  Multi-fidelity optimization of super-cavitating hydrofoils , 2018 .

[11]  Gianluigi Rozza,et al.  Multi‐fidelity data fusion for the approximation of scalar functions with low intrinsic dimensionality through active subspaces , 2020, PAMM.

[12]  Gianluigi Rozza,et al.  ATHENA: Advanced Techniques for High Dimensional Parameter Spaces to Enhance Numerical Analysis , 2021, Softw. Impacts.

[13]  G. Rozza,et al.  Combined Parameter and Model Reduction of Cardiovascular Problems by Means of Active Subspaces and POD-Galerkin Methods , 2017, 1711.10884.

[14]  Paris Perdikaris,et al.  Inferring solutions of differential equations using noisy multi-fidelity data , 2016, J. Comput. Phys..

[15]  Loic Le Gratiet,et al.  RECURSIVE CO-KRIGING MODEL FOR DESIGN OF COMPUTER EXPERIMENTS WITH MULTIPLE LEVELS OF FIDELITY , 2012, 1210.0686.

[16]  Gianluigi Rozza,et al.  Advances in reduced order methods for parametric industrial problems in computational fluid dynamics , 2018, 1811.08319.

[17]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[18]  Carl E. Rasmussen,et al.  Warped Gaussian Processes , 2003, NIPS.

[19]  C. Othmer,et al.  Reduced-Order Modeling of Vehicle Aerodynamics via Proper Orthogonal Decomposition , 2019, SAE International Journal of Passenger Cars - Mechanical Systems.

[20]  Neil D. Lawrence,et al.  Deep Gaussian Processes , 2012, AISTATS.

[21]  Carl E. Rasmussen,et al.  Sparse Spectrum Gaussian Process Regression , 2010, J. Mach. Learn. Res..

[22]  Gianluigi Rozza,et al.  A non-intrusive approach for the reconstruction of POD modal coefficients through active subspaces , 2019 .

[23]  David M. Steinberg,et al.  Modeling Data from Computer Experiments: An Empirical Comparison of Kriging with MARS and Projection Pursuit Regression , 2007 .

[24]  Paul G. Constantine,et al.  Global sensitivity metrics from active subspaces , 2015, Reliab. Eng. Syst. Saf..

[25]  Youssef Marzouk,et al.  Gradient-Based Dimension Reduction of Multivariate Vector-Valued Functions , 2020, SIAM J. Sci. Comput..

[26]  Gianluigi Rozza,et al.  Hull shape design optimization with parameter space and model reductions, and self-learning mesh morphing , 2021, Journal of Marine Science and Engineering.

[27]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[28]  Haitao Liu,et al.  When Gaussian Process Meets Big Data: A Review of Scalable GPs , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Hrvoje Jasak,et al.  A tensorial approach to computational continuum mechanics using object-oriented techniques , 1998 .

[30]  Alexander I. J. Forrester,et al.  Multi-fidelity optimization via surrogate modelling , 2007, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[31]  Robert A. Bridges,et al.  Active Manifolds: A non-linear analogue to Active Subspaces , 2019, ICML.

[32]  Samuel Friedman,et al.  Adaptive Dimensionality Reduction for Fast Sequential Optimization With Gaussian Processes , 2019, Journal of Mechanical Design.

[33]  Gianluigi Rozza,et al.  Efficient Reduction in Shape Parameter Space Dimension for Ship Propeller Blade Design , 2019, ArXiv.

[34]  A. O'Hagan,et al.  Predicting the output from a complex computer code when fast approximations are available , 2000 .

[35]  Gianluigi Rozza,et al.  A local approach to parameter space reduction for regression and classification tasks , 2021, ArXiv.

[36]  George Em Karniadakis,et al.  A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems , 2019, J. Comput. Phys..

[37]  Eldad Haber,et al.  Reversible Architectures for Arbitrarily Deep Residual Neural Networks , 2017, AAAI.

[38]  Juan J. Alonso,et al.  Active Subspaces for Shape Optimization , 2014 .

[39]  Gianluigi Rozza,et al.  Dimension reduction in heterogeneous parametric spaces with application to naval engineering shape design problems , 2017, Advanced Modeling and Simulation in Engineering Sciences.

[40]  Lili Ju,et al.  Nonlinear Level Set Learning for Function Approximation on Sparse Data with Applications to Parametric Differential Equations , 2021, Numerical Mathematics: Theory, Methods and Applications.

[41]  Paul G. Constantine,et al.  Active Subspaces - Emerging Ideas for Dimension Reduction in Parameter Studies , 2015, SIAM spotlights.

[42]  Mark Pullin,et al.  Emulation of physical processes with Emukit , 2021, ArXiv.

[43]  Gianluigi Rozza,et al.  Kernel‐based active subspaces with application to computational fluid dynamics parametric problems using the discontinuous Galerkin method , 2020, International journal for numerical methods in engineering.

[44]  Benjamin Peherstorfer,et al.  Multifidelity probability estimation via fusion of estimators , 2019, J. Comput. Phys..

[45]  G. Rozza,et al.  An integrated data-driven computational pipeline with model order reduction for industrial and applied mathematics , 2018, 1810.12364.

[46]  Paris Perdikaris,et al.  Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations , 2019, J. Comput. Phys..

[47]  Gianluigi Rozza,et al.  Advances in geometrical parametrization and reduced order models and methods for computational fluid dynamics problems in applied sciences and engineering: overview and perspectives , 2016 .

[48]  Jiaxin Zhang,et al.  Learning nonlinear level sets for dimensionality reduction in function approximation , 2019, NeurIPS.

[49]  Dino Sejdinovic,et al.  Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences , 2018, ArXiv.

[50]  Karen Willcox,et al.  Multifidelity Dimension Reduction via Active Subspaces , 2018, SIAM J. Sci. Comput..

[51]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[52]  Eldad Haber,et al.  Stable architectures for deep neural networks , 2017, ArXiv.