Gradient-Based Dimension Reduction of Multivariate Vector-Valued Functions

We propose a gradient-based method for detecting and exploiting low-dimensional input parameter dependence of multivariate functions. The methodology consists in minimizing an upper bound, obtained by Poincar\'e-type inequalities, on the approximation error. The resulting method can be used to approximate vector-valued functions (e.g., functions taking values in $\mathbb{R}^n$ or functions taking values in function spaces) and generalizes the notion of active subspaces associated with scalar-valued functions. A comparison with the truncated Karhunen-Lo\`eve decomposition shows that using gradients of the function can yield more effective dimension reduction. Numerical examples reveal that the choice of norm on the codomain of the function can have a significant impact on the function's low-dimensional approximation.

[1]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[2]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[3]  O. Kallenberg Foundations of Modern Probability , 2021, Probability Theory and Stochastic Modelling.

[4]  W. Marsden I and J , 2012 .

[5]  Bing Li,et al.  Groupwise Dimension Reduction , 2010 .

[6]  J. Guermond,et al.  Theory and practice of finite elements , 2004 .

[7]  Kerby Shedden,et al.  Dimension Reduction for Multivariate Response Data , 2003 .

[8]  Paul G. Constantine,et al.  Exploring the Sensitivity of Photosynthesis and Stomatal Resistance Parameters in a Land Surface Model , 2017 .

[9]  Robert Scheichl,et al.  Finite Element Error Analysis of Elliptic PDEs with Random Coefficients and Its Application to Multilevel Monte Carlo Methods , 2013, SIAM J. Numer. Anal..

[10]  Lixing Zhu,et al.  ON DIMENSION REDUCTION IN REGRESSIONS WITH MULTIVARIATE RESPONSES , 2010 .

[11]  Paul G. Constantine,et al.  Global sensitivity metrics from active subspaces , 2015, Reliab. Eng. Syst. Saf..

[12]  Paul G. Constantine,et al.  Active Subspaces - Emerging Ideas for Dimension Reduction in Parameter Studies , 2015, SIAM spotlights.

[13]  Stefano Tarantola,et al.  Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models , 2004 .

[14]  P. Frauenfelder,et al.  Finite elements for elliptic problems with stochastic coefficients , 2005 .

[15]  Marco Ratto,et al.  Global Sensitivity Analysis , 2008 .

[16]  Habib N. Najm,et al.  Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems , 2008, J. Comput. Phys..

[17]  Ilya M. Sobol,et al.  Sensitivity Estimates for Nonlinear Mathematical Models , 1993 .

[18]  Constantinos C. Pantelides,et al.  Monte Carlo evaluation of derivative-based global sensitivity measures , 2009, Reliab. Eng. Syst. Saf..

[19]  Gesine Reinert,et al.  Second order Poincaré inequalities and CLTs on Wiener space , 2008 .

[20]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[21]  Jan Vybíral,et al.  Entropy and Sampling Numbers of Classes of Ridge Functions , 2013, 1311.2005.

[22]  Jan Vybíral,et al.  Learning Functions of Few Arbitrary Linear Parameters in High Dimensions , 2010, Found. Comput. Math..

[23]  Ali Gannoun,et al.  Some extensions of multivariate sliced inverse regression , 2007 .

[24]  B. Iooss,et al.  A Review on Global Sensitivity Analysis Methods , 2014, 1404.2405.

[25]  P. Diaconis,et al.  On Nonlinear Functions of Linear Combinations , 1984 .

[26]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[27]  A. Samarov Exploring Regression Structure Using Nonparametric Functional Estimation , 1993 .

[28]  David Makowski,et al.  Multivariate sensitivity analysis to measure global contribution of input factors in dynamic models , 2011, Reliab. Eng. Syst. Saf..

[29]  Allan Pinkus Polynomial Ridge Functions , 2015 .

[30]  Harvey M. Wagner,et al.  Global Sensitivity Analysis , 1995, Oper. Res..

[31]  Olivier Roustant,et al.  Poincaré inequalities on intervals – application to sensitivity analysis , 2016, 1612.03689.

[32]  Ker-Chau Li,et al.  Sliced Inverse Regression for Dimension Reduction , 1991 .

[33]  S. Kucherenko,et al.  Derivative-Based Global Sensitivity Measures and Their Link with Sobol' Sensitivity Indices , 2016, MCQMC.

[34]  R. Cook,et al.  Sufficient dimension reduction and prediction in regression , 2009, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[35]  I. Sobola,et al.  Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates , 2001 .

[36]  Fabrice Gamboa,et al.  Sensitivity analysis for multidimensional and functional outputs , 2013, 1311.1797.

[37]  Robin Sibson,et al.  What is projection pursuit , 1987 .

[38]  Raul Tempone,et al.  An Adaptive Sparse Grid Algorithm for Elliptic PDEs with Lognormal Diffusion Coefficient , 2016 .

[39]  R. H. Moore,et al.  Regression Graphics: Ideas for Studying Regressions Through Graphics , 1998, Technometrics.

[40]  Tiangang Cui,et al.  Certified dimension reduction in nonlinear Bayesian inverse problems , 2018, Math. Comput..

[41]  Paul G. Constantine,et al.  Time‐dependent global sensitivity analysis with active subspaces for a lithium ion battery model , 2016, Stat. Anal. Data Min..

[42]  Matieyendou Lamboni,et al.  Derivative-based global sensitivity measures: General links with Sobol' indices and numerical tests , 2012, Math. Comput. Simul..

[43]  A. Saltelli,et al.  Importance measures in global sensitivity analysis of nonlinear models , 1996 .

[44]  Jérôme Saracco,et al.  Asymptotics for pooled marginal slicing estimator based on SIRα approach , 2005 .

[45]  Christoph Schwab,et al.  Karhunen-Loève approximation of random fields by generalized fast multipole methods , 2006, J. Comput. Phys..

[46]  R. Plessix A review of the adjoint-state method for computing the gradient of a functional with geophysical applications , 2006 .

[47]  Qiqi Wang,et al.  Erratum: Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces , 2013, SIAM J. Sci. Comput..

[48]  Louis H. Y. Chen An inequality for the multivariate normal distribution , 1982 .

[49]  Juan J. Alonso,et al.  Active Subspaces for Shape Optimization , 2014 .

[50]  I. Daubechies,et al.  Capturing Ridge Functions in High Dimensions from Point Queries , 2012 .

[51]  Ker-Chau Li,et al.  On Principal Hessian Directions for Data Visualization and Dimension Reduction: Another Application of Stein's Lemma , 1992 .

[52]  Fabrice Gamboa,et al.  Sensitivity indices for multivariate outputs , 2013, 1303.3574.

[53]  Trent Michael Russi,et al.  Uncertainty Quantification with Experimental Data and Complex System Models , 2010 .

[54]  Zhuyin Ren,et al.  Shared low-dimensional subspaces for propagating kinetic uncertainty to multiple outputs , 2018 .

[55]  Joseph Hart,et al.  An approximation theoretic perspective of the Sobol' indices with dependent variables. , 2017, 1801.01359.

[56]  Sergei S. Kucherenko,et al.  Derivative based global sensitivity measures and their link with global sensitivity indices , 2009, Math. Comput. Simul..