Accelerating Markov Chain Monte Carlo with Active Subspaces

The Markov chain Monte Carlo (MCMC) method is the computational workhorse for Bayesian inverse problems. However, MCMC struggles in high-dimensional parameter spaces, since its iterates must sequentially explore the high-dimensional space. This struggle is compounded in physical applications when the nonlinear forward model is computationally expensive. One approach to accelerate MCMC is to reduce the dimension of the state space. Active subspaces are part of an emerging set of tools for subspace-based dimension reduction. An active subspace in a given inverse problem indicates a separation between a low-dimensional subspace that is informed by the data and its orthogonal complement that is constrained by the prior. With this information, one can run the sequential MCMC on the active variables while sampling independently according to the prior on the inactive variables. However, this approach to increase efficiency may introduce bias. We provide a bound on the Hellinger distance between the true posterio...

[1]  Habib N. Najm,et al.  Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems , 2008, J. Comput. Phys..

[2]  Johan Larsson,et al.  Exploiting active subspaces to quantify uncertainty in the numerical simulation of the HyShot II scramjet , 2014, J. Comput. Phys..

[3]  Murali Haran,et al.  Markov chain Monte Carlo: Can we trust the third significant figure? , 2007, math/0703746.

[4]  D. Gleich,et al.  Computing active subspaces with Monte Carlo , 2014, 1408.0545.

[5]  Qiqi Wang,et al.  Erratum: Active Subspace Methods in Theory and Practice: Applications to Kriging Surfaces , 2013, SIAM J. Sci. Comput..

[6]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[7]  Daniela Calvetti,et al.  Introduction to Bayesian Scientific Computing: Ten Lectures on Subjective Computing , 2007 .

[8]  Faming Liang,et al.  Statistical and Computational Inverse Problems , 2006, Technometrics.

[9]  James Martin,et al.  A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion , 2012, SIAM J. Sci. Comput..

[10]  Heikki Haario,et al.  DRAM: Efficient adaptive MCMC , 2006, Stat. Comput..

[11]  Andrew Gelman,et al.  Handbook of Markov Chain Monte Carlo , 2011 .

[12]  Bart G. van Bloemen Waanders,et al.  Fast Algorithms for Bayesian Uncertainty Quantification in Large-Scale Linear Inverse Problems Based on Low-Rank Partial Hessian Approximations , 2011, SIAM J. Sci. Comput..

[13]  M. Girolami,et al.  Solving large-scale PDE-constrained Bayesian inverse problems with Riemann manifold Hamiltonian Monte Carlo , 2014, 1407.1517.

[14]  Tiangang Cui,et al.  Optimal Low-rank Approximations of Bayesian Linear Inverse Problems , 2014, SIAM J. Sci. Comput..

[15]  Paul G. Constantine,et al.  Active Subspaces - Emerging Ideas for Dimension Reduction in Parameter Studies , 2015, SIAM spotlights.

[16]  Juan J. Alonso,et al.  Active Subspaces for Shape Optimization , 2014 .

[17]  Andrew M. Stuart,et al.  Inverse problems: A Bayesian perspective , 2010, Acta Numerica.

[18]  Tiangang Cui,et al.  Likelihood-informed dimension reduction for nonlinear inverse problems , 2014, 1403.4680.

[19]  A. Stuart,et al.  Sampling the posterior: An approach to non-Gaussian data assimilation , 2007 .

[20]  Alison L Gibbs,et al.  On Choosing and Bounding Probability Metrics , 2002, math/0209021.

[21]  M. Girolami,et al.  Riemann manifold Langevin and Hamiltonian Monte Carlo methods , 2011, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[22]  D. Higdon,et al.  Accelerating Markov Chain Monte Carlo Simulation by Differential Evolution with Self-Adaptive Randomized Subspace Sampling , 2009 .

[23]  Georg Stadler,et al.  Extreme-scale UQ for Bayesian inverse problems governed by PDEs , 2012, 2012 International Conference for High Performance Computing, Networking, Storage and Analysis.

[24]  Tiangang Cui,et al.  Dimension-independent likelihood-informed MCMC , 2014, J. Comput. Phys..

[25]  Louis H. Y. Chen.  An inequality for the multivariate normal distribution , 1982 .

[26]  Paul G. Constantine,et al.  Discovering an active subspace in a single‐diode solar cell model , 2014, Stat. Anal. Data Min..

[27]  Paul G. Constantine,et al.  Active subspaces for sensitivity analysis and dimension reduction of an integrated hydrologic model , 2015, Comput. Geosci..