Optimal experimental design for infinite-dimensional Bayesian inverse problems governed by PDEs: a review

We present a review of methods for optimal experimental design (OED) for Bayesian inverse problems governed by partial differential equations with infinite-dimensional parameters. The focus is on problems where one seeks to optimize the placement of measurement points, at which data are collected, such that the uncertainty in the estimated parameters is minimized. We present the mathematical foundations of OED in this context and survey the computational methods for the class of OED problems under study. We also outline some directions for future research in this area.

[1]  X. Huan,et al.  GRADIENT-BASED STOCHASTIC OPTIMIZATION METHODS IN BAYESIAN EXPERIMENTAL DESIGN , 2012, 1212.2228.

[2]  D. Ucinski.  Optimal measurement methods for distributed parameter system identification , 2004 .

[3]  Daniel Rudolf,et al.  On a Generalization of the Preconditioned Crank–Nicolson Metropolis Algorithm , 2015, Found. Comput. Math..

[4]  Georg Stadler,et al.  Optimal experimental design under irreducible uncertainty for inverse problems governed by PDEs , 2019, ArXiv.

[5]  K. J. Ryan,et al.  Estimating Expected Information Gains for Experimental Designs With Application to the Random Fatigue-Limit Model , 2003 .

[6]  Valerii V. Fedorov,et al.  Optimal Design for Nonlinear Response Models , 2013 .

[7]  Arvind K. Saibaba,et al.  Efficient D-Optimal Design of Experiments for Infinite-Dimensional Bayesian Linear Inverse Problems , 2018, SIAM J. Sci. Comput..

[8]  Dariusz Ucinski,et al.  D-optimal sensor selection in the presence of correlated measurement noise , 2020 .

[9]  P. M. Prenter,et al.  Stochastic inversion of linear first kind integral equations I. Continuous theory and the stochastic generalized inverse , 1985 .

[10]  Tiangang Cui,et al.  Dimension-independent likelihood-informed MCMC , 2014, J. Comput. Phys..

[11]  Chi Feng,et al.  A layered multiple importance sampling scheme for focused optimal Bayesian experimental design. , 2019, 1903.11187.

[12]  Stig Larsson,et al.  Posterior Contraction Rates for the Bayesian Approach to Linear Ill-Posed Inverse Problems , 2012, 1203.5753.

[13]  John D. Jakeman,et al.  Optimal Experimental Design Using A Consistent Bayesian Approach , 2017, 1705.09395.

[14]  Michael Prange,et al.  Bayesian survey design to optimize resolution in waveform inversion , 2012 .

[15]  Roland Herzog,et al.  Optimal sensor placement for joint parameter and state estimation problems in large-scale dynamical systems with applications to thermo-mechanics , 2018, Optimization and Engineering.

[16]  Georg Stadler,et al.  Optimal Design of Large-scale Bayesian Linear Inverse Problems Under Reducible Model Uncertainty: Good to Know What You Don't Know , 2020, SIAM/ASA J. Uncertain. Quantification.

[17]  Albert Tarantola,et al.  Inverse problem theory - and methods for model parameter estimation , 2004 .

[18]  J. Kiefer,et al.  Optimum Designs in Regression Problems , 1959 .

[19]  Arvind K. Saibaba,et al.  Goal-oriented optimal design of experiments for large-scale Bayesian linear inverse problems , 2018, Inverse Problems.

[20]  Roland Herzog,et al.  Optimum Experimental Design by Shape Optimization of Specimens in Linear Elasticity , 2018, SIAM J. Appl. Math..

[21]  E. Haber,et al.  Numerical methods for the design of large-scale nonlinear discrete ill-posed inverse problems , 2010 .

[22]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[23]  M. Reed.  Methods of Modern Mathematical Physics. I: Functional Analysis , 1972 .

[24]  Erkki Somersalo,et al.  Linear inverse problems for generalised random variables , 1989 .

[25]  Andreas Krause,et al.  Near-Optimal Sensor Placements in Gaussian Processes: Theory, Efficient Algorithms and Empirical Studies , 2008, J. Mach. Learn. Res..

[26]  Luis Tenorio,et al.  Numerical methods for A-optimal designs with a sparsity constraint for ill-posed inverse problems , 2012, Comput. Optim. Appl..

[27]  M. Hutchinson.  A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines , 1989 .

[28]  Matti Lassas. Eero Saksman,et al.  Discretization-invariant Bayesian inversion and Besov space priors , 2009, 0901.4220.

[29]  Georg Stadler,et al.  Scalable and efficient algorithms for the propagation of uncertainty from data through inference to prediction for large-scale problems, with application to flow of the Antarctic ice sheet , 2014, J. Comput. Phys..

[30]  Roland Eils,et al.  Optimal Experimental Design for Parameter Estimation of a Cell Signaling Model , 2009, PLoS Comput. Biol..

[31]  Tim Wildey,et al.  Optimal experimental design for prediction based on push-forward probability measures , 2020, J. Comput. Phys..

[32]  Hans Bock,et al.  Numerical methods for optimum experimental design in DAE systems , 2000 .

[33]  Sundeep Prabhakar Chepuri,et al.  Sensor Selection for Estimation with Correlated Measurement Noise , 2015, IEEE Transactions on Signal Processing.

[34]  E. Haber,et al.  Numerical methods for experimental design of large-scale linear ill-posed inverse problems , 2008 .

[35]  Andrew M. Stuart,et al.  Geometric MCMC for infinite-dimensional inverse problems , 2016, J. Comput. Phys..

[36]  T. Kurtz,et al.  Stochastic equations in infinite dimensions , 2006 .

[37]  Arvind K. Saibaba,et al.  Efficient Marginalization-Based MCMC Methods for Hierarchical Bayesian Inverse Problems , 2019, SIAM/ASA J. Uncertain. Quantification.

[38]  Giuseppe Da Prato,et al.  Second Order Partial Differential Equations in Hilbert Spaces: Bibliography , 2002 .

[39]  Lior Horesh,et al.  Experimental Design for Nonparametric Correction of Misspecified Dynamical Models , 2017, SIAM/ASA J. Uncertain. Quantification.

[40]  Peng Chen,et al.  A fast and scalable computational framework for large-scale and high-dimensional Bayesian optimal experimental design , 2020, SIAM/ASA Journal on Uncertainty Quantification.

[41]  Sivan Toledo,et al.  Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix , 2011, JACM.

[42]  Andrew M. Stuart,et al.  Inverse problems: A Bayesian perspective , 2010, Acta Numerica.

[43]  A. Stuart,et al.  The Bayesian Approach to Inverse Problems , 2013, 1302.6989.

[44]  Gideon Simpson,et al.  Kullback-Leibler Approximation for Probability Measures on Infinite Dimensional Spaces , 2013, SIAM J. Math. Anal..

[45]  Victor M. Zavala,et al.  A scalable design of experiments framework for optimal sensor placement , 2017, Journal of Process Control.

[46]  Daniel Walter,et al.  On sparse sensor placement for parameter identification problems with partial differential equations , 2019 .

[47]  Werner G. Müller,et al.  Collecting Spatial Data: Optimum Design of Experiments for Random Fields , 1998 .

[48]  A. Stuart,et al.  MAP estimators and their consistency in Bayesian nonparametric inverse problems , 2013, 1303.4795.

[49]  G. Prato.  An Introduction to Infinite-Dimensional Analysis , 2006 .

[50]  Georg Stadler,et al.  A Fast and Scalable Method for A-Optimal Design of Experiments for Infinite-dimensional Bayesian Nonlinear Inverse Problems , 2014, SIAM J. Sci. Comput..

[51]  J. D. Jakeman,et al.  A Consistent Bayesian Formulation for Stochastic Inverse Problems Based on Push-forward Measures , 2017, 1704.00680.

[52]  A. Alexanderian,et al.  Hyper-differential sensitivity analysis for inverse problems constrained by partial differential equations , 2020, Inverse Problems.

[53]  Paula Castro,et al.  A bilevel learning approach for optimal observation placement in variational data assimilation , 2018, Inverse Problems.

[54]  James Martin,et al.  A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems, Part II: Stochastic Newton MCMC with Application to Ice Sheet Flow Inverse Problems , 2013, SIAM J. Sci. Comput..

[55]  J. Rosenthal,et al.  Optimal scaling for various Metropolis-Hastings algorithms , 2001 .

[56]  Anthony N. Pettitt,et al.  A Review of Modern Computational Algorithms for Bayesian Optimal Design , 2016 .

[57]  Georg Stadler,et al.  A-Optimal Design of Experiments for Infinite-Dimensional Bayesian Linear Inverse Problems with Regularized ℓ0-Sparsification , 2013, SIAM J. Sci. Comput..

[58]  Joel Franklin,et al.  Well-posed stochastic extensions of ill-posed linear problems☆ , 1970 .

[59]  Antti Hannukainen,et al.  Inverse heat source problem and experimental design for determining iron loss distribution , 2020, SIAM J. Sci. Comput..

[60]  Merlise A. Clyde,et al.  Experimental Design: A Bayesian Perspective , 2001 .

[61]  S. Siltanen,et al.  Can one use total variation prior for edge-preserving Bayesian inversion? , 2004 .

[62]  Raul Tempone,et al.  Fast Bayesian experimental design: Laplace-based importance sampling for the expected information gain , 2017, Computer Methods in Applied Mechanics and Engineering.

[63]  A. Stuart,et al.  Besov priors for Bayesian inverse problems , 2011, 1105.0889.

[64]  K. Chaloner,et al.  Bayesian Experimental Design: A Review , 1995 .

[65]  Jesús López-Fidalgo,et al.  Design of experiments for nonlinear models , 2011 .

[66]  Quan Long,et al.  Fast Bayesian Optimal Experimental Design for Seismic Source Inversion , 2015, 1502.07873.

[67]  Alejandro Ribeiro,et al.  Approximate Supermodularity Bounds for Experimental Design , 2017, NIPS.

[68]  Hans Bock,et al.  A sequential approach for nonlinear optimum experimental design in DAE systems , 1999 .

[69]  James Martin,et al.  A Computational Framework for Infinite-Dimensional Bayesian Inverse Problems Part I: The Linearized Case, with Application to Global Seismic Inversion , 2013, SIAM J. Sci. Comput..

[70]  Julianne Chung,et al.  Optimal Experimental Design for Inverse Problems with State Constraints , 2018, SIAM J. Sci. Comput..

[71]  Christoph Schwab,et al.  Analysis of multilevel MCMC-FEM for Bayesian inversion of log-normal diffusions , 2019 .

[72]  Martin Weiser,et al.  Optimal design of experiments for estimating the time of death in forensic medicine , 2018, Inverse Problems.

[73]  Hans Bock,et al.  Parameter Estimation and Optimum Experimental Design for Differential Equation Models , 2013 .

[74]  Alen Alexanderian,et al.  Randomization and Reweighted ℓ1-Minimization for A-Optimal Design of Linear Inverse Problems , 2019, SIAM J. Sci. Comput..

[75]  Mikolaj Bojanczyk,et al.  XPath evaluation in linear time , 2011, JACM.

[76]  Raul Tempone,et al.  Fast estimation of expected information gains for Bayesian experimental designs based on Laplace approximations , 2013 .

[77]  Ben G. Fitzpatrick,et al.  Bayesian analysis in inverse problems , 1991 .

[78]  Eldad Haber,et al.  Experimental Design for Biological Systems , 2012, SIAM J. Control. Optim..

[79]  A. Atkinson.  The Usefulness of Optimum Experimental Designs , 1996 .

[80]  J. Conway.  A course in operator theory , 1999 .

[81]  Boris Vexler,et al.  A sparse control approach to optimal sensor placement in PDE-constrained parameter estimation problems , 2019, Numerische Mathematik.

[82]  G. Roberts,et al.  MCMC Methods for Functions: ModifyingOld Algorithms to Make Them Faster , 2012, 1202.0709.

[83]  A. Mandelbaum,et al.  Linear estimators and measurable linear transformations on a Hilbert space , 1984 .

[84]  Ilse C. F. Ipsen,et al.  Randomized matrix-free trace and log-determinant estimators , 2016, Numerische Mathematik.

[85]  T. Bui-Thanh,et al.  FEM-based discretization-invariant MCMC methods for PDE-constrained Bayesian inverse problems , 2016 .

[86]  M. L. Fisher,et al.  An analysis of approximations for maximizing submodular set functions—I , 1978, Math. Program..

[87]  Tim Wildey,et al.  Combining Push-Forward Measures and Bayes' Rule to Construct Consistent Solutions to Stochastic Inverse Problems , 2018, SIAM J. Sci. Comput..

[88]  A. Stuart,et al.  Spectral gaps for a Metropolis–Hastings algorithm in infinite dimensions , 2011, 1112.1392.

[89]  Xun Huan,et al.  Simulation-based optimal Bayesian experimental design for nonlinear systems , 2011, J. Comput. Phys..

[90]  O. Ghattas,et al.  On Bayesian A- and D-optimal experimental designs in infinite dimensions , 2014, 1408.6323.

[91]  NUUTTI HYVÖNEN,et al.  Optimizing Electrode Positions in Electrical Impedance Tomography , 2014, SIAM J. Appl. Math..

[92]  Thomas Ward,et al.  Functional analysis, spectral theory, and applications (graduate texts in mathematics) , 2017 .

[93]  Y. Marzouk,et al.  Batch greedy maximization of non-submodular functions: Guarantees and applications to experimental design , 2020, J. Mach. Learn. Res..

[94]  J. M. Sanz-Serna,et al.  Hybrid Monte Carlo on Hilbert spaces , 2011 .

[95]  U. von Toussaint,et al.  Bayesian experimental design , 2014 .

[96]  Johannes P. Schlöder,et al.  Numerical methods for optimal control problems in design of robust optimal experiments for nonlinear dynamic processes , 2004, Optim. Methods Softw..

[97]  Tiangang Cui,et al.  Scalable Optimization-Based Sampling on Function Space , 2019, SIAM J. Sci. Comput..

[98]  Y. Marzouk,et al.  Large-Scale Inverse Problems and Quantification of Uncertainty , 1994 .

[99]  Björn Sprungk.  On the local Lipschitz stability of Bayesian inverse problems , 2019 .

[100]  Martin Burger,et al.  Maximum a posteriori probability estimates in infinite-dimensional Bayesian inverse problems , 2014, 1412.5816.

[101]  P. Müller.  Simulation Based Optimal Design , 2005 .

[102]  E. Haber,et al.  Optimal Experimental Design for the Large‐Scale Nonlinear Ill‐Posed Problem of Impedance Imaging , 2010 .