Optimal Experimental Design for Constrained Inverse Problems

In this paper, we address the challenging problem of optimal experimental design (OED) of constrained inverse problems. We consider two OED formulations that allow reducing the experimental costs by minimizing the number of measurements. The first formulation assumes a fine discretization of the design parameter space and uses sparsity promoting regularization to obtain an efficient design. The second formulation parameterizes the design and seeks optimal placement for these measurements by solving a small-dimensional optimization problem. We consider both problems in a Bayes risk as well as an empirical Bayes risk minimization framework. For the unconstrained inverse state problem, we exploit the closed form solution for the inner problem to efficiently compute derivatives for the outer OED problem. The empirical formulation does not require an explicit solution of the inverse problem and therefore allows to integrate constraints efficiently. A key contribution is an efficient optimization method for solving the resulting, typically high-dimensional, bilevel optimization problem using derivative-based methods. To overcome the lack of non-differentiability in active set methods for inequality constraints problems, we use a relaxed interior point method. To address the growing computational complexity of empirical Bayes OED, we parallelize the computation over the training models. Numerical examples and illustrations from tomographic reconstruction, for various data sets and under different constraints, demonstrate the impact of constraints on the optimal design and highlight the importance of OED for constrained problems.

[1]  Lior Horesh,et al.  Optimal estimation of L1 regularization prior from a regularized empirical bayesian risk standpoint , 2012 .

[2]  Xun Huan,et al.  Simulation-based optimal Bayesian experimental design for nonlinear systems , 2011, J. Comput. Phys..

[3]  Daniela Calvetti,et al.  Introduction to Bayesian Scientific Computing: Ten Lectures on Subjective Computing , 2007 .

[4]  P. Hansen Discrete Inverse Problems: Insight and Algorithms , 2010 .

[5]  J. D. Jakeman,et al.  A Consistent Bayesian Formulation for Stochastic Inverse Problems Based on Push-forward Measures , 2017, 1704.00680.

[6]  F. Natterer The Mathematics of Computerized Tomography , 1986 .

[7]  Luis Tenorio,et al.  Stochastic Newton and Quasi-Newton Methods for Large Linear Least-squares Problems , 2017, ArXiv.

[8]  Anthony C. Atkinson,et al.  Optimum Experimental Designs , 1992 .

[9]  David Isaacson,et al.  Electrical Impedance Tomography , 2002, IEEE Trans. Medical Imaging.

[10]  Avinash C. Kak,et al.  Principles of computerized tomographic imaging , 2001, Classics in applied mathematics.

[11]  John D. Jakeman,et al.  Optimal Experimental Design Using A Consistent Bayesian Approach , 2017, 1705.09395.

[12]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[13]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[14]  Georg Stadler,et al.  A-Optimal Design of Experiments for Infinite-Dimensional Bayesian Linear Inverse Problems with Regularized ℓ0-Sparsification , 2013, SIAM J. Sci. Comput..

[15]  S. Arridge Optical tomography in medical imaging , 1999 .

[16]  Luis Tenorio,et al.  Numerical methods for A-optimal designs with a sparsity constraint for ill-posed inverse problems , 2012, Comput. Optim. Appl..

[17]  E. Haber,et al.  Numerical methods for experimental design of large-scale linear ill-posed inverse problems , 2008 .

[18]  A. Rukhin Bayes and Empirical Bayes Methods for Data Analysis , 1997 .

[19]  Eran Treister,et al.  jInv-a Flexible Julia Package for PDE Parameter Estimation , 2016, SIAM J. Sci. Comput..

[20]  Eldad Haber,et al.  Computational Methods in Geophysical Electromagnetics , 2014, Mathematics in Industry.

[21]  K. Chaloner,et al.  Bayesian Experimental Design: A Review , 1995 .

[22]  B. F. Logan,et al.  The Fourier reconstruction of a head section , 1974 .

[23]  Georg Stadler,et al.  A Fast and Scalable Method for A-Optimal Design of Experiments for Infinite-dimensional Bayesian Nonlinear Inverse Problems , 2014, SIAM J. Sci. Comput..

[24]  Anil K. Jain Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[25]  E. Haber,et al.  Numerical methods for the design of large-scale nonlinear discrete ill-posed inverse problems , 2010 .

[26]  G. Stewart Matrix Algorithms, Volume II: Eigensystems , 2001 .

[27]  Jiang Hsieh,et al.  Computed Tomography: Principles, Design, Artifacts, and Recent Advances, Fourth Edition , 2022 .

[28]  Eldad Haber,et al.  Experimental Design for Biological Systems , 2012, SIAM J. Control. Optim..

[29]  Andrej Pázman,et al.  Foundations of Optimum Experimental Design , 1986 .

[30]  F. Pukelsheim Optimal Design of Experiments , 1993 .

[31]  Alexander Shapiro,et al.  Lectures on Stochastic Programming: Modeling and Theory , 2009 .

[32]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[33]  Samuel Kotz,et al.  Discrete Distributions: Distributions in Statistics , 1971 .

[34]  E. Haber,et al.  Optimal design of simultaneous source encoding , 2015 .

[35]  Y. Rubin,et al.  Bayesian geostatistical design: Task‐driven optimal site investigation when the geostatistical model is uncertain , 2010 .