Expectation propagation for nonlinear inverse problems - with an application to electrical impedance tomography

In this paper, we study a fast approximate inference method based on expectation propagation for exploring the posterior probability distribution arising from the Bayesian formulation of nonlinear inverse problems. It is capable of efficiently delivering reliable estimates of the posterior mean and covariance, thereby providing an inverse solution together with quantified uncertainties. Some theoretical properties of the iterative algorithm are discussed, and the efficient implementation for an important class of problems of projection type is described. The method is illustrated with one typical nonlinear inverse problem, electrical impedance tomography with complete electrode model, under sparsity constraints. Numerical results for real experimental data are presented, and compared with that by Markov chain Monte Carlo. The results indicate that the method is accurate and computationally very efficient.

[1]  Tim Hesterberg,et al.  Monte Carlo Strategies in Scientific Computing , 2002, Technometrics.

[2]  Dave Higdon,et al.  A Bayesian approach to characterizing uncertainty in inverse problems using coarse and fine-scale information , 2002, IEEE Trans. Signal Process..

[3]  D. Isaacson,et al.  Electrode models for electric current computed tomography , 1989, IEEE Transactions on Biomedical Engineering.

[4]  Bangti Jin,et al.  A variational Bayesian method to inverse problems with impulsive noise , 2011, J. Comput. Phys..

[5]  Joel Franklin,et al.  Well-posed stochastic extensions of ill-posed linear problems☆ , 1970 .

[6]  Bangti Jin,et al.  A Semismooth Newton Method for Nonlinear Parameter Identification Problems with Impulsive Noise , 2012, SIAM J. Imaging Sci..

[7]  Jari P. Kaipio,et al.  Sparsity reconstruction in electrical impedance tomography: An experimental evaluation , 2012, J. Comput. Appl. Math..

[8]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[9]  J. Kaipio,et al.  The Bayesian approximation error approach for electrical impedance tomography—experimental results , 2007 .

[10]  Ilias Bilionis,et al.  A stochastic optimization approach to coarse-graining using a relative-entropy framework. , 2013, The Journal of chemical physics.

[11]  Bangti Jin,et al.  A reconstruction algorithm for electrical impedance tomography based on sparsity regularization , 2012 .

[12]  Marko Vauhkonen,et al.  Simultaneous reconstruction of electrode contact impedances and internal electrical properties: II. Laboratory experiments , 2002 .

[13]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[14]  R. Tweedie,et al.  Langevin-Type Models II: Self-Targeting Candidates for MCMC Algorithms* , 1999 .

[15]  Florian Steinke,et al.  Bayesian Inference and Optimal Design in the Sparse Linear Model , 2007, AISTATS.

[16]  Youssef M. Marzouk,et al.  Bayesian inference with optimal maps , 2011, J. Comput. Phys..

[17]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[18]  Andrew Gelman,et al.  General methods for monitoring convergence of iterative simulations , 1998 .

[19]  M. Girolami,et al.  Riemann manifold Langevin and Hamiltonian Monte Carlo methods , 2011, Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[20]  Tom Minka,et al.  Expectation Propagation for approximate Bayesian inference , 2001, UAI.

[21]  Robert G Aykroyd,et al.  Markov chain Monte Carlo techniques and spatial-temporal modelling for medical EIT. , 2004, Physiological measurement.

[22]  Daniel Watzenig,et al.  A review of statistical modelling and inference for electrical capacitance tomography , 2009 .

[23]  Geoff K. Nicholls,et al.  Prior modeling and posterior sampling in impedance imaging , 1998, Optics & Photonics.

[24]  P. Maass,et al.  An analysis of electrical impedance tomography with applications to Tikhonov regularization , 2012 .

[25]  Matthew J. Beal.  Variational algorithms for approximate Bayesian inference , 2003 .

[26]  Tom Minka,et al.  A family of algorithms for approximate Bayesian inference , 2001 .

[27]  James Martin,et al.  A Stochastic Newton MCMC Method for Large-Scale Statistical Inverse Problems with Application to Seismic Inversion , 2012, SIAM J. Sci. Comput..

[28]  Habib N. Najm,et al.  Stochastic spectral methods for efficient Bayesian solution of inverse problems , 2005, J. Comput. Phys..

[29]  M Scott Shell,et al.  Coarse-graining errors and numerical optimization using a relative entropy framework. , 2011, The Journal of chemical physics.

[30]  E. Somersalo,et al.  Statistical inversion and Monte Carlo sampling methods in electrical impedance tomography , 2000 .

[31]  Gene H. Golub,et al.  Methods for modifying matrix factorizations , 1972, Milestones in Matrix Computation.

[32]  Junbin Gao,et al.  Robust L1 Principal Component Analysis and Its Bayesian Variational Inference , 2008, Neural Computation.

[33]  Jari P. Kaipio,et al.  Simultaneous reconstruction of electrode contact impedances and internal electrical properties: I. Theory , 2002 .

[34]  Faming Liang,et al.  Statistical and Computational Inverse Problems , 2006, Technometrics.

[35]  Yalchin Efendiev,et al.  Preconditioning Markov Chain Monte Carlo Simulations Using Coarse-Scale Models , 2006, SIAM J. Sci. Comput..

[36]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[37]  E. Somersalo,et al.  Existence and uniqueness for electrode models for electric current computed tomography , 1992 .

[38]  V. Bogachev.  Gaussian Measures on a , 2022 .

[39]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine Learning.

[40]  Tom Heskes,et al.  Efficient Bayesian multivariate fMRI analysis using a sparsifying spatio-temporal prior , 2010, NeuroImage.

[41]  P. S. Dwyer.  Some Applications of Matrix Derivatives in Multivariate Analysis , 1967 .

[42]  P. Maass,et al.  Sparsity regularization for parameter identification problems , 2012 .

[43]  Nicholas Zabaras,et al.  Hierarchical Bayesian models for inverse problems in heat conduction , 2005 .

[44]  Ole Winther,et al.  Expectation Consistent Approximate Inference , 2005, J. Mach. Learn. Res..

[45]  Bart G. van Bloemen Waanders,et al.  Fast Algorithms for Bayesian Uncertainty Quantification in Large-Scale Linear Inverse Problems Based on Low-Rank Partial Hessian Approximations , 2011, SIAM J. Sci. Comput..

[46]  Bangti Jin,et al.  Fast Bayesian approach for parameter estimation , 2008 .

[47]  E. Somersalo,et al.  Approximation errors and model reduction with an application in optical diffusion tomography , 2006 .

[48]  L. Brown.  Fundamentals of statistical exponential families: with applications in statistical decision theory , 1986 .

[49]  E. Lieb,et al.  On extensions of the Brunn-Minkowski and Prékopa-Leindler theorems, including inequalities for log concave functions, and with an application to the diffusion equation , 1976 .

[50]  C. Fox,et al.  Markov chain Monte Carlo Using an Approximation , 2005 .