Hierarchical Bayesian inference for Ill-posed problems via variational method

This paper investigates a novel approximate Bayesian inference procedure for numerically solving inverse problems. A hierarchical formulation which determines automatically the regularization parameter and the noise level together with the inverse solution is adopted. The framework is of variational type, and it can deliver the inverse solution and regularization parameter together with their uncertainties calibrated. It approximates the posteriori probability distribution by separable distributions based on Kullback-Leibler divergence. Two approximations are derived within the framework, and some theoretical properties, e.g. variance estimate and consistency, are also provided. Algorithms for their efficient numerical realization are described, and their convergence properties are also discussed. Extensions to nonquadratic regularization/nonlinear forward models are also briefly studied. Numerical results for linear and nonlinear Cauchy-type problems arising in heat conduction with both smooth and nonsmooth solutions are presented for the proposed method, and compared with that by Markov chain Monte Carlo. The results illustrate that the variational method can faithfully capture the posteriori distribution in a computationally efficient way.

[1]  Alberto Malinverno,et al.  Expanded uncertainty quantification in inverse problems: Hierarchical Bayes and empirical Bayes , 2004 .

[2]  Mokhtar S. Bazaraa,et al.  Nonlinear Programming: Theory and Algorithms , 1993 .

[3]  Bangti Jin,et al.  A new choice rule for regularization parameters in Tikhonov regularization , 2011 .

[4]  Bo Wang,et al.  Inadequacy of interval estimates corresponding to variational Bayesian approximations , 2005, AISTATS.

[5]  Matthew J. Beal.  Variational algorithms for approximate Bayesian inference , 2003 .

[6]  B. Blackwell,et al.  Inverse Heat Conduction: Ill-Posed Problems , 1985 .

[7]  Ting Wei,et al.  Backus-Gilbert algorithm for the Cauchy problem of the Laplace equation , 2001 .

[8]  P. P. B. Eggermont,et al.  Maximum entropy regularization for Fredholm integral equations of the first kind , 1993 .

[9]  Albert Tarantola,et al.  Inverse problem theory - and methods for model parameter estimation , 2004 .

[10]  V. Šmídl,et al.  The Variational Bayes Method in Signal Processing , 2005 .

[11]  Bangti Jin,et al.  Fast Bayesian approach for parameter estimation , 2008 .

[12]  H. Engl,et al.  Regularization of Inverse Problems , 1996 .

[13]  Aggelos K. Katsaggelos,et al.  Non-convex priors in Bayesian compressed sensing , 2009, 2009 17th European Signal Processing Conference.

[14]  A. Osman.  Nonlinear inverse problem for the estimation of time-and-space dependent heat transfer coefficients , 1987 .

[15]  Y. Y. Belov,et al.  Inverse Problems for Partial Differential Equations , 2002 .

[16]  Per Christian Hansen,et al.  Rank-Deficient and Discrete Ill-Posed Problems , 1996 .

[17]  Heikki Haario,et al.  DRAM: Efficient adaptive MCMC , 2006, Stat. Comput..

[18]  E. Somersalo,et al.  Statistical and computational inverse problems , 2004 .

[19]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[20]  Mark W. Woolrich,et al.  Variational Bayesian Inference for a Nonlinear Forward Model , 2020, IEEE Transactions on Signal Processing.

[21]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[22]  Jun S. Liu,et al.  Monte Carlo strategies in scientific computing , 2001 .

[23]  Bangti Jin,et al.  A Bayesian inference approach to the ill‐posed Cauchy problem of steady‐state heat conduction , 2008 .

[24]  Mike West,et al.  Markov Random Field Models for High-Dimensional Parameters in Simulations of Fluid Flow in Porous Media , 2002, Technometrics.

[25]  Bangti Jin,et al.  Augmented Tikhonov regularization , 2009 .

[26]  Ashley F. Emery,et al.  Using Bayesian inference for parameter estimation when the system response and experimental conditions are measured with error and some variables are considered as nuisance variables , 2006 .

[27]  Alison L Gibbs,et al.  On Choosing and Bounding Probability Metrics , 2002, math/0209021.

[28]  Bangti Jin,et al.  The plane wave method for inverse problems associated with Helmholtz-type equations , 2008 .

[29]  P. Hansen.  Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion , 1987 .

[30]  Nicholas Zabaras,et al.  A Bayesian inference approach to the inverse heat conduction problem , 2004 .

[31]  Bangti Jin,et al.  Numerical estimation of the Robin coefficient in a stationary diffusion equation , 2010 .

[32]  Aggelos K. Katsaggelos,et al.  Parameter Estimation in TV Image Restoration Using Variational Distribution Approximation , 2008, IEEE Transactions on Image Processing.

[33]  Nicholas Zabaras,et al.  Hierarchical Bayesian models for inverse problems in heat conduction , 2005 .

[34]  Michael I. Jordan,et al.  An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[35]  Masa-aki Sato,et al.  Hierarchical Bayesian estimation for MEG inverse problem , 2004, NeuroImage.

[36]  R. Anderssen,et al.  Joint additive Kullback–Leibler residual minimization and regularization for linear inverse problems , 2007 .

[37]  Hagai Attias,et al.  A Variational Bayesian Framework for Graphical Models , 1999 .

[38]  Vaclav Smidl,et al.  The Variational Bayes Method in Signal Processing (Signals and Communication Technology) , 2005 .

[39]  George Casella,et al.  Estimating bayesian credible intervals , 2003 .