Sparsity-promoting and edge-preserving maximum a posteriori estimators in non-parametric Bayesian inverse problems

We consider the inverse problem of recovering an unknown functional parameter $u$ in a separable Banach space, from a noisy observation $y$ of its image through a known possibly non-linear ill-posed map ${\mathcal G}$. The data $y$ is finite-dimensional and the noise is Gaussian. We adopt a Bayesian approach to the problem and consider Besov space priors (see Lassas et al. 2009), which are well-known for their edge-preserving and sparsity-promoting properties and have recently attracted wide attention especially in the medical imaging community. Our key result is to show that in this non-parametric setup the maximum a posteriori (MAP) estimates are characterized by the minimizers of a generalized Onsager--Machlup functional of the posterior. This is done independently for the so-called weak and strong MAP estimates, which as we show coincide in our context. In addition, we prove a form of weak consistency for the MAP estimators in the infinitely informative data limit. Our results are remarkable for two reasons: first, the prior distribution is non-Gaussian and does not meet the smoothness conditions required in previous research on non-parametric MAP estimates. Second, the result analytically justifies existing uses of the MAP estimate in finite but high dimensional discretizations of Bayesian inverse problems with the considered Besov priors.

[1]  Simon R. Arridge,et al.  Bayesian parameter estimation in spectral quantitative photoacoustic tomography , 2016, SPIE BiOS.

[2]  M. Lassas,et al.  Hierarchical models in statistical inverse problems and the Mumford–Shah functional , 2009, 0908.3396.

[3]  Larry A Shepp,et al.  Distinguishing a Sequence of Random Variables from a Translate of Itself , 1965 .

[4]  R. Ramlau,et al.  A stochastic convergence analysis for Tikhonov regularization with sparsity constraints , 2014 .

[5]  Albert Tarantola,et al.  Inverse problem theory - and methods for model parameter estimation , 2004 .

[6]  Johnathan M. Bardsley,et al.  Hierarchical regularization for edge-preserving reconstruction of PET images , 2010 .

[7]  Steven A. Orszag,et al.  CBMS-NSF REGIONAL CONFERENCE SERIES IN APPLIED MATHEMATICS , 1978 .

[8]  A. W. van der Vaart,et al.  Bayesian Recovery of the Initial Condition for the Heat Equation , 2011, 1111.5876.

[9]  G. Roberts,et al.  Nonparametric estimation of diffusions: a differential equations approach , 2012 .

[10]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[11]  G. Prato An Introduction to Infinite-Dimensional Analysis , 2006 .

[12]  C. Borell Convex measures on locally convex spaces , 1974 .

[13]  V. Bogachev,et al.  Analytic properties of infinite-dimensional distributions , 1990 .

[14]  David L Donoho,et al.  NMR data processing using iterative thresholding and minimum l(1)-norm reconstruction. , 2007, Journal of magnetic resonance.

[15]  E Somersalo,et al.  Statistical inversion for medical x-ray tomography with few radiographs: II. Application to dental radiology. , 2003, Physics in medicine and biology.

[16]  Omar Ghattas,et al.  A scalable algorithm for MAP estimators in Bayesian inverse problems with Besov priors , 2015 .

[17]  Tapio Helin,et al.  On infinite-dimensional hierarchical probability models in statistical inverse problems , 2009, 0907.5322.

[18]  Andrew M. Stuart,et al.  Inverse problems: A Bayesian perspective , 2010, Acta Numerica.

[19]  T. J. Sullivan,et al.  Equivalence of weak and strong modes of measures on topological vector spaces , 2017, Inverse Problems.

[20]  Daniela Calvetti,et al.  A Gaussian hypermodel to recover blocky objects , 2007 .

[21]  A. M. Stuart,et al.  Posterior consistency via precision operators for Bayesian nonparametric drift estimation in SDEs , 2012, 1202.0976.

[22]  Sari Lasanen,et al.  Non-Gaussian statistical inverse problems. Part I: Posterior distributions , 2012 .

[23]  A. V. D. Vaart,et al.  BAYESIAN INVERSE PROBLEMS WITH GAUSSIAN PRIORS , 2011, 1103.2692.

[24]  A. Stuart,et al.  MAP estimators and their consistency in Bayesian nonparametric inverse problems , 2013, 1303.4795.

[25]  A. Stuart,et al.  The Bayesian Approach to Inverse Problems , 2013, 1302.6989.

[26]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[27]  H. Zanten,et al.  Gaussian process methods for one-dimensional diffusions: optimal rates and adaptation , 2015, 1506.00515.

[28]  E Somersalo,et al.  Statistical inversion for medical x-ray tomography with few radiographs: I. General theory. , 2003, Physics in medicine and biology.

[29]  Daniela Calvetti,et al.  Variable order smoothness priors for ill-posed inverse problems , 2014, Math. Comput..

[30]  David Leporini,et al.  Bayesian wavelet denoising: Besov priors and non-Gaussian noises , 2001, Signal Process..

[31]  A. Stuart,et al.  Besov priors for Bayesian inverse problems , 2011, 1105.0889.

[32]  B. Knapik,et al.  A general approach to posterior contraction in nonparametric inverse problems , 2014, Bernoulli.

[33]  Matti Lassas,et al.  Wavelet-based reconstruction for limited-angle X-ray tomography , 2006, IEEE Transactions on Medical Imaging.

[34]  A. V. D. Vaart,et al.  Empirical Bayes scaling of Gaussian priors in the white noise model , 2013 .

[35]  V. Kolehmainen,et al.  Sparsity-promoting Bayesian inversion , 2012 .

[36]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Vladimir I. Bogachev,et al.  Differentiable measures and the Malliavin calculus , 2010 .

[38]  I. Daubechies,et al.  Tomographic inversion using L1-norm regularization of wavelet coefficients , 2006, physics/0608094.

[39]  T. W. Anderson The integral of a symmetric unimodal function over a symmetric convex set and some probability inequalities , 1955 .

[40]  S. Siltanen,et al.  Can one use total variation prior for edge-preserving Bayesian inversion? , 2004 .

[41]  Matti Lassas,et al.  Posterior consistency and convergence rates for Bayesian inversion with hypoelliptic operators , 2015, 1507.01772.

[42]  T. Chan,et al.  Edge-preserving and scale-dependent properties of total variation regularization , 2003 .

[43]  Jinghuai Gao,et al.  Bayesian approach to inverse problems for functions with a variable-index Besov prior , 2015, 1508.05680.

[44]  T. J. Sullivan,et al.  Well-posed Bayesian inverse problems and heavy-tailed stable quasi-Banach space priors , 2016, 1605.05898.

[45]  Daniela Calvetti,et al.  Hypermodels in the Bayesian imaging framework , 2008 .

[46]  I. Johnstone Minimax Bayes, Asymptotic Minimax and Sparse Wavelet Priors , 1994 .

[47]  Fadil Santosa,et al.  Recovery of Blocky Images from Noisy and Blurred Data , 1996, SIAM J. Appl. Math..

[48]  Dong H. Park Probability and its Applications for Engineers , 1993 .

[49]  W. Rudin Real and complex analysis , 1968 .

[50]  H. Engl,et al.  Regularization of Inverse Problems , 1996 .

[51]  Joel Franklin,et al.  Well-posed stochastic extensions of ill-posed linear problems☆ , 1970 .

[52]  Stig Larsson,et al.  Posterior Contraction Rates for the Bayesian Approach to Linear Ill-Posed Inverse Problems , 2012, 1203.5753.

[53]  Erkki Somersalo,et al.  Linear inverse problems for generalised random variables , 1989 .

[54]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[55]  Matti Lassas. Eero Saksman,et al.  Discretization-invariant Bayesian inversion and Besov space priors , 2009, 0901.4220.

[56]  S. Vollmer,et al.  Posterior consistency for Bayesian inverse problems through stability and regression results , 2013, 1302.4101.

[57]  A. Mandelbaum,et al.  Linear estimators and measurable linear transformations on a Hilbert space , 1984 .

[58]  Andrew M. Stuart,et al.  MAP estimators for piecewise continuous inversion , 2015, 1509.03136.

[59]  A. W. Vaart,et al.  Bayes procedures for adaptive inference in inverse problems for the white noise model , 2012, Probability Theory and Related Fields.

[60]  Kolyan Ray,et al.  Bayesian inverse problems with non-conjugate priors , 2012, 1209.6156.

[61]  Andrew M. Stuart,et al.  Bayesian posterior contraction rates for linear severely ill-posed inverse problems , 2012, 1210.1563.

[62]  Detlef Dürr,et al.  The Onsager-Machlup function as Lagrangian for the most probable path of a diffusion process , 1978 .

[63]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[64]  Tanja Tarvainen,et al.  Image reconstruction with uncertainty quantification in photoacoustic tomography. , 2016, The Journal of the Acoustical Society of America.

[65]  E. Candès,et al.  Curvelets: A Surprisingly Effective Nonadaptive Representation for Objects with Edges , 2000 .

[66]  D. Dobson,et al.  An image-enhancement technique for electrical impedance tomography , 1994 .

[67]  E. Somersalo,et al.  Statistical and computational inverse problems , 2004 .

[68]  E. Candes,et al.  11-magic : Recovery of sparse signals via convex programming , 2005 .

[69]  G. Kallianpur Stochastic differential equations and diffusion processes , 1981 .

[70]  Martin Burger,et al.  Maximum a posteriori probability estimates in infinite-dimensional Bayesian inverse problems , 2014, 1412.5816.