Cauchy Markov random field priors for Bayesian inversion

The use of Cauchy Markov random field priors in statistical inverse problems can potentially lead to posterior distributions which are non-Gaussian, high-dimensional, multimodal and heavy-tailed. In order to use such priors successfully, sophisticated optimization and Markov chain Monte Carlo (MCMC) methods are usually required. In this paper, our focus is largely on reviewing recently developed Cauchy difference priors, while introducing interesting new variants, whilst providing a comparison. We firstly propose a one-dimensional second order Cauchy difference prior, and construct new first and second order two-dimensional isotropic Cauchy difference priors. Another new Cauchy prior is based on the stochastic partial differential equation approach, derived from Matérn type Gaussian presentation. The comparison also includes Cauchy sheets. Our numerical computations are based on both maximum a posteriori and conditional mean estimation. We exploit state-of-the-art MCMC methodologies such as Metropolis-within-Gibbs, RepellingAttracting Metropolis, and No-U-Turn sampler variant of Hamiltonian Monte Carlo. We demonstrate the models and methods constructed for one-dimensional and two-dimensional deconvolution problems. Thorough MCMC statistics are provided for all test cases, including potential scale reduction factors. AMS subject classifications: 62F15, 60G52, 60J22, 65C40

[1]  Lassi Roininen,et al.  Cauchy difference priors for edge-preserving Bayesian inversion , 2016, Journal of Inverse and Ill-posed Problems.

[2]  P. Fearnhead,et al.  The Random Walk Metropolis: Linking Theory and Practice Through a Case Study , 2010, 1011.6217.

[3]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[4]  Xiao-Li Meng,et al.  A Repelling–Attracting Metropolis Algorithm for Multimodality , 2016, Journal of Computational and Graphical Statistics.

[5]  Andrew M. Stuart,et al.  How Deep Are Deep Gaussian Processes? , 2017, J. Mach. Learn. Res..

[6]  C. Geyer,et al.  Annealing Markov chain Monte Carlo with applications to ancestral inference , 1995 .

[7]  Ju A Rozanov MARKOV RANDOM FIELDS AND STOCHASTIC PARTIAL DIFFERENTIAL EQUATIONS , 1977 .

[8]  H. Haario,et al.  An adaptive Metropolis algorithm , 2001 .

[9]  F. S.,et al.  Necessary conditions for geometric and polynomial ergodicity of random walk-type Markov chains , 2001 .

[10]  Andrew Gelman,et al.  General methods for monitoring convergence of iterative simulations , 1998 .

[11]  Aaron Smith,et al.  Does Hamiltonian Monte Carlo mix faster than a random walk on multimodal densities? , 2018, ArXiv.

[12]  Andrew Gelman,et al.  The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo , 2011, J. Mach. Learn. Res..

[13]  Radford M. Neal Slice Sampling , 2003, The Annals of Statistics.

[14]  Heikki Haario,et al.  Componentwise adaptation for high dimensional MCMC , 2005, Comput. Stat..

[15]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[16]  Bamdad Hosseini,et al.  Well-Posed Bayesian Inverse Problems with Infinitely Divisible and Heavy-Tailed Prior Measures , 2016, SIAM/ASA J. Uncertain. Quantification.

[17]  É. Moulines,et al.  Irreducibility and geometric ergodicity of Hamiltonian Monte Carlo , 2020 .

[18]  Matti Lassas. Eero Saksman,et al.  Discretization-invariant Bayesian inversion and Besov space priors , 2009, 0901.4220.

[19]  竹中 茂夫 G.Samorodnitsky,M.S.Taqqu:Stable non-Gaussian Random Processes--Stochastic Models with Infinite Variance , 1996 .

[20]  M. Girolami,et al.  Hyperpriors for Matérn fields with applications in Bayesian inversion , 2016, Inverse Problems & Imaging.

[21]  A. Stuart,et al.  Besov priors for Bayesian inverse problems , 2011, 1105.0889.

[22]  V. Zolotarev Integral Transformations of Distributions and Estimates of Parameters of Multidimensional Spherically Symmetric Stable Laws , 1981 .

[23]  L. Roininen,et al.  A Bayesian-based approach to improving acoustic Born waveform inversion of seismic data for viscoelastic media , 2019, Inverse Problems.

[24]  Christian Musso,et al.  An insight into the issue of dimensionality in particle filtering , 2010, 2010 13th International Conference on Information Fusion.

[25]  G. Roberts,et al.  Kinetic energy choice in Hamiltonian/hybrid Monte Carlo , 2017, Biometrika.

[26]  A. V. D. Vaart,et al.  BAYESIAN INVERSE PROBLEMS WITH GAUSSIAN PRIORS , 2011, 1103.2692.

[27]  Neil D. Lawrence,et al.  Deep Gaussian Processes , 2012, AISTATS.

[28]  J. Møller,et al.  An efficient Markov chain Monte Carlo method for distributions with intractable normalising constants , 2006 .

[29]  C. Holmes,et al.  A framework for adaptive MCMC targeting multimodal distributions , 2018, The Annals of Statistics.

[30]  S. Siltanen,et al.  Can one use total variation prior for edge-preserving Bayesian inversion? , 2004 .

[31]  John Geweke,et al.  Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments , 1991 .

[32]  Simon R. Arridge,et al.  Solving inverse problems using data-driven models , 2019, Acta Numerica.

[33]  Michael Betancourt,et al.  A Conceptual Introduction to Hamiltonian Monte Carlo , 2017, 1701.02434.

[34]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[35]  Marco A. Iglesias,et al.  Parameterizations for ensemble Kalman inversion , 2017, 1709.01781.

[36]  John P. Nolan,et al.  Multivariate elliptically contoured stable distributions: theory and estimation , 2013, Computational Statistics.

[37]  Bamdad Hosseini,et al.  Well-Posed Bayesian Inverse Problems: Priors with Exponential Tails , 2016, SIAM/ASA J. Uncertain. Quantification.

[38]  W. Linde STABLE NON‐GAUSSIAN RANDOM PROCESSES: STOCHASTIC MODELS WITH INFINITE VARIANCE , 1996 .

[39]  Torben Sell DIMENSION-ROBUST FUNCTION SPACE MCMC WITH NEURAL NETWORK PRIORS , 2020 .

[40]  Ville Kolehmainen,et al.  Isotropic and anisotropic total variation regularization in electrical impedance tomography , 2017, Comput. Math. Appl..

[41]  S. Särkkä,et al.  Enhancing industrial X-ray tomography by data-centric statistical methods , 2020, Data-Centric Engineering.

[42]  Albert Tarantola,et al.  Inverse problem theory - and methods for model parameter estimation , 2004 .

[43]  Mónica F. Bugallo,et al.  A survey of Monte Carlo methods for parameter estimation , 2020, EURASIP J. Adv. Signal Process..

[44]  Geoffrey E. Hinton,et al.  Bayesian Learning for Neural Networks , 1995 .

[45]  Zoubin Ghahramani,et al.  Turing: A Language for Flexible Probabilistic Inference , 2018 .

[46]  Carlos E. Rodríguez,et al.  Searching for efficient Markov chain Monte Carlo proposal kernels , 2013, Proceedings of the National Academy of Sciences.

[47]  Patrick Kofod Mogensen,et al.  Optim: A mathematical optimization package for Julia , 2018, J. Open Source Softw..

[48]  Lassi Roininen,et al.  Whittle-Matérn priors for Bayesian statistical inversion with applications in electrical impedance tomography , 2014 .

[49]  H. Rue,et al.  An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach , 2011 .

[50]  D. Bolin,et al.  Geostatistical Modelling Using Non‐Gaussian Matérn Fields , 2015 .

[51]  Andrew M. Stuart,et al.  Inverse problems: A Bayesian perspective , 2010, Acta Numerica.

[52]  Marco A. Iglesias,et al.  Well-posed Bayesian geometric inverse problems arising in subsurface flow , 2014, 1401.5571.

[53]  T. J. Sullivan,et al.  Well-posed Bayesian inverse problems and heavy-tailed stable quasi-Banach space priors , 2016, 1605.05898.

[54]  Dhruv V. Patel,et al.  Bayesian Inference with Generative Adversarial Network Priors , 2019, ArXiv.

[55]  Mark Girolami,et al.  Statistical methods to enable practical on-site tomographic imaging of whole-core samples , 2018, GEOPHYSICS.

[56]  Luca Martino,et al.  Independent Doubly Adaptive Rejection Metropolis Sampling Within Gibbs Sampling , 2012, IEEE Transactions on Signal Processing.

[57]  Radford M. Neal MCMC Using Hamiltonian Dynamics , 2011, 1206.1901.