Stability of doubly-intractable distributions

Doubly-intractable distributions appear naturally as posterior distributions in Bayesian inference frameworks whenever the likelihood contains a normalizing function $Z$. Having two such functions $Z$ and $\widetilde Z$ we provide estimates of the total variation and Wasserstein distance of the resulting posterior probability measures. As a consequence this leads to local Lipschitz continuity w.r.t. $Z$. In the more general framework of a random function $\widetilde Z$ we derive bounds on the expected total variation and expected Wasserstein distance. The applicability of the estimates is illustrated within the setting of two representative Monte Carlo recovery scenarios.

[1]  Robert J. Kunsch,et al.  Optimal confidence for Monte Carlo integration of smooth functions , 2018, Advances in Computational Mathematics.

[2]  Michael Habeck,et al.  Bayesian approach to inverse statistical mechanics. , 2014, Physical review. E, Statistical, nonlinear, and soft matter physics.

[3]  Christian P. Robert,et al.  Bayesian computation for statistical models with intractable normalizing constants , 2008, 0804.3152.

[4]  Faming Liang,et al.  An Adaptive Exchange Algorithm for Sampling From Distributions With Intractable Normalizing Constants , 2016 .

[5]  C. Villani.  Optimal Transport: Old and New , 2008 .

[6]  D. Rudolf,et al.  Perturbation bounds for Monte Carlo within Metropolis via restricted approximations , 2018, Stochastic processes and their applications.

[7]  Jaewoo Park,et al.  Bayesian Inference in the Presence of Intractable Normalizing Functions , 2017, Journal of the American Statistical Association.

[8]  D. Hunter,et al.  Inference in Curved Exponential Family Models for Networks , 2006 .

[9]  Pierre Alquier,et al.  Noisy Monte Carlo: convergence of Markov chains with approximate transition kernels , 2014, Statistics and Computing.

[10]  D. Rudolf,et al.  Perturbation theory for Markov chains via Wasserstein distance , 2015, Bernoulli.

[11]  Yoav Zemel,et al.  Statistical Aspects of Wasserstein Distances , 2018, Annual Review of Statistics and Its Application.

[12]  Zoubin Ghahramani,et al.  MCMC for Doubly-intractable Distributions , 2006, UAI.

[13]  T. J. Sullivan,et al.  Error bounds for some approximate posterior measures in Bayesian inference , 2019, ArXiv.

[14]  J. Møller,et al.  An efficient Markov chain Monte Carlo method for distributions with intractable normalising constants , 2006 .

[15]  Vladimir Kolmogorov,et al.  A Faster Approximation Algorithm for the Gibbs Partition Function , 2016, COLT.

[16]  A. Stuart,et al.  The Bayesian Approach to Inverse Problems , 2013, 1302.6989.

[17]  Erich Novak,et al.  Solvable integration problems and optimal sample size selection , 2018, J. Complex..

[18]  Andrew M. Stuart,et al.  Inverse problems: A Bayesian perspective , 2010, Acta Numerica.

[19]  Björn Sprungk.  On the local Lipschitz stability of Bayesian inverse problems , 2019 .

[20]  Aretha L. Teckentrup,et al.  Random forward models and log-likelihoods in Bayesian inverse problems , 2017, SIAM/ASA J. Uncertain. Quantification.

[21]  Jonas Latz,et al.  On the Well-posedness of Bayesian Inverse Problems , 2019, SIAM/ASA J. Uncertain. Quantification.