Improving Approximate Bayesian Computation via Quasi-Monte Carlo

ABSTRACT ABC (approximate Bayesian computation) is a general approach for dealing with models with an intractable likelihood. In this work, we derive ABC algorithms based on QMC (quasi-Monte Carlo) sequences. We show that the resulting ABC estimates have a lower variance than their Monte Carlo counter-parts. We also develop QMC variants of sequential ABC algorithms, which progressively adapt the proposal distribution and the acceptance threshold. We illustrate our QMC approach through several examples taken from the ABC literature.

[1]  N. Chopin,et al.  Sequential Quasi-Monte Carlo , 2014, 1402.4039.

[2]  Aaron Smith,et al.  The use of a single pseudo-sample in approximate Bayesian computation , 2014, Stat. Comput..

[3]  Paul Glasserman,et al.  Monte Carlo Methods in Financial Engineering , 2003 .

[4]  C. Robert,et al.  ABC random forests for Bayesian parameter inference , 2016, Bioinform..

[5]  Paul Fearnhead,et al.  Constructing Summary Statistics for Approximate Bayesian Computation: Semi-automatic ABC , 2010, 1004.1112.

[6]  P. Donnelly,et al.  Inferring coalescence times from DNA sequence data. , 1997, Genetics.

[7]  Mark M. Tanaka,et al.  Sequential Monte Carlo without likelihoods , 2007, Proceedings of the National Academy of Sciences.

[8]  Andrew M. Stuart,et al.  Importance Sampling: Computational Complexity and Intrinsic Dimension , 2015 .

[9]  Paul Marjoram,et al.  Markov chain Monte Carlo without likelihoods , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[10]  P. L’Ecuyer,et al.  Randomized quasi-Monte Carlo: An introduction for practitioners , 2016 .

[11]  Arnaud Doucet,et al.  An adaptive sequential Monte Carlo method for approximate Bayesian computation , 2011, Statistics and Computing.

[12]  C. Lemieux.  Monte Carlo and Quasi-Monte Carlo Sampling , 2009 .

[13]  Iain Murray,et al.  Fast $\epsilon$-free Inference of Simulation Models with Bayesian Conditional Density Estimation , 2016, 1605.06376.

[14]  David Welch,et al.  Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems , 2009, Journal of The Royal Society Interface.

[15]  A. W. Kemp,et al.  Univariate Discrete Distributions: Johnson/Univariate Discrete Distributions , 2005 .

[16]  Jean-Michel Marin,et al.  Approximate Bayesian computational methods , 2011, Statistics and Computing.

[17]  H. Niederreiter.  Quasi-Monte Carlo methods and pseudo-random numbers , 1978 .

[18]  Anthony Lee,et al.  On the choice of MCMC kernels for approximate Bayesian computation with SMC samplers , 2012, Proceedings Title: Proceedings of the 2012 Winter Simulation Conference (WSC).

[19]  Richard Wilkinson,et al.  Accelerating ABC methods using Gaussian processes , 2014, AISTATS.

[20]  Jun S. Liu,et al.  Sequential Imputations and Bayesian Missing Data Problems , 1994 .

[21]  Anthony Lee,et al.  Variance bounding and geometric ergodicity of Markov chain Monte Carlo kernels for approximate Bayesian computation , 2012, 1210.6703.

[22]  A. W. Kemp,et al.  Univariate Discrete Distributions , 1993 .

[23]  M. Blum.  Approximate Bayesian Computation: A Nonparametric Perspective , 2009, 0904.0635.

[24]  Christian P. Robert,et al.  On parameter estimation with the Wasserstein distance , 2017, Information and Inference: A Journal of the IMA.

[25]  Lauwerens Kuipers,et al.  Uniform distribution of sequences , 1974 .

[26]  Michael U. Gutmann,et al.  Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Models , 2015, J. Mach. Learn. Res..

[27]  G. Leobacher,et al.  Introduction to Quasi-Monte Carlo Integration and Applications , 2014 .

[28]  Frances Y. Kuo,et al.  High-dimensional integration: The quasi-Monte Carlo way*† , 2013, Acta Numerica.

[29]  Bruno Tuffin,et al.  A central limit theorem and improved error bounds for a hybrid-Monte Carlo sequence with applications in computational finance , 2006, J. Complex..

[30]  F. J. Hickernell.  Koksma–Hlawka Inequality , 2006 .

[31]  Aurélien Garivier,et al.  On the Complexity of Best-Arm Identification in Multi-Armed Bandit Models , 2014, J. Mach. Learn. Res..

[32]  J. Møller.  Discussion on the paper by Feranhead and Prangle , 2012 .

[33]  M. Gutmann,et al.  Fundamentals and Recent Developments in Approximate Bayesian Computation , 2016, Systematic biology.

[34]  Jean-Michel Marin,et al.  ABC random forests for Bayesian parameter inference , 2019, Bioinform..

[35]  Nicolas Chopin,et al.  Expectation Propagation for Likelihood-Free Inference , 2011, 1107.5959.

[36]  Mathieu Gerber.  On integration methods based on scrambled nets of arbitrary size , 2015, J. Complex..

[37]  David M. Blei,et al.  Variational Inference: A Review for Statisticians , 2016, ArXiv.

[38]  C. Robert,et al.  Inference in generative models using the Wasserstein distance , 2017, 1701.05146.

[39]  Jean-Marie Cornuet,et al.  Efficient learning in ABC algorithms , 2012, 1210.1388.

[40]  A. Owen.  Scrambled net variance for integrals of smooth functions , 1997 .

[41]  F. Pillichshammer,et al.  Digital Nets and Sequences: Discrepancy Theory and Quasi-Monte Carlo Integration , 2010 .

[42]  A. Owen.  Local antithetic sampling with scrambled nets , 2008, 0811.0528.

[43]  Art B. Owen,et al.  Monte Carlo extension of quasi-Monte Carlo , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).

[44]  O. Papaspiliopoulos,et al.  Importance Sampling: Intrinsic Dimension and Computational Cost , 2015, 1511.06196.

[45]  Andrew R. Francis,et al.  Using Approximate Bayesian Computation to Estimate Tuberculosis Transmission Parameters From Genotype Data , 2006, Genetics.

[46]  C. Andrieu,et al.  The pseudo-marginal approach for efficient Monte Carlo computations , 2009, 0903.5480.