Robust Bayesian synthetic likelihood via a semi-parametric approach

Bayesian synthetic likelihood (BSL) is now a well-established method for performing approximate Bayesian parameter estimation for simulation-based models that do not possess a tractable likelihood function. BSL approximates an intractable likelihood function of a carefully chosen summary statistic at a parameter value with a multivariate normal distribution. The mean and covariance matrix of this normal distribution are estimated from independent simulations of the model. Due to the parametric assumption implicit in BSL, it can be preferred to its nonparametric competitor, approximate Bayesian computation, in certain applications where a high-dimensional summary statistic is of interest. However, despite several successful applications of BSL, its widespread use in scientific fields may be hindered by the strong normality assumption. In this paper, we develop a semi-parametric approach to relax this assumption to an extent and maintain the computational advantages of BSL without any additional tuning. We test our new method, semiBSL, on several challenging examples involving simulated and real data and demonstrate that semiBSL can be significantly more robust than BSL and another approach in the literature.

[1]  Larry A. Wasserman,et al.  The Nonparanormal: Semiparametric Estimation of High Dimensional Undirected Graphs , 2009, J. Mach. Learn. Res..

[2]  Scott A. Sisson,et al.  Extending approximate Bayesian computation methods to high dimensions via a Gaussian copula model , 2015, 1504.04093.

[3]  David T. Frazier,et al.  Bayesian Synthetic Likelihood , 2017, 2305.05120.

[4]  Christopher C. Drovandi,et al.  Likelihood-free inference in high dimensions with synthetic likelihood , 2018, Comput. Stat. Data Anal..

[5]  Yanan Fan,et al.  Handbook of Approximate Bayesian Computation , 2018 .

[6]  Paul Fearnhead,et al.  Constructing summary statistics for approximate Bayesian computation: semi‐automatic approximate Bayesian computation , 2012 .

[7]  David M. Green,et al.  A stochastic movement model reproduces patterns of site fidelity and long-distance dispersal in a population of Fowler’s toads (Anaxyrus fowleri) , 2017 .

[8]  Radford M. Neal,et al.  On Bayesian inference for the M/G/1 queue with efficient MCMC sampling , 2014, 1401.5548.

[9]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[10]  C C Drovandi,et al.  Estimation of Parameters for Macroparasite Population Evolution Using Approximate Bayesian Computation , 2011, Biometrics.

[11]  David I. Warton,et al.  Penalized Normal Likelihood and Ridge Regularization of Correlation and Covariance Matrices , 2008 .

[12]  C. Andrieu,et al.  The pseudo-marginal approach for efficient Monte Carlo computations , 2009, 0903.5480.

[13]  T. Bedford,et al.  Vines: A new graphical model for dependent random variables , 2002 .

[14]  Jukka Corander,et al.  Likelihood-Free Inference by Ratio Estimation , 2016, Bayesian Analysis.

[15]  Christopher C. Drovandi,et al.  Accelerating Bayesian Synthetic Likelihood With the Graphical Lasso , 2019, Journal of Computational and Graphical Statistics.

[16]  Christopher C. Drovandi,et al.  Variational Bayes with synthetic likelihood , 2016, Statistics and Computing.

[17]  M. Hazelton.  Variable kernel density estimation , 2003 .

[18]  Olivier François,et al.  Non-linear regression models for Approximate Bayesian Computation , 2008, Stat. Comput..

[19]  Florian Hartig,et al.  An Extended Empirical Saddlepoint Approximation for Intractable Likelihoods , 2016, 1601.01849.

[20]  S. Sahu,et al.  A new class of multivariate skew distributions with applications to bayesian regression models , 2003 .

[21]  A. Bowman,et al.  Applied smoothing techniques for data analysis : the kernel approach with S-plus illustrations , 1999 .

[22]  M. C. Jones,et al.  Sinh-arcsinh distributions , 2009 .

[23]  S. Wood.  Statistical inference for noisy nonlinear ecological dynamic systems , 2010, Nature.

[24]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[25]  Christophe Croux,et al.  The Gaussian rank correlation estimator: robustness properties , 2010, Statistics and Computing.

[26]  Richard G. Everitt.  Bootstrapped synthetic likelihood , 2017, ArXiv.

[27]  Stuart Coles,et al.  The Largest Inclusions in a Piece of Steel , 2002 .

[28]  Guillermo Rus-Carlborg,et al.  Approximate Bayesian Computation by Subset Simulation , 2014, SIAM J. Sci. Comput..

[29]  C. Mallows,et al.  A Method for Simulating Stable Random Variables , 1976 .

[30]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[31]  Corentin M Barbu,et al.  Two‐scale dispersal estimation for biological invasions via synthetic likelihood , 2018, Ecography.

[32]  Anthony N. Pettitt,et al.  Bayesian indirect inference using a parametric auxiliary model , 2015, 1505.03372.

[33]  M. Sklar.  Fonctions de repartition a n dimensions et leurs marges , 1959 .

[34]  S. Coles,et al.  Inference for Stereological Extremes , 2007 .

[35]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[36]  J. Møller.  Discussion on the paper by Feranhead and Prangle , 2012 .

[37]  V. A. Epanechnikov.  Non-Parametric Estimation of a Multivariate Probability Density , 1969 .

[38]  A. Pettitt,et al.  Approximate Bayesian computation using indirect inference , 2011 .