Approximating the likelihood in approximate Bayesian computation

This chapter will appear in the forthcoming Handbook of Approximate Bayesian Computation (2018). The conceptual and methodological framework that underpins approximate Bayesian computation (ABC) is targetted primarily towards problems in which the likelihood is either challenging or missing. ABC uses a simulation-based non-parametric estimate of the likelihood of a summary statistic and assumes that the generation of data from the model is computationally cheap. This chapter reviews two alternative approaches for estimating the intractable likelihood, with the goal of reducing the necessary model simulations to produce an approximate posterior. The first of these is a Bayesian version of the synthetic likelihood (SL), initially developed by Wood (2010), which uses a multivariate normal approximation to the summary statistic likelihood. Using the parametric approximation as opposed to the non-parametric approximation of ABC, it is possible to reduce the number of model simulations required. The second likelihood approximation method we consider in this chapter is based on the empirical likelihood (EL), which is a non-parametric technique and involves maximising a likelihood constructed empirically under a set of moment constraints. Mengersen et al (2013) adapt the EL framework so that it can be used to form an approximate posterior for problems where ABC can be applied, that is, for models with intractable likelihoods. However, unlike ABC and the Bayesian SL (BSL), the Bayesian EL (BCel) approach can be used to completely avoid model simulations in some cases. The BSL and BCel methods are illustrated on models of varying complexity.

[1]  J. N. K. Rao,et al.  Pseudo‐empirical likelihood ratio confidence intervals for complex surveys , 2006 .

[2]  Paul Fearnhead,et al.  Constructing Summary Statistics for Approximate Bayesian Computation: Semi-automatic ABC , 2010, 1004.1112.

[3]  Ian W. McKeague,et al.  Simultaneous confidence bands for ratios of survival functions via empirical likelihood , 2002 .

[4]  Brenda N. Vo,et al.  Quantifying uncertainty in parameter estimates for stochastic models of collective cell spreading using approximate Bayesian computation. , 2015, Mathematical biosciences.

[5]  Anthony N. Pettitt,et al.  Melanoma Cell Colony Expansion Parameters Revealed by Approximate Bayesian Computation , 2015, PLoS Comput. Biol..

[6]  Simon N. Wood.  Numerical maximum likelihood estimation , 2015 .

[7]  M. Blum.  Approximate Bayesian Computation: A Nonparametric Perspective , 2009, 0904.0635.

[8]  C C Drovandi,et al.  Estimation of Parameters for Macroparasite Population Evolution Using Approximate Bayesian Computation , 2011, Biometrics.

[9]  J. N. K. Rao,et al.  University of Waterloo Department of Statistics and Actuarial Science Bayesian Pseudo Empirical Likelihood Intervals for Complex Surveys Bayesian Pseudo Empirical Likelihood Intervals for Complex Surveys , 2022 .

[10]  Bing-Yi Jing,et al.  Empirical Likelihood for a Class of Functionals of Survival Distribution with Censored Data , 2001 .

[11]  David T. Frazier,et al.  Bayesian Synthetic Likelihood , 2017, 2305.05120.

[12]  L. Hansen,et al.  Finite Sample Properties of Some Alternative Gmm Estimators , 2015 .

[13]  Susanne M. Schennach,et al.  Bayesian exponentially tilted empirical likelihood , 2005 .

[14]  S. Wood.  Statistical inference for noisy nonlinear ecological dynamic systems , 2010, Nature.

[15]  Rafael Schmidt,et al.  Non‐parametric Estimation of Tail Dependence , 2006 .

[16]  Jean-Marie Cornuet,et al.  Adaptive Multiple Importance Sampling , 2009, 0907.1254.

[17]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[18]  Giuseppe Ragusa,et al.  Bayesian Likelihoods for Moment Condition Models , 2007 .

[19]  Song Xi Chen,et al.  Empirical Likelihood Confidence Intervals for Linear Regression Coefficients , 1994 .

[20]  Max Welling,et al.  GPS-ABC: Gaussian Process Surrogate Approximate Bayesian Computation , 2014, UAI.

[21]  Thomas J. DiCiccio,et al.  Empirical Likelihood is Bartlett-Correctable , 1991 .

[22]  Malay Ghosh,et al.  Empirical likelihood for small area estimation , 2011 .

[23]  B. Efron.  Nonparametric standard errors and confidence intervals , 1981 .

[24]  Kosuke Imai,et al.  Survey Sampling , 1998, Nov/Dec 2017.

[25]  Gareth W. Peters,et al.  Bayesian Inference, Monte Carlo Sampling and Operational Risk. , 2006 .

[26]  R. Kiss,et al.  Can anti‐migratory drugs be screened in vitro? A review of 2D and 3D assays for the quantitative analysis of cell migration , 2007, Medicinal research reviews.

[27]  A. Seheult,et al.  Pressure Matching for Hydrocarbon Reservoirs: A Case Study in the Use of Bayes Linear Strategies for Large Computer Experiments , 1997 .

[28]  Christopher C. Drovandi,et al.  Pre-processing for approximate Bayesian computation in image analysis , 2015, Stat. Comput..

[29]  Song Xi Chen,et al.  On the accuracy of empirical likelihood confidence regions for linear regression model , 1993 .

[30]  Anthony A. Smith,et al.  Estimating Nonlinear Time-Series Models Using Simulated Vector Autoregressions , 1993 .

[31]  Kerry A Landman,et al.  Multi-scale modeling of a wound-healing cell migration assay. , 2007, Journal of theoretical biology.

[32]  Eric Gilleland,et al.  Empirical Bayes estimation for the conditional extreme value model , 2014 .

[33]  Jonathan A. Tawn,et al.  A conditional approach for multivariate extreme values (with discussion) , 2004 .

[34]  M. Smith.  Bayesian Approaches to Copula Modelling , 2011 .

[35]  George Judge,et al.  Asymptotic equivalence of empirical likelihood and Bayesian MAP , 2009, 0908.3397.

[36]  Ingram Olkin,et al.  Unbiased Estimation of Some Multivariate Probability Densities and Related Functions , 1969 .

[37]  Mai Zhou.  Empirical Likelihood Method in Survival Analysis , 2015 .

[38]  Christopher K. Wikle,et al.  Bayesian semiparametric hierarchical empirical likelihood spatial models , 2014, 1405.3880.

[39]  Richard Wilkinson,et al.  Accelerating ABC methods using Gaussian processes , 2014, AISTATS.

[40]  Joseph Fourier,et al.  Approximate Bayesian Computation: a non-parametric perspective , 2013 .

[41]  Ingrid Van Keilegom,et al.  A goodness-of-fit test for parametric and semi-parametric models in multiresponse regression , 2010, 1001.1667.

[42]  Laura Ventura,et al.  Bayesian composite marginal likelihoods , 2011 .

[43]  Raul Cano.  On The Bayesian Bootstrap , 1992 .

[44]  Ian W McKeague,et al.  Empirical likelihood-based tests for stochastic ordering. , 2013, Bernoulli : official journal of the Bernoulli Society for Mathematical Statistics and Probability.

[45]  Friedrich Schmid,et al.  Multivariate Extensions of Spearman's Rho and Related Statistics , 2007 .

[46]  Ingrid Van Keilegom,et al.  A review on empirical likelihood methods for regression , 2009 .

[47]  D. Clayton.  A model for association in bivariate life tables and its application in epidemiological studies of familial tendency in chronic disease incidence , 1978 .

[48]  Aaron T. Porter,et al.  Multivariate spatial hierarchical Bayesian empirical likelihood methods for small area estimation , 2015 .

[49]  Richard G. Everitt,et al.  Bayesian model comparison with un-normalised likelihoods , 2015, Stat. Comput..

[50]  Till Großmaß.  Copulae and tail dependence , 2007 .

[51]  A. Owen.  Empirical likelihood ratio confidence intervals for a single functional , 1988 .

[52]  Art B. Owen,et al.  Empirical Likelihood for Linear Models , 1991 .

[53]  Ruth E. Baker,et al.  Multiple types of data are required to identify the mechanisms influencing the spatial expansion of melanoma cell colonies , 2013, BMC Systems Biology.

[54]  C. De Michele,et al.  On the Use of Copulas in Hydrology: Theory and Practice , 2007 .

[55]  Jianqing Fan,et al.  Local polynomial modelling and its applications , 1994 .

[56]  J. Rao,et al.  A NEW ESTIMATION THEORY FOR SAMPLE SURVEYS , 2013 .

[57]  W. Härdle.  Applied Nonparametric Regression , 1992 .

[58]  David Allingham,et al.  Bayesian estimation of quantile distributions , 2009, Stat. Comput..

[59]  C. Andrieu,et al.  The pseudo-marginal approach for efficient Monte Carlo computations , 2009, 0903.5480.

[60]  N. Lazar.  Bayesian empirical likelihood , 2003 .

[61]  Peter Hall,et al.  Methodology and algorithms of empirical likelihood , 1990 .

[62]  Andreas Huth,et al.  Technical Note: Approximate Bayesian parameterization of a process-based tropical forest model , 2014, 1401.8205.

[63]  Erlis Ruli,et al.  Approximate Bayesian computation with composite score functions , 2013, Stat. Comput..

[64]  David J. Nott,et al.  A note on approximating ABC‐MCMC using flexible classifiers , 2014 .

[65]  A. .,et al.  SEMIPARAMETRIC LIKELIHOOD RATIO INFERENCE , 1996 .

[66]  W. Härdle.  Applied Nonparametric Regression , 1991 .

[67]  Christopher C. Drovandi,et al.  Variational Bayes with synthetic likelihood , 2016, Statistics and Computing.

[68]  Xiaotong Shen,et al.  Empirical Likelihood , 2002 .

[69]  Anthony N. Pettitt,et al.  Bayesian indirect inference using a parametric auxiliary model , 2015, 1505.03372.

[70]  Yuichi Kitamura,et al.  Empirical Likelihood Methods in Econometrics: Theory and Practice , 2006 .

[71]  Bing-Yi Jing,et al.  EMPIRICAL LIKELIHOOD FOR COX REGRESSION MODEL UNDER RANDOM CENSORSHIP , 2001 .

[72]  E. Luciano,et al.  Copula methods in finance , 2004 .

[73]  Ross McVinish,et al.  Improving ABC for quantile distributions , 2012, Stat. Comput..

[74]  George G. Judge,et al.  A Bayesian Large Deviations Probabilistic Interpretation and Justification of Empirical Likelihood , 2007 .

[75]  Jing Qin,et al.  Empirical likelihood-based confidence intervals for data with possible zero observations , 2003 .

[76]  D. L. Sean McElwain,et al.  Interpreting scratch assays using pair density dynamics and approximate Bayesian computation , 2014, Open Biology.

[77]  J. Monahan,et al.  Proper likelihoods for Bayesian analysis , 1992 .

[78]  M. Li,et al.  Particle Markov chain Monte Carlo methods , 2015 .

[79]  Jiti Gao,et al.  An adaptive empirical likelihood test for parametric time series regression models , 2007 .

[80]  Hengjian Cui,et al.  On Bartlett correction of empirical likelihood in the presence of nuisance parameters , 2006 .

[81]  Ian W. McKeague,et al.  Testing for uniform stochastic ordering via empirical likelihood , 2016 .

[82]  Christopher C. Drovandi,et al.  Likelihood-free inference in high dimensions with synthetic likelihood , 2018, Comput. Stat. Data Anal..

[83]  Alexander J. McNeil,et al.  Multivariate Archimedean copulas, $d$-monotone functions and $\ell_1$-norm symmetric distributions , 2009, 0908.3750.

[84]  Francesco Pauli,et al.  Bayesian inference with a pairwise likelihood: an approach based on empirical likelihood , 2010 .

[85]  J. Lawless,et al.  Empirical Likelihood and General Estimating Equations , 1994 .

[86]  Paul Rochet,et al.  Bayesian interpretation of Generalized empirical likelihood by maximum entropy , 2011, 1202.6469.

[87]  Pejman Rohani,et al.  Neutrality, Cross-Immunity and Subtype Dominance in Avian Influenza Viruses , 2014, PloS one.

[88]  J. N. K. Rao,et al.  Bayesian pseudo-empirical-likelihood intervals for complex surveys , 2010 .

[89]  I. McKeague,et al.  Empirical likelihood based hypothesis testing , 2003 .

[90]  Matteo Fasiolo,et al.  A comparison of inferential methods for highly nonlinear state space models in ecology and epidemiology , 2014 .

[91]  Dennis Prangle,et al.  Summary statistics and sequential methods for approximate Bayesian computation , 2011 .

[92]  G. D. Rayner,et al.  Numerical maximum likelihood estimation for the g-and-k and generalized g-and-h distributions , 2002, Stat. Comput..

[93]  Randy R. Sitter,et al.  A PSEUDO EMPIRICAL LIKELIHOOD APPROACH TO THE EFFECTIVE USE OF AUXILIARY INFORMATION IN COMPLEX SURVEYS , 1999 .

[94]  Christopher C. Drovandi,et al.  Accelerating Bayesian Synthetic Likelihood With the Graphical Lasso , 2019, Journal of Computational and Graphical Statistics.

[95]  Heng Lian,et al.  Empirical likelihood confidence intervals for nonparametric functional data analysis , 2009, 0904.0843.

[96]  Yichuan Zhao,et al.  Weighted empirical likelihood estimates and their robustness properties , 2007, Comput. Stat. Data Anal..

[97]  Christian P. Robert,et al.  Approximate Bayesian computation via empirical likelihood , 2012 .