Split-BOLFI for for misspecification-robust likelihood free inference in high dimensions

Likelihood-free inference for simulator-based statistical models has recently grown rapidly from its infancy to a useful tool for practitioners. However, models with more than a very small number of parameters as the target of inference have remained an enigma, in particular for the approximate Bayesian computation (ABC) community. To advance the possibilities for performing likelihood-free inference in high-dimensional parameter spaces, here we introduce an extension of the popular Bayesian optimisation based approach to approximate discrepancy functions in a probabilistic manner which lends itself to an efficient exploration of the parameter space. Our method achieves computational scalability by using separate acquisition procedures for the discrepancies defined for different parameters. These efficient high-dimensional simulation acquisitions are combined with exponentiated loss-likelihoods to provide a misspecification-robust characterisation of the marginal posterior distribution for all model parameters. The method successfully performs computationally efficient inference in a 100-dimensional space on canonical examples and compares favourably to existing Copula-ABC methods. We further illustrate the potential of this approach by fitting a bacterial transmission dynamics model to daycare centre data, which provides biologically coherent results on the strain competition in a 30-dimensional parameter space.

[1]  A. Almudevar Nasopharyngeal Bacterial Interactions in Children , 2014, Emerging Infectious Diseases.

[2]  Christian P. Robert,et al.  Model Misspecification in ABC: Consequences and Diagnostics , 2017, 1708.01974.

[3]  A. N. Pettitt,et al.  Approximate Bayesian Computation for astronomical model analysis: a case study in galaxy demographics and morphological transformation at high redshift , 2012, 1202.1426.

[4]  Helmut Ltkepohl,et al.  New Introduction to Multiple Time Series Analysis , 2007 .

[5]  Jasper Snoek,et al.  Multi-Task Bayesian Optimization , 2013, NIPS.

[6]  Ritabrata Dutta,et al.  Likelihood-free inference via classification , 2014, Stat. Comput..

[7]  Jukka Corander,et al.  A Bayesian Approach to Modelling Graphical Vector Autoregressions , 2006 .

[8]  Aki Vehtari,et al.  Efficient Acquisition Rules for Model-Based Approximate Bayesian Computation , 2017, Bayesian Analysis.

[9]  Gilles Louppe,et al.  Likelihood-free MCMC with Approximate Likelihood Ratios , 2019, ArXiv.

[10]  Jukka Corander,et al.  Diagnosing model misspecification and performing generalized Bayes' updates via probabilistic classifiers , 2019, 1912.05810.

[11]  Jukka Corander,et al.  PYLFIRE: Python implementation of likelihood-free inference by ratio estimation , 2019, Wellcome open research.

[12]  Christopher C. Drovandi,et al.  Variational Bayes with synthetic likelihood , 2016, Statistics and Computing.

[13]  Tina Toni,et al.  The ABC of reverse engineering biological signalling systems. , 2009, Molecular bioSystems.

[14]  Scott A. Sisson,et al.  Extending approximate Bayesian computation methods to high dimensions via a Gaussian copula model , 2015, 1504.04093.

[15]  Geoff K. Nicholls,et al.  Calibration Procedures for Approximate Bayesian Credible Sets , 2018, Bayesian Analysis.

[16]  João André Carriço,et al.  High Rates of Transmission of and Colonization by Streptococcus pneumoniae and Haemophilus influenzae within a Day Care Center Revealed in a Longitudinal Study , 2007, Journal of Clinical Microbiology.

[17]  J. Hinds,et al.  Impact of the 13-valent pneumococcal conjugate vaccine on Streptococcus pneumoniae multiple serotype carriage. , 2016, Vaccine.

[18]  C C Drovandi,et al.  Estimation of Parameters for Macroparasite Population Evolution Using Approximate Bayesian Computation , 2011, Biometrics.

[19]  Wouter A. A. de Steenhuijsen Piters,et al.  The microbiota of the respiratory tract: gatekeeper to respiratory health , 2017, Nature Reviews Microbiology.

[20]  M. Gutmann,et al.  Frequency-dependent selection in vaccine-associated pneumococcal population dynamics , 2017, Nature Ecology & Evolution.

[21]  Kirthevasan Kandasamy,et al.  High Dimensional Bayesian Optimisation and Bandits via Additive Models , 2015, ICML.

[22]  Adam Powell,et al.  The Origins of Lactase Persistence in Europe , 2009, PLoS Comput. Biol..

[23]  Jukka Corander,et al.  Likelihood-Free Inference by Ratio Estimation , 2016, Bayesian Analysis.

[24]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.

[25]  David T. Frazier,et al.  Bayesian Synthetic Likelihood , 2017, 2305.05120.

[26]  Yanan Fan,et al.  Handbook of Approximate Bayesian Computation , 2018 .

[27]  Pekka Marttinen,et al.  A Bayesian model of acquisition and clearance of bacterial colonization incorporating within-host variation , 2018, bioRxiv.

[28]  Jean-Michel Marin,et al.  Approximate Bayesian computational methods , 2011, Statistics and Computing.

[29]  Anthony N. Pettitt,et al.  Bayesian indirect inference using a parametric auxiliary model , 2015, 1505.03372.

[30]  Mats Gyllenberg,et al.  Estimating the Transmission Dynamics of Streptococcus pneumoniae from Strain Prevalence Data , 2013, Biometrics.

[31]  Michael A. Osborne,et al.  Probabilistic numerics and uncertainty in computations , 2015, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[32]  D. J. Nott,et al.  Approximate Bayesian computation via regression density estimation , 2012, 1212.1479.

[33]  Gilles Louppe,et al.  Likelihood-free MCMC with Amortized Approximate Ratio Estimators , 2019, ICML.

[34]  Shakir Mohamed,et al.  Learning in Implicit Generative Models , 2016, ArXiv.

[35]  S. Wood Statistical inference for noisy nonlinear ecological dynamic systems , 2010, Nature.

[36]  Peter Grünwald,et al.  The Safe Bayesian - Learning the Learning Rate via the Mixability Gap , 2012, ALT.

[37]  Aki Vehtari,et al.  ELFI: Engine for Likelihood Free Inference , 2016, J. Mach. Learn. Res..

[38]  Michael U. Gutmann,et al.  Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Models , 2015, J. Mach. Learn. Res..

[39]  D. Barge,et al.  Machine learning accelerated likelihood-free event reconstruction in dark matter direct detection , 2018, Journal of Instrumentation.

[40]  Pier Giovanni Bissiri,et al.  A general framework for updating belief distributions , 2013, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[41]  Yoshua Bengio,et al.  Generative Adversarial Networks , 2014, ArXiv.

[42]  M. Gutmann,et al.  Fundamentals and Recent Developments in Approximate Bayesian Computation , 2016, Systematic biology.

[43]  M. Söderström,et al.  Bacterial interactions in the nasopharynx - effects of host factors in children attending day-care centers. , 2012, Journal of infection and public health.

[44]  M. Gomes,et al.  How direct competition shapes coexistence and vaccine effects in multi-strain pathogen systems. , 2016, Journal of theoretical biology.

[45]  Sam P. Brown,et al.  Pneumococcal quorum sensing drives an asymmetric owner–intruder competitive strategy during carriage via the competence regulon , 2018, Nature Microbiology.

[46]  Arnaud Doucet,et al.  An adaptive sequential Monte Carlo method for approximate Bayesian computation , 2011, Statistics and Computing.

[47]  Gilles Louppe,et al.  Approximating Likelihood Ratios with Calibrated Discriminative Classifiers , 2015, 1506.02169.

[48]  Christopher C. Drovandi,et al.  Likelihood-free inference in high dimensions with synthetic likelihood , 2018, Comput. Stat. Data Anal..

[49]  Iain Murray,et al.  Fast $\epsilon$-free Inference of Simulation Models with Bayesian Conditional Density Estimation , 2016, 1605.06376.

[50]  Iain Murray,et al.  Sequential Neural Likelihood: Fast Likelihood-free Inference with Autoregressive Flows , 2018, AISTATS.

[51]  James Ridgway,et al.  Probably approximate Bayesian computation: nonasymptotic convergence of ABC under misspecification , 2017, ArXiv.

[52]  L. Van Melderen,et al.  Differences in nasopharyngeal bacterial carriage in preschool children from different socio-economic origins. , 2011, Clinical microbiology and infection : the official publication of the European Society of Clinical Microbiology and Infectious Diseases.

[53]  Stephan R. Sain,et al.  Fast Sequential Computer Model Calibration of Large Nonstationary Spatial-Temporal Processes , 2013, Technometrics.