A practical guide to non-parametric approximate Bayesian computation

7 A critical task in modelling is to determine how well the theoretical assumptions encoded in 8 a model account for observations. Bayesian methods are an ideal framework for doing just this. 9 Existing approximate Bayesian computation (ABC) methods however rely on often insufficient 10 “summary statistics”. Here, I present and analyze a highly efficient extension of a recently pro11 posed (Turner et al., 2014) non-parametric approximate Bayesian computation (npABC) algorithm, 12 which circumvents this insufficiency. This method combines Markov Chain Monte Carlo simulation 13 with tools from non-parametric statistics to improve upon existing ABC methods. The primary 14 contributions of this article: 1) A more efficient implementation of this method is described, that 15 substantially improves computational performance and chain mixing. 2) Theoretical results describ16 ing the influence of methodological approximation errors on posterior estimation are discussed. In 17 particular, while this method is highly accurate, even small errors have a strong influence on model 18 comparisons when using standard statistical approaches (such as deviance information criterion). 19 Thus care must be taken when using this (or any other ABC) method for model comparison. 3) An 20 augmentation of the standard MCMC procedure, termed “Resampled MCMC”, that reduces the 21 negative influence of approximation errors on performance and accuracy, is presented. 4) In order 22 to make this method accessible to a broader audience, a number of examples of varying complexity 23 are presented along with supplementary code for their implementation. 24

[1]  Brandon M. Turner,et al.  A method for efficiently sampling from distributions with correlated dimensions. , 2013, Psychological methods.

[2]  George B. Dantzig,et al.  Linear Programming 1: Introduction , 1997 .

[3]  Jukka Corander,et al.  Approximate Bayesian Computation , 2013, PLoS Comput. Biol..

[4]  Bryan E. Shepherd,et al.  Global Sensitivity Analysis. The Primer by SALTELLI, A., RATTO, M., ANDRES, T., CAMPOLONGO, F., CARIBONI, J., GATELLI, D., SAISANA, M., and TARANTOLA, S , 2009 .

[5]  Timothy D. Hanks,et al.  Bounded Integration in Parietal Cortex Underlies Decisions Even When Viewing Duration Is Dictated by the Environment , 2008, The Journal of Neuroscience.

[6]  C. Gallistel,et al.  The Importance of Proving the Null , 2022 .

[7]  R. Ratcliff,et al.  Estimating parameters of the diffusion model: Approaches to dealing with contaminant reaction times and parameter variability , 2002, Psychonomic bulletin & review.

[8]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[9]  Brandon M. Turner,et al.  A generalized, likelihood-free method for posterior estimation , 2014, Psychonomic bulletin & review.

[10]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[11]  V. A. Epanechnikov Non-Parametric Estimation of a Multivariate Probability Density , 1969 .

[12]  A. Heathcote,et al.  Reply to Speckman and Rouder: A theoretical basis for QML , 2004 .

[13]  O. Cappé,et al.  Population Monte Carlo , 2004 .

[14]  James L. McClelland,et al.  The time course of perceptual choice: the leaky, competing accumulator model. , 2001, Psychological review.

[15]  B. Silverman,et al.  Kernel Density Estimation Using the Fast Fourier Transform , 1982 .

[16]  B. Silverman,et al.  Algorithm AS 176: Kernel Density Estimation Using the Fast Fourier Transform , 1982 .

[17]  Scott D. Brown,et al.  Quantile maximum likelihood estimation of response time distributions , 2002, Psychonomic bulletin & review.

[18]  Bruce Tidor,et al.  Sloppy models, parameter uncertainty, and the role of experimental design. , 2010, Molecular bioSystems.

[19]  Xiao-Jing Wang,et al.  Erratum to: Effects of neuromodulation in a cortical network model of object working memory dominated by recurrent inhibition , 2014, Journal of Computational Neuroscience.

[20]  J BraakCajo A Markov Chain Monte Carlo version of the genetic algorithm Differential Evolution , 2006 .

[21]  Francis Tuerlinckx,et al.  Fitting the ratcliff diffusion model to experimental data , 2007, Psychonomic bulletin & review.

[22]  Andrew Heathcote,et al.  Brain and Behavior in Decision-Making , 2014, PLoS Comput. Biol..

[23]  Scott D. Brown,et al.  The simplest complete model of choice response time: Linear ballistic accumulation , 2008, Cognitive Psychology.

[24]  M. Lee Three case studies in the Bayesian analysis of cognitive models , 2008, Psychonomic bulletin & review.

[25]  Marius Usher,et al.  Using Time-Varying Evidence to Test Models of Decision Dynamics: Bounded Diffusion vs. the Leaky Competing Accumulator Model , 2012, Front. Neurosci..

[26]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[27]  D. Meyer,et al.  Analyses of multinomial mixture distributions: new tests for stochastic models of cognition and action. , 1991, Psychological bulletin.

[28]  C. D. Kemp,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[29]  Andrew Heathcote,et al.  A ballistic model of choice response time. , 2005, Psychological review.

[30]  John I. Yellott,et al.  Correction for guessing in choice reaction time , 1967 .

[31]  R. Ollman Fast guesses in choice reaction time , 1966 .

[32]  Gustavo Deco,et al.  The encoding of alternatives in multiple-choice decision making , 2009, Proceedings of the National Academy of Sciences.

[33]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[34]  Cajo J. F. ter Braak,et al.  A Markov Chain Monte Carlo version of the genetic algorithm Differential Evolution: easy Bayesian computing for real parameter spaces , 2006, Stat. Comput..

[35]  M. Shadlen,et al.  Neural Activity in Macaque Parietal Cortex Reflects Temporal Integration of Visual Motion Signals during Perceptual Decision Making , 2005, The Journal of Neuroscience.

[36]  R. Abelson Statistics As Principled Argument , 1995 .

[37]  Christian P Robert,et al.  Lack of confidence in approximate Bayesian computation model choice , 2011, Proceedings of the National Academy of Sciences.

[38]  Jeffrey N. Rouder,et al.  Modeling Response Times for Two-Choice Decisions , 1998 .

[39]  Jacob Marschak,et al.  Lack of Confidence , 1974 .

[40]  Brandon M. Turner,et al.  Journal of Mathematical Psychology a Tutorial on Approximate Bayesian Computation , 2022 .

[41]  K. Friedrichs The identity of weak and strong extensions of differential operators , 1944 .

[42]  M. Lee,et al.  Bayesian Cognitive Modeling: A Practical Course , 2014 .

[43]  Martyn Plummer,et al.  JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling , 2003 .

[44]  M. Gutmann,et al.  Approximate Bayesian Computation , 2019, Annual Review of Statistics and Its Application.

[45]  Paul Cisek,et al.  Decision making by urgency gating: theory and experimental support. , 2012, Journal of neurophysiology.

[46]  J. Townsend,et al.  Decision field theory: a dynamic-cognitive approach to decision making in an uncertain environment. , 1993, Psychological review.

[47]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[48]  J. Yellott Correction for fast guessing and the speed-accuracy tradeoff in choice reaction time , 1971 .

[49]  Christopher R. Myers,et al.  Universally Sloppy Parameter Sensitivities in Systems Biology Models , 2007, PLoS Comput. Biol..

[50]  P. Moral,et al.  Sequential Monte Carlo samplers , 2002, cond-mat/0212648.