MultiBUGS: A Parallel Implementation of the BUGS Modeling Framework for Faster Bayesian Inference

MultiBUGS is a new version of the general-purpose Bayesian modelling software BUGS that implements a generic algorithm for parallelising Markov chain Monte Carlo (MCMC) algorithms to speed up posterior inference of Bayesian models. The algorithm parallelises evaluation of the product-form likelihoods formed when a parameter has many children in the directed acyclic graph (DAG) representation; and parallelises sampling of conditionally-independent sets of parameters. A heuristic algorithm is used to decide which approach to use for each parameter and to apportion computation across computational cores. This enables MultiBUGS to automatically parallelise the broad range of statistical models that can be fitted using BUGS-language software, making the dramatic speed-ups of modern multi-core computing accessible to applied statisticians, without requiring any experience of parallel programming. We demonstrate the use of MultiBUGS on simulated data designed to mimic a hierarchical e-health linked-data study of methadone prescriptions including 425,112 observations and 20,426 random effects. Posterior inference for the e-health model takes several hours in existing software, but MultiBUGS can perform inference in only 28 minutes using 48 computational cores.

[1]  M. Plummer.  JAGS Version 4.0.0 user manual , 2015 .

[2]  Anthony Brockwell.  Parallel Markov chain Monte Carlo Simulation by Pre-Fetching , 2006 .

[3]  Lorenz Wernisch,et al.  Joining and splitting models with Markov melding. , 2016, Bayesian analysis.

[4]  Andrew Thomas,et al.  The BUGS language , 2007 .

[5]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[6]  David J. Lunn,et al.  Markov Chain Monte Carlo Techniques for Studying Interoccasion and Intersubject Variability: Application to Pharmacokinetic Data , 1997 .

[7]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[8]  Edward I. George,et al.  Bayes and big data: the consensus Monte Carlo algorithm , 2016, Big Data and Information Theory.

[9]  Bradley P. Carlin,et al.  Markov Chain Monte Carlo conver-gence diagnostics: a comparative review , 1996 .

[10]  N. Breslow,et al.  Approximate inference in generalized linear mixed models , 1993 .

[11]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[12]  Alun Thomas,et al.  Markov chain Monte Carlo methods for family trees using a parallel processor , 1996, Stat. Comput..

[13]  Sally Rosenthal,et al.  Parallel computing and Monte Carlo algorithms , 1999 .

[14]  G. Amdhal,et al.  Validity of the single processor approach to achieving large scale computing capabilities , 1967, AFIPS '67 (Spring).

[15]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Lu Gao,et al.  Risk-factors for methadone-specific deaths in Scotland’s methadone-prescription clients between 2009 and 2013* , 2016, Drug and alcohol dependence.

[17]  M. Plummer,et al.  CODA: convergence diagnosis and output analysis for MCMC , 2006 .

[18]  G. Roberts,et al.  Bayesian analysis for emerging infectious diseases , 2009 .

[19]  Steffen L. Lauritzen,et al.  Independence properties of directed markov fields , 1990, Networks.

[20]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[21]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[22]  Andrew Gelman,et al.  General methods for monitoring convergence of iterative simulations , 1998 .

[23]  G. Roberts,et al.  Updating Schemes, Correlation Structure, Blocking and Parameterization for the Gibbs Sampler , 1997 .

[24]  Peter S. Pacheco.  Parallel programming with MPI , 1996 .

[25]  Judea Pearl,et al.  A Constraint-Propagation Approach to Probabilistic Reasoning , 1985, UAI.

[26]  David J. Lunn,et al.  The BUGS Book: A Practical Introduction to Bayesian Analysis , 2013 .

[27]  J. Stanley Warford,et al.  Computing Fundamentals: The Theory And Practice Of Software Design With Blackbox Component Builder , 2002 .

[28]  Christian P. Robert,et al.  MCMC Convergence Diagnostics : A « Reviewww » , 1998 .

[29]  J. Pearl.  Causality: Models, Reasoning and Inference , 2000 .

[30]  Andrew Thomas,et al.  The BUGS project: Evolution, critique and future directions , 2009, Statistics in medicine.

[31]  Darren J. Wilkinson,et al.  Parallel Bayesian Computation , 2005 .

[32]  Roman Hovorka,et al.  Rapid model exploration for complex hierarchical data: application to pharmacokinetics of insulin aspart , 2015, Statistics in medicine.

[33]  Eddie Kohler,et al.  Accelerating MCMC via Parallel Predictive Prefetching , 2014, UAI.

[34]  Ben Calderhead,et al.  A general construction for parallelizing Metropolis−Hastings algorithms , 2014, Proceedings of the National Academy of Sciences.

[35]  J. Danesh,et al.  GUESS-ing Polygenic Associations with Multiple Phenotypes Using a GPU-Based Evolutionary Stochastic Search Algorithm , 2013, PLoS genetics.

[36]  Martin Crowder,et al.  Beta-binomial Anova for Proportions , 1978 .

[37]  Simon P. Wilson,et al.  Parallel algorithms for Markov chain Monte Carlo methods in latent spatial Gaussian models , 2004, Stat. Comput..