Reducing the run-time of MCMC programs by multithreading on SMP architectures

The increasing availability of multi-core and multiprocessor architectures provides new opportunities for improving the performance of many computer simulations. Markov chain Monte Carlo (MCMC) simulations are widely used for approximate counting problems, Bayesian inference and as a means for estimating very high-dimensional integrals. As such MCMC has found a wide variety of applications infields including computational biology and physics, financial econometrics, machine learning and image processing. This paper presents a new method for reducing the run-time of Markov chain Monte Carlo simulations by using SMP machines to speculatively perform iterations in parallel, reducing the runtime of MCMC programs whilst producing statistically identical results to conventional sequential implementations. We calculate the theoretical reduction in runtime that may be achieved using our technique under perfect conditions, and test and compare the method on a selection of multi-core and multi-processor architectures. Experiments are presented that show reductions in runtime of 35% using two cores and 55% using four cores.

[1]  Sandhya Dwarkadas,et al.  Parallel Metropolis coupled Markov chain Monte Carlo for Bayesian phylogenetic inference , 2002, Bioinform..

[2]  Sally Rosenthal,et al.  Parallel computing and Monte Carlo algorithms , 1999 .

[3]  L Tierney,et al.  Some adaptive monte carlo methods for Bayesian inference. , 1999, Statistics in medicine.

[4]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[5]  Josiane Zerubia,et al.  A marked point process model for tree crown extraction in plantations , 2005, IEEE International Conference on Image Processing 2005.

[6]  Xizhou Feng,et al.  Parallel algorithms for Bayesian phylogenetic inference , 2003, J. Parallel Distributed Comput..

[7]  Charles J. Geyer,et al.  Practical Markov Chain Monte Carlo , 1992 .

[8]  P. Green,et al.  Parallel Chains, Delayed Rejection and Reversible Jump MCMC for Object Recognition , 2000, BMVC.

[9]  Hani Doss,et al.  Phylogenetic Tree Construction Using Markov Chain Monte Carlo , 2000 .

[10]  P. Green,et al.  Delayed rejection in reversible jump Metropolis–Hastings , 2001 .

[11]  Roland Wilson,et al.  A Bayesian approach to inferring vascular tree structure from 2D imagery , 2002, Proceedings. International Conference on Image Processing.

[12]  J. Carlstrom,et al.  Markov Chain Monte Carlo Joint Analysis of Chandra X-Ray Imaging Spectroscopy and Sunyaev-Zel'dovich Effect Data , 2004, astro-ph/0403016.

[13]  Nicholas G. Polson,et al.  MCMC Methods for Financial Econometrics , 2002 .