Memory (and Time) Efficient Sequential Monte Carlo

Memory efficiency is an important issue in Sequential Monte Carlo (SMC) algorithms, arising for example in inference of high-dimensional latent variables via Rao-Blackwellized SMC algorithms, where the size of individual particles combined with the required number of particles can stress the main memory. Standard SMC methods have a memory requirement that scales linearly in the number of particles present at all stage of the algorithm. Our contribution is a simple scheme that makes the memory cost of SMC methods depends on the number of distinct particles that survive resampling. We show that this difference has a large empirical impact on the quality of the approximation in realistic scenarios, and also--since memory access is generally slow--on the running time. The method is based on a two pass generation of the particles, which are represented implicitly in the first pass. We parameterize the accuracy of our algorithm with a memory budget rather than with a fixed number of particles. Our algorithm adaptively selects an optimal number of particle to exploit this fixed memory budget. We show that this adaptation does not interfere with the usual consistency guarantees that come with SMC algorithms.

[1]  Anthony Lee,et al.  The Alive Particle Filter , 2013 .

[2]  Liangliang Wang,et al.  Entangled Monte Carlo , 2012, NIPS.

[3]  Xuan Liu,et al.  Efficient Delay-Tolerant Particle Filtering , 2011, IEEE Transactions on Signal Processing.

[4]  Pierre Del Moral,et al.  Biips: Software for Bayesian Inference with Interacting Particle Systems , 2014, 1412.3779.

[5]  R. Douc,et al.  Limit theorems for weighted samples with applications to sequential Monte Carlo methods , 2008 .

[6]  E. J. J. Rensburg,et al.  TOPICAL REVIEW: Monte Carlo methods for the self-avoiding walk , 2009 .

[7]  Arnaud Doucet,et al.  A survey of convergence results on particle filtering methods for practitioners , 2002, IEEE Trans. Signal Process..

[8]  Yee Whye Teh,et al.  An Efficient Sequential Monte Carlo Algorithm for Coalescent Clustering , 2008, NIPS.

[9]  Michael I. Jordan,et al.  Phylogenetic Inference via Sequential Monte Carlo , 2012, Systematic biology.

[10]  Arnaud Doucet,et al.  On the Utility of Graphics Cards to Perform Massively Parallel Simulation of Advanced Monte Carlo Methods , 2009, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[11]  Jun S. Liu,et al.  Sequential Imputations and Bayesian Missing Data Problems , 1994 .

[12]  Andreas Krause,et al.  Advances in Neural Information Processing Systems (NIPS) , 2014 .

[13]  A. Doucet,et al.  Particle Markov chain Monte Carlo methods , 2010 .

[14]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[15]  A. Doucet,et al.  Efficient implementation of Markov chain Monte Carlo when using an unbiased likelihood estimator , 2012, 1210.1871.

[16]  P. Moral,et al.  Sequential Monte Carlo samplers , 2002, cond-mat/0212648.

[17]  Liangliang Wang,et al.  Bayesian phylogenetic inference via Monte Carlo methods , 2012 .

[18]  A. Doucet,et al.  A note on auxiliary particle filters , 2008 .

[19]  P. Moral,et al.  Sequential Monte Carlo for Bayesian Computation , 2006 .

[20]  Clemens Elster,et al.  Efficient implementation of a Monte Carlo method for uncertainty evaluation in dynamic measurements , 2012 .

[21]  A. Chorin,et al.  Implicit sampling for particle filters , 2009, Proceedings of the National Academy of Sciences.

[22]  Horst-Michael Groß,et al.  Memory-Efficient Gridmaps in Rao-Blackwellized Particle Filters for SLAM using Sonar Range Sensors , 2007, EMCR.

[23]  M. Pitt,et al.  Filtering via Simulation: Auxiliary Particle Filters , 1999 .

[24]  Nadia Oudjane,et al.  A sequential particle algorithm that keeps the particle system alive , 2005, 2005 13th European Signal Processing Conference.

[25]  A. Doucet,et al.  A Tutorial on Particle Filtering and Smoothing: Fifteen years later , 2008 .

[26]  Eric Moulines,et al.  Comparison of resampling schemes for particle filtering , 2005, ISPA 2005. Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, 2005..

[27]  Yee Whye Teh,et al.  Bayesian Agglomerative Clustering with Coalescents , 2007, NIPS.

[28]  P. Moral.  Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications , 2004 .

[29]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[30]  David R. Bickel.  Comment on "Sequential Monte Carlo for Bayesian Computation" (P. Del Moral, A. Doucet, A. Jasra) , 2006 .

[31]  C. Geyer.  Markov Chain Monte Carlo Maximum Likelihood , 1991 .