An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo

It is now over a decade since the pioneering contribution of Gordon (1993), which is commonly regarded as the first instance of modern sequential Monte Carlo (SMC) approaches. Initially focussed on applications to tracking and vision, these techniques are now very widespread and have had a significant impact in virtually all areas of signal and image processing concerned with Bayesian dynamical models. This paper is intended to serve both as an introduction to SMC algorithms for nonspecialists and as a reference to recent contributions in domains where the techniques are still under significant development, including smoothing, estimation of fixed parameters and use of SMC methods beyond the standard filtering contexts.

[1]  Michael A. West,et al.  Combined Parameter and State Estimation in Simulation-Based Filtering , 2001, Sequential Monte Carlo Methods in Practice.

[2]  Christophe Andrieu,et al.  Particle methods for change detection, system identification, and control , 2004, Proceedings of the IEEE.

[3]  Radford M. Neal.  Annealed importance sampling , 1998, Stat. Comput..

[4]  Heikki Haario,et al.  Adaptive proposal distribution for random walk Metropolis algorithm , 1999, Comput. Stat..

[5]  T. Rydén.  Consistent and Asymptotically Normal Parameter Estimates for Hidden Markov Models , 1994 .

[6]  A. Doucet,et al.  Computational Advances for and from Bayesian Analysis , 2004 .

[7]  P. Moral.  Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications , 2004 .

[8]  R. Douc,et al.  Minimum variance importance sampling via Population Monte Carlo , 2007 .

[9]  H. Sorenson,et al.  Nonlinear Bayesian estimation using Gaussian sum approximations , 1972 .

[10]  David Madigan,et al.  A Sequential Monte Carlo Method for Bayesian Analysis of Massive Datasets , 2003, Data Mining and Knowledge Discovery.

[11]  O. Cappé,et al.  Population Monte Carlo , 2004 .

[12]  Arnaud Doucet,et al.  Particle methods for optimal filter derivative: application to parameter estimation , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[13]  P. Fearnhead.  Markov chain Monte Carlo, Sufficient Statistics, and Particle Filters , 2002 .

[14]  Hans Kiinsch,et al.  State Space and Hidden Markov Models , 2000 .

[15]  L. Wood,et al.  From the Authors , 2003, European Respiratory Journal.

[16]  D. Mayne,et al.  Monte Carlo techniques to estimate the conditional expectation in multi-stage non-linear filtering† , 1969 .

[17]  G. Kitagawa.  A self-organizing state-space model , 1998 .

[18]  Arnaud Doucet,et al.  A survey of convergence results on particle filtering methods for practitioners , 2002, IEEE Trans. Signal Process..

[19]  P. Moral,et al.  Sequential Monte Carlo samplers for rare events , 2006 .

[20]  D. Rubin,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[21]  F. Gustafsson,et al.  Complexity analysis of the marginalized particle filter , 2005, IEEE Transactions on Signal Processing.

[22]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[23]  Jun S. Liu,et al.  Sequential Monte Carlo methods for dynamic systems , 1997 .

[24]  Simon J. Godsill,et al.  Sequential Bayesian Kernel Regression , 2003, NIPS.

[25]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[26]  P. Moral,et al.  Sequential Monte Carlo samplers , 2002, cond-mat/0212648.

[27]  N. Chopin.  A sequential particle filter method for static models , 2002 .

[28]  Geir Storvik,et al.  Particle filters for state-space models with the presence of unknown static parameters , 2002, IEEE Trans. Signal Process..

[29]  Simon J. Godsill,et al.  Monte Carlo smoothing with application to audio signal enhancement , 2002, IEEE Trans. Signal Process..

[30]  R. E. Kalman,et al.  New Results in Linear Filtering and Prediction Theory , 1961 .

[31]  Arnaud Doucet,et al.  Particle filters for state estimation of jump Markov linear systems , 2001, IEEE Trans. Signal Process..

[32]  D. Rubin.  Using the SIR algorithm to simulate posterior distributions , 1988 .

[33]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[34]  N. Chopin.  Central limit theorem for sequential Monte Carlo methods and its application to Bayesian inference , 2004, math/0508594.

[35]  Michael Isard,et al.  Active Contours , 2000, Springer London.

[36]  Hisashi Tanizaki,et al.  Ch. 22. Nonlinear and non-gaussian state-space modeling with monte carlo techniques: A survey and comparative study , 2003 .

[37]  Frank Dellaert,et al.  MCMC-based particle filtering for tracking a variable number of interacting targets , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Jun S. Liu,et al.  Mixture Kalman filters , 2000 .

[39]  Ali H. Sayed,et al.  Linear Estimation (Information and System Sciences Series) , 2000 .

[40]  Walter R. Gilks,et al.  RESAMPLE-MOVE Filtering with Cross-Model Jumps , 2001, Sequential Monte Carlo Methods in Practice.

[41]  John B. Moore,et al.  Hidden Markov Models: Estimation and Control , 1994 .

[42]  M. Pitt,et al.  Filtering via Simulation: Auxiliary Particle Filters , 1999 .

[43]  A. Jazwinski.  Stochastic Processes and Filtering Theory , 1970 .

[44]  Jun S. Liu,et al.  Blind Deconvolution via Sequential Imputations , 1995 .

[45]  Jun S. Liu,et al.  Monte Carlo strategies in scientific computing , 2001 .

[46]  Simon J. Godsill,et al.  Fixed-lag smoothing using sequential importance sampling , 1999 .

[47]  Pierre Del Moral,et al.  Feynman-Kac formulae , 2004 .

[48]  A. Doucet.  On sequential Monte Carlo methods for Bayesian filtering , 1998 .

[49]  P. Fearnhead,et al.  Exact and computationally efficient likelihood‐based estimation for discretely observed diffusion processes (with discussion) , 2006 .

[50]  D. Madigan,et al.  A one-pass sequential Monte Carlo method for Bayesian analysis of massive datasets , 2006 .

[51]  Harold J. Kushner,et al.  Stochastic Approximation Algorithms and Applications , 1997, Applications of Mathematics.

[52]  Petros G. Voulgaris,et al.  On optimal ℓ∞ to ℓ∞ filtering , 1995, Autom..

[53]  Arnaud Doucet,et al.  Maximum Likelihood Parameter Estimation for Latent Variable Models Using Sequential Monte Carlo , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[54]  Hisashi Tanizaki.  Nonlinear and Non-Gaussian State-Space Modeling with Monte Carlo Techniques : A Survey and Comparative Study , 2000 .

[55]  Radford M. Neal.  Markov Chain Monte Carlo Methods Based on `Slicing' the Density Function , 1997 .

[56]  Michael A. West.  Mixture Models, Monte Carlo, Bayesian Updating and Dynamic Models , 1992 .

[57]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[58]  Yukito IBA,et al.  Population Monte Carlo algorithms , 2000, cond-mat/0008226.

[59]  Jean-Michel Marin,et al.  Convergence of Adaptive Sampling Schemes , 2004 .

[60]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[61]  B. M. Fulk.  MATH , 1992 .

[62]  A. Doucet,et al.  Particle filtering for partially observed Gaussian state space models , 2002 .

[63]  F. Gland,et al.  MLE for partially observed diffusions: direct maximization vs. the em algorithm , 1989 .

[64]  Jean Jacod,et al.  Interacting Particle Filtering with Discrete-Time Observations: Asymptotic Behaviour in the Gaussian Case , 2001 .

[65]  Eric Moulines,et al.  Inference in hidden Markov models , 2010, Springer series in statistics.

[66]  Jun S. Liu,et al.  Sequential Imputations and Bayesian Missing Data Problems , 1994 .

[67]  Yukito Iba,et al.  Population-based Monte Carlo algorithms , 2000 .

[68]  Arnaud Doucet,et al.  Stochastic sampling algorithms for state estimation of jump Markov linear systems , 2000, IEEE Trans. Autom. Control..

[69]  D. M. Titterington,et al.  Improved Particle Filters and Smoothing , 2001, Sequential Monte Carlo Methods in Practice.

[70]  Neil J. Gordon,et al.  Editors: Sequential Monte Carlo Methods in Practice , 2001 .

[71]  G. Kitagawa.  Non-Gaussian State—Space Modeling of Nonstationary Time Series , 1987 .

[72]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[73]  Y. Ho,et al.  A Bayesian approach to problems in stochastic estimation and control , 1964 .

[74]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[75]  A. Papavasiliou.  A uniformly convergent adaptive particle filter , 2005 .

[76]  Gerhard Tutz,et al.  State Space and Hidden Markov Models , 2001 .

[77]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[78]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[79]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[80]  Hisashi Tanizaki.  Nonlinear and Non-Gaussian State Space Modeling Using Sampling Techniques , 2001 .

[81]  R. Shumway,et al.  AN APPROACH TO TIME SERIES SMOOTHING AND FORECASTING USING THE EM ALGORITHM , 1982 .

[82]  Eric Moulines,et al.  Comparison of resampling schemes for particle filtering , 2005, ISPA 2005. Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, 2005..

[83]  Thomas B. Schön,et al.  Marginalized particle filters for mixed linear/nonlinear state-space models , 2005, IEEE Transactions on Signal Processing.

[84]  P. Moral,et al.  Genealogical particle analysis of rare events , 2005, math/0602525.

[85]  P. Moral.  Nonlinear filtering : Interacting particle resolution , 1997 .

[86]  Christophe Andrieu,et al.  Efficient particle filtering for jump Markov systems. Application to time-varying autoregressions , 2003, IEEE Trans. Signal Process..

[87]  Simon J. Godsill,et al.  Improvement Strategies for Monte Carlo Particle Filters , 2001, Sequential Monte Carlo Methods in Practice.

[88]  H. Künsch.  Recursive Monte Carlo filters: algorithms and theoretical analysis , 2005 .

[89]  A. Doucet,et al.  Maximum a Posteriori Sequence Estimation Using Monte Carlo Particle Filters , 2001, Annals of the Institute of Statistical Mathematics.

[90]  J. E. Handschin.  Monte Carlo techniques for prediction and filtering of non-linear stochastic processes , 1970 .

[91]  P. Fearnhead,et al.  Improved particle filter for nonlinear problems , 1999 .

[92]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[93]  P. Fearnhead,et al.  An improved particle filter for non-linear problems , 1999 .

[94]  Andrew Harvey,et al.  Forecasting, Structural Time Series Models and the Kalman Filter , 1990 .

[95]  Ajay Jasra,et al.  On population-based simulation for static inference , 2007, Stat. Comput..

[96]  Hiromitsu Kumamoto,et al.  Random sampling approach to state estimation in switching environments , 1977, Autom..

[97]  Van Nostrand,et al.  Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm , 1967 .

[98]  G. Kitagawa.  Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[99]  Olivier Cappé,et al.  Recursive computation of smoothed functionals of hidden Markovian processes using a particle approximation , 2001, Monte Carlo Methods Appl..

[100]  J. Geweke,et al.  Bayesian Inference in Econometric Models Using Monte Carlo Integration , 1989 .

[101]  P. Fearnhead,et al.  On‐line inference for hidden Markov models via particle filters , 2003 .

[102]  O. Cappé,et al.  Sequential Monte Carlo smoothing with application to parameter estimation in nonlinear state space models , 2006, math/0609514.

[103]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[104]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[105]  Eric Moulines,et al.  On the use of particle filtering for maximum likelihood parameter estimation , 2005, 2005 13th European Signal Processing Conference.

[106]  M. Pitt,et al.  Likelihood analysis of non-Gaussian measurement time series , 1997 .

[107]  E. Weinstein,et al.  A new method for evaluating the log-likelihood gradient, the Hessian, and the Fisher information matrix for linear dynamic systems , 1989, IEEE Trans. Inf. Theory.

[108]  Nando de Freitas,et al.  Fast particle smoothing: if I had a million particles , 2006, ICML.

[109]  Rudolf Kulhavý,et al.  Recursive nonlinear estimation: A geometric approach , 1996, Autom..

[110]  Johann Fichou,et al.  Particle-based methods for parameter estimation and tracking: Numerical experiments , 2004 .

[111]  P. Moral.  Measure-valued processes and interacting particle systems. Application to nonlinear filtering problems , 1998 .

[112]  W. Gilks.  Markov Chain Monte Carlo , 2005 .

[113]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[114]  W. Gilks,et al.  Following a moving target—Monte Carlo inference for dynamic Bayesian models , 2001 .

[115]  Jun S. Liu,et al.  Sequential importance sampling for nonparametric Bayes models: The next generation , 1999 .

[116]  Nando de Freitas,et al.  The Unscented Particle Filter , 2000, NIPS.

[117]  Eric Moulines,et al.  Inference in Hidden Markov Models (Springer Series in Statistics) , 2005 .

[118]  Nicholas G. Polson,et al.  State Space and Unobserved Component Models: Practical filtering for stochastic volatility models , 2004 .

[119]  A. Doucet,et al.  Sequential Monte Carlo methods for multitarget filtering with random finite sets , 2005, IEEE Transactions on Aerospace and Electronic Systems.

[120]  A. Doucet,et al.  Parameter estimation in general state-space models using particle methods , 2003 .

[121]  Jun S. Liu,et al.  A new sequential importance sampling method and its application to the two-dimensional hydrophobic-hydrophilic model , 2002 .

[122]  Amir Dembo,et al.  Exact filters for the estimation of the number of transitions of finite-state continuous-time Markov processes , 1988, IEEE Trans. Inf. Theory.

[123]  P. Djurić,et al.  Particle filtering , 2003, IEEE Signal Process. Mag..

[124]  A. Doucet,et al.  Monte Carlo Smoothing for Nonlinear Time Series , 2004, Journal of the American Statistical Association.

[125]  Christian Musso,et al.  Improving Regularised Particle Filters , 2001, Sequential Monte Carlo Methods in Practice.

[126]  Yukito Iba.  EXTENDED ENSEMBLE MONTE CARLO , 2001 .

[127]  Masahito Yamada,et al.  Structural Time Series Models and the Kalman Filter , 1989 .

[128]  Markus Hürzeler,et al.  Monte Carlo Approximations for General State-Space Models , 1998 .

[129]  Branko Ristic,et al.  Beyond the Kalman Filter: Particle Filters for Tracking Applications , 2004 .

[130]  Kazufumi Ito,et al.  Gaussian filters for nonlinear filtering problems , 2000, IEEE Trans. Autom. Control..

[131]  A. Doucet,et al.  Smoothing algorithms for state–space models , 2010 .

[132]  Siem Jan Koopman,et al.  State Space and Unobserved Component Models , 2004 .

[133]  Pierre Del Moral,et al.  Limit theorems for the multilevel splitting algorithm in the simulation of rare events , 2005, Proceedings of the Winter Simulation Conference, 2005..

[134]  C. Olivier,et al.  Recursive computation of smoothed functionals of hidden Markovian processes using a particle approximation , 2001 .

[135]  R. Douc,et al.  Convergence of Adaptive Sampling Schemes , 2007, 0708.0711.

[136]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.

[137]  A. Doucet,et al.  On-Line Parameter Estimation in General State-Space Models , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.