Theory of Parallel Particle Filters for Hidden Markov Models

The objective of this article is to study the asymptotic behavior of a new particle filtering approach in the context of hidden Markov models (HMMs). In particular, we develop an algorithm where the latent-state sequence is segmented into multiple shorter portions, with an estimation technique based upon a separate particle filter in each portion. The partitioning facilitates the use of parallel processing. Based upon this approach, we introduce new estimators of the latent states and likelihood which have similar or better variance properties compared to estimators derived from standard particle filters. Moreover due to parallelization there is less wall-clock computational time. We show that the likelihood function estimator is unbiased, prove central limit theorem convergences of estimators, and provide consistent in-sample estimation of the asymptotic variances. The theoretical analyses, supported by a numerical study, show that the segmentation reduces the variances in smoothed latent-state estimators, in addition to the savings in wall-clock time.

[1]  C. O'Connor An introduction to multivariate statistical analysis: 2nd edn. by T. W. Anderson. 675 pp. Wiley, New York (1984) , 1987 .

[2]  H. Kunsch Recursive Monte Carlo filters: Algorithms and theoretical analysis , 2006, math/0602211.

[3]  Nicolas Chopin,et al.  SMC2: an efficient algorithm for sequential analysis of state space models , 2011, 1101.1528.

[4]  Eric Moulines,et al.  Inference in hidden Markov models , 2010, Springer series in statistics.

[5]  P. Moral Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications , 2004 .

[6]  A. Doucet,et al.  Sequential auxiliary particle belief propagation , 2005, 2005 7th International Conference on Information Fusion.

[7]  P. Fearnhead,et al.  A sequential smoothing algorithm with linear computational cost. , 2010 .

[8]  Nick Whiteley,et al.  Forest resampling for distributed sequential Monte Carlo , 2014, Stat. Anal. Data Min..

[9]  Eric Moulines,et al.  On parallel implementation of sequential Monte Carlo methods: the island particle model , 2013, Stat. Comput..

[10]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[11]  T. W. Anderson An Introduction to Multivariate Statistical Analysis , 1959 .

[12]  A. Doucet,et al.  Particle Markov chain Monte Carlo methods , 2010 .

[13]  Anthony Lee,et al.  On the role of interaction in sequential Monte Carlo algorithms , 2013, 1309.2918.

[14]  T. Lai,et al.  A general theory of particle filters in hidden Markov models and some applications , 2013, 1312.5114.

[15]  A. Doucet,et al.  A Tutorial on Particle Filtering and Smoothing: Fifteen years later , 2008 .

[16]  N. Chopin Central limit theorem for sequential Monte Carlo methods and its application to Bayesian inference , 2004, math/0508594.

[17]  Arnaud Doucet,et al.  On the Utility of Graphics Cards to Perform Massively Parallel Simulation of Advanced Monte Carlo Methods , 2009, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[18]  R. Douc,et al.  Limit theorems for weighted samples with applications to sequential Monte Carlo methods , 2008 .

[19]  P. Spreij Probability and Measure , 1996 .