Monte Carlo Smoothing for Nonlinear Time Series

We develop methods for performing smoothing computations in general state-space models. The methods rely on a particle representation of the filtering distributions, and their evolution through time using sequential importance sampling and resampling ideas. In particular, novel techniques are presented for generation of sample realizations of historical state sequences. This is carried out in a forward-filtering backward-smoothing procedure that can be viewed as the nonlinear, non-Gaussian counterpart of standard Kalman filter-based simulation smoothers in the linear Gaussian case. Convergence in the mean squared error sense of the smoothed trajectories is proved, showing the validity of our proposed method. The methods are tested in a substantial application for the processing of speech signals represented by a time-varying autoregression and parameterized in terms of time-varying partial correlation coefficients, comparing the results of our algorithm with those from a simple smoother based on the filtered trajectories.

[1]  Andrew J. Viterbi,et al.  Error bounds for convolutional codes and an asymptotically optimum decoding algorithm , 1967, IEEE Trans. Inf. Theory.

[2]  B. Friedlander,et al.  Lattice filters for adaptive processing , 1982, Proceedings of the IEEE.

[3]  M. West,et al.  Bayesian forecasting and dynamic models , 1989 .

[4]  Michael A. West.  Mixture Models, Monte Carlo, Bayesian Updating and Dynamic Models , 1992 .

[5]  Nicholas G. Polson,et al.  A Monte Carlo Approach to Nonnormal and Nonlinear State-Space Modeling , 1992 .

[6]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[7]  John H. L. Hansen,et al.  Discrete-Time Processing of Speech Signals , 1993 .

[8]  N. Shephard.  Partial non-Gaussian state space , 1994 .

[9]  R. Kohn,et al.  On Gibbs sampling for state space models , 1994 .

[10]  S. Frühwirth-Schnatter.  Data Augmentation and Dynamic Linear Models , 1994 .

[11]  N. Shephard,et al.  The simulation smoother for time series models , 1995 .

[12]  Jun S. Liu,et al.  Blind Deconvolution via Sequential Imputations , 1995 .

[13]  G. Kitagawa.  Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[14]  G. Kitagawa.  Smoothness priors analysis of time series , 1996 .

[15]  P. D. Jong.  The scan sampler for time series models , 1997 .

[16]  Jun S. Liu,et al.  Sequential Monte Carlo methods for dynamic systems , 1997 .

[17]  M. Pitt,et al.  Likelihood analysis of non-Gaussian measurement time series , 1997 .

[18]  Michael A. West,et al.  Bayesian Forecasting and Dynamic Models (2nd edn) , 1997, J. Oper. Res. Soc..

[19]  M. Pitt,et al.  Time Varying Covariances: A Factor Stochastic Volatility Approach (with discussion , 1998 .

[20]  Pierre Del Moral,et al.  Discrete Filtering Using Branching and Interacting Particle Systems , 1998 .

[21]  P. Moral.  Measure-valued processes and interacting particle systems. Application to nonlinear filtering problems , 1998 .

[22]  O. Aguilar,et al.  Bayesian Inference on Latent Structure in Time Series , 1998 .

[23]  Markus Hürzeler,et al.  Monte Carlo Approximations for General State-Space Models , 1998 .

[24]  D. Crisan,et al.  A particle approximation of the solution of the Kushner–Stratonovitch equation , 1999 .

[25]  Peter J. W. Rayner,et al.  Digital Audio Restoration: A Statistical Model Based Approach , 1998 .

[26]  M. West,et al.  Analysis of hospital quality monitors using hierarchical time series models , 1999 .

[27]  Jun S. Liu,et al.  Sequential importance sampling for nonparametric Bayes models: The next generation , 1999 .

[28]  M. Pitt,et al.  Filtering via Simulation: Auxiliary Particle Filters , 1999 .

[29]  Michael A. West,et al.  Evaluation and Comparison of EEG Traces: Latent Structure in Nonstationary Time Series , 1999 .

[30]  Matthew West,et al.  Priors and component structures in autoregressive time series models , 1999 .

[31]  M. West,et al.  Bayesian Inference on Periodicities and Component Spectral Structure in Time Series , 1999 .

[32]  Simon J. Godsill,et al.  On-line Bayesian modelling and enhancement of speech signals , 2000 .

[33]  M. West,et al.  Bayesian dynamic factor models and variance matrix dis-counting for portfolio allocation , 2000 .

[34]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[35]  A. Doucet,et al.  Maximum a Posteriori Sequence Estimation Using Monte Carlo Particle Filters , 2001, Annals of the Institute of Statistical Mathematics.

[36]  Simon J. Godsill,et al.  Improvement Strategies for Monte Carlo Particle Filters , 2001, Sequential Monte Carlo Methods in Practice.

[37]  W. Gilks,et al.  Following a moving target—Monte Carlo inference for dynamic Bayesian models , 2001 .

[38]  Hans R. Künsch,et al.  Approximating and Maximising the Likelihood for a General State-Space Model , 2001, Sequential Monte Carlo Methods in Practice.

[39]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[40]  Michael A. West,et al.  Combined Parameter and State Estimation in Simulation-Based Filtering , 2001, Sequential Monte Carlo Methods in Practice.

[41]  Dan Crisan,et al.  Particle Filters - A Theoretical Perspective , 2001, Sequential Monte Carlo Methods in Practice.

[42]  Simon J. Godsill,et al.  Particle methods for Bayesian modeling and enhancement of speech signals , 2002, IEEE Trans. Speech Audio Process..

[43]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .