Bayesian analysis of nonlinear and non-Gaussian state space models via multiple-try sampling methods

We develop in this paper three multiple-try blocking schemes for Bayesian analysis of nonlinear and non-Gaussian state space models. To reduce the correlations between successive iterates and to avoid getting trapped in a local maximum, we construct Markov chains by drawing state variables in blocks with multiple trial points. The first and second methods adopt autoregressive and independent kernels to produce the trial points, while the third method uses samples along suitable directions. Using the time series structure of the state space models, the three sampling schemes can be implemented efficiently. In our multimodal examples, the three multiple-try samplers are able to generate the desired posterior sample, whereas existing methods fail to do so.

[1]  G. Kitagawa.  Non-Gaussian state space modeling of time series , 1987, 26th IEEE Conference on Decision and Control.

[2]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[3]  J. Geweke,et al.  On markov chain monte carlo methods for nonlinear and non-gaussian state-space models , 1999 .

[4]  M. West,et al.  Bayesian forecasting and dynamic models , 1989 .

[5]  Kerrie Mengersen,et al.  [Bayesian Computation and Stochastic Systems]: Rejoinder , 1995 .

[6]  Michael A. West,et al.  Bayesian Forecasting and Dynamic Models (2nd edn) , 1997, J. Oper. Res. Soc..

[7]  M. Netto,et al.  On the optimal and suboptimal nonlinear filtering problem for discrete-time systems , 1977 .

[8]  George E. P. Box,et al.  Time Series Analysis: Forecasting and Control , 1977 .

[9]  N. Shephard.  Partial non-Gaussian state space , 1994 .

[10]  M. Pitt,et al.  Likelihood analysis of non-Gaussian measurement time series , 1997 .

[11]  G. Kitagawa.  Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[12]  Peter E. Rossi,et al.  Bayesian Analysis of Stochastic Volatility Models: Comments: Reply , 1994 .

[13]  Andrew Harvey,et al.  Forecasting, Structural Time Series Models and the Kalman Filter. , 1991 .

[14]  Jun S. Liu,et al.  Covariance structure of the Gibbs sampler with applications to the comparisons of estimators and augmentation schemes , 1994 .

[15]  P. Green,et al.  Metropolis Methods, Gaussian Proposals and Antithetic Variables , 1992 .

[16]  Mike K. P. So.  Time series with additive noise , 1999 .

[17]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1972 .

[18]  R. Kohn,et al.  On Gibbs sampling for state space models , 1994 .

[19]  A. Doucet,et al.  Monte Carlo Smoothing for Nonlinear Time Series , 2004, Journal of the American Statistical Association.

[20]  J. Besag,et al.  Bayesian Computation and Stochastic Systems , 1995 .

[21]  Chjan C. Lim,et al.  The Monte Carlo Approach , 2007 .

[22]  Siem Jan Koopman,et al.  Time Series Analysis of Non-Gaussian Observations Based on State Space Models from Both Classical and Bayesian Perspectives , 1999 .

[23]  N. Shephard,et al.  The simulation smoother for time series models , 1995 .

[24]  L. Gimeno,et al.  On the optimal and suboptimal nonlinear filtering problem for discrete time systems , 1977, 1977 IEEE Conference on Decision and Control including the 16th Symposium on Adaptive Processes and A Special Symposium on Fuzzy Set Theory and Applications.

[25]  Peter E. Rossi,et al.  Bayesian Analysis of Stochastic Volatility Models , 1994 .

[26]  Mike K. P. So.  POSTERIOR MODE ESTIMATION FOR NONLINEAR AND NON-GAUSSIAN STATE SPACE MODELS , 2003 .

[27]  Hisashi Tanizaki,et al.  Computational methods in statistics and econometrics , 2004 .

[28]  Nicholas G. Polson,et al.  A Monte Carlo Approach to Nonnormal and Nonlinear State-Space Modeling , 1992 .

[29]  G. Roberts,et al.  Convergence of adaptive direction sampling , 1994 .

[30]  N. Shephard.  Statistical aspects of ARCH and stochastic volatility , 1996 .

[31]  Nicholas G. Polson,et al.  Nonlinear State-Space Models With State-Dependent Variances , 2003 .

[32]  R. Kohn,et al.  Efficient Bayesian Inference for Dynamic Mixture Models , 2000 .

[33]  Jun S. Liu,et al.  The Multiple-Try Method and Local Optimization in Metropolis Sampling , 2000 .

[34]  Walter R. Gilks,et al.  Adaptive Direction Sampling , 1994 .