Some new results in sequential Monte Carlo

Sequential Monte Carlo (SMC) methods have been well studied within the context of performing inference with respect to partially observed Markov processes, and their use in this context relies upon the ability to evaluate or estimate the likelihood of a set of observed data, given the state of the latent process. In many real-world applications such as the study of population genetics and econometrics, however, this likelihood can neither be analytically evaluated nor replaced by an unbiased estimator, and so the application of exact SMC methods to these problems may be infeasible, or even impossible. The models in many of these applications are complex, yet realistic, and so development of techniques that can deal with problems of likelihood intractability can help us to perform inference for many important yet otherwise inaccessible problems; this motivates the research presented within this thesis. The main focus of this work is the application of approximate Bayesian computation (ABC) methodology to state-space models (SSMs) and the development of SMC methods in the context of these ABC SSMs for filtering and smoothing of the latent process. The introduction of ABC here avoids the need to evaluate the likelihood, at the cost of introducing a bias into the resulting filtering and smoothing estimators; this bias is explored theoretically and through simulation studies. An alternative SMC procedure, incorporating an additional rejection step, is also considered and the novel application of this rejection-based SMC procedure to the ABC approximation of the SSM is considered. 5 This thesis will also consider the application of MCMC and SMC methods to a class of partially observed point process (PP) models. We investigate the problem of performing sequential inference for these models and note that current methods often fail. We present a new approach to smoothing in this context, using SMC samplers (Del Moral et al., 2006). This approach is illustrated, with some theoretical discussion, on a doubly stochastic PP applied in the context of finance.

[1]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[2]  D. Cox.  Some Statistical Methods Connected with Series of Events , 1955 .

[3]  R. Kalman,et al.  New results in linear prediction and filtering theory Trans. AMSE , 1961 .

[4]  H. Sorenson,et al.  NONLINEAR FILTERING BY APPROXIMATION OF THE A POSTERIORI DENSITY , 1968 .

[5]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[6]  Donald L. Snyder,et al.  Filtering and detection for doubly stochastic Poisson processes , 1972, IEEE Trans. Inf. Theory.

[7]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  B. Efron.  The jackknife, the bootstrap, and other resampling plans , 1987 .

[9]  Alan E. Gelfand,et al.  Bayesian statistics without tears: A sampling-resampling perspective , 1992 .

[10]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[11]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[12]  Jun S. Liu,et al.  Sequential Imputations and Bayesian Missing Data Problems , 1994 .

[13]  G. Evensen.  Inverse methods and data assimilation in nonlinear ocean models , 1994 .

[14]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[15]  B. Carlin,et al.  Bayesian Model Choice Via Markov Chain Monte Carlo Methods , 1995 .

[16]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[17]  G. Kitagawa.  Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[18]  G. Casella,et al.  Rao-Blackwellisation of sampling schemes , 1996 .

[19]  Jun S. Liu,et al.  Metropolized independent sampling with comparisons to rejection sampling and importance sampling , 1996, Stat. Comput..

[20]  A. Gelman,et al.  Weak convergence and optimal scaling of random walk Metropolis algorithms , 1997 .

[21]  Jun S. Liu,et al.  Sequential Monte Carlo methods for dynamic systems , 1997 .

[22]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[23]  P. Donnelly,et al.  Inferring coalescence times from DNA sequence data. , 1997, Genetics.

[24]  L. Tierney.  A note on Metropolis-Hastings kernels for general state spaces , 1998 .

[25]  M. Pitt,et al.  Filtering via Simulation: Auxiliary Particle Filters , 1999 .

[26]  M. Feldman,et al.  Population growth of human Y chromosomes: a study of Y chromosome microsatellites. , 1999, Molecular biology and evolution.

[27]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[28]  Simon J. Godsill,et al.  On sequential Monte Carlo sampling methods for Bayesian filtering , 2000, Stat. Comput..

[29]  P. Moral,et al.  On the stability of interacting processes with applications to filtering and genetic algorithms , 2001 .

[30]  Thia Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software , 2001 .

[31]  N. Shephard,et al.  Non‐Gaussian Ornstein–Uhlenbeck‐based models and some of their uses in financial economics , 2001 .

[32]  W. Gilks,et al.  Following a moving target—Monte Carlo inference for dynamic Bayesian models , 2001 .

[33]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[34]  Tim Hesterberg,et al.  Monte Carlo Strategies in Scientific Computing , 2002, Technometrics.

[35]  P. Moral,et al.  Sequential Monte Carlo samplers , 2002, cond-mat/0212648.

[36]  Simon J. Godsill,et al.  Particle methods for Bayesian modeling and enhancement of speech signals , 2002, IEEE Trans. Speech Audio Process..

[37]  Paul Marjoram,et al.  Markov chain Monte Carlo without likelihoods , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[38]  A. Shapiro.  Monte Carlo Sampling Methods , 2003 .

[39]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[40]  J. Rosenthal,et al.  General state space Markov chains and MCMC algorithms , 2004, math/0404033.

[41]  G. Roberts,et al.  Bayesian inference for non‐Gaussian Ornstein–Uhlenbeck stochastic volatility processes , 2004 .

[42]  N. Chopin.  Central limit theorem for sequential Monte Carlo methods and its application to Bayesian inference , 2004, math/0508594.

[43]  A. Doucet,et al.  Monte Carlo Smoothing for Nonlinear Time Series , 2004, Journal of the American Statistical Association.

[44]  Paul Fearnhead,et al.  Filtering recursions for calculating likelihoods for queues based on inter-departure time data , 2004, Stat. Comput..

[45]  A. Doucet,et al.  Computational Advances for and from Bayesian Analysis , 2004 .

[46]  P. Moral.  Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications , 2004 .

[47]  A. Doucet,et al.  Exponential forgetting and geometric ergodicity for optimal filtering in general state-space models , 2005 .

[48]  Eric Moulines,et al.  Comparison of resampling schemes for particle filtering , 2005, ISPA 2005. Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, 2005..

[49]  M. Minozzo,et al.  Estimation and filtering by reversible jump MCMC for a doubly stochastic Poisson model for ultra-high-frequency financial data , 2006 .

[50]  M. Minozzo,et al.  A Monte Carlo Approach to Filtering for a Class of Marked Doubly Stochastic Poisson Processes , 2006 .

[51]  P. Moral,et al.  Sequential Monte Carlo for Bayesian Computation , 2006 .

[52]  Arnaud Doucet,et al.  Optimal Filtering For Partially Observed Point Processes Using Trans-Dimensional Sequential Monte Carlo , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[53]  Ajay Jasra,et al.  On population-based simulation for static inference , 2007, Stat. Comput..

[54]  Václav Smídl,et al.  Variational Bayesian Filtering , 2008, IEEE Transactions on Signal Processing.

[55]  P. Bickel,et al.  Obstacles to High-Dimensional Particle Filtering , 2008 .

[56]  Elisa Varini,et al.  A Monte Carlo method for filtering a marked doubly stochastic Poisson process , 2008, Stat. Methods Appl..

[57]  P. Bickel,et al.  Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems , 2008, 0805.3034.

[58]  Aarnout Brombacher,et al.  Probability... , 2009, Qual. Reliab. Eng. Int..

[59]  F. Campillo,et al.  Convolution Particle Filter for Parameter Estimation in General State-Space Models , 2009, IEEE Transactions on Aerospace and Electronic Systems.

[60]  Roman Holenstein,et al.  Particle Markov chain Monte Carlo , 2009 .

[61]  A. Doucet,et al.  Smoothing algorithms for state–space models , 2010 .

[62]  P. Bickel,et al.  Ensemble Filtering for High Dimensional Non-linear State Space Models , 2009 .

[63]  Sumeetpal S. Singh,et al.  A backward particle interpretation of Feynman-Kac formulae , 2009, 0908.2556.

[64]  Sumeetpal S. Singh,et al.  Forward Smoothing using Sequential Monte Carlo , 2010, 1012.5390.

[65]  Simon J. Godsill,et al.  Monte Carlo Filtering of Piecewise Deterministic Processes , 2011 .

[66]  A. Beskos,et al.  On the stability of sequential Monte Carlo methods in high dimensions , 2011, 1103.3965.

[67]  Sumeetpal S. Singh,et al.  Parameter Estimation for Hidden Markov Models with Intractable Likelihoods , 2011 .

[68]  Sumeetpal S. Singh,et al.  Particle approximations of the score and observed information matrix in state space models with application to parameter estimation , 2011 .

[69]  A. Beskos,et al.  Error Bounds and Normalizing Constants for Sequential Monte Carlo in High Dimensions , 2011, 1112.1544.

[70]  O. Papaspiliopoulos,et al.  SMC^2: A sequential Monte Carlo algorithm with particle Markov chain Monte Carlo updates , 2011 .

[71]  Sumeetpal S. Singh,et al.  Filtering via approximate Bayesian computation , 2010, Statistics and Computing.

[72]  Jean-Michel Marin,et al.  Approximate Bayesian computational methods , 2011, Statistics and Computing.

[73]  Sumeetpal S. Singh,et al.  Approximate Bayesian Computation for Smoothing , 2012, 1206.5208.

[74]  Ajay Jasra,et al.  Inference for a class of partially observed point process models , 2012, 1201.4529.

[75]  J. Stoyanov.  The Oxford Handbook of Nonlinear Filtering , 2012 .

[76]  Lucy Marshall,et al.  The ensemble Kalman filter is an ABC algorithm , 2012, Stat. Comput..

[77]  N. Whiteley.  Stability properties of some particle filters , 2011, 1109.6779.

[78]  Robert L. Wolpert,et al.  Statistical Inference , 2019, Encyclopedia of Social Network Analysis and Mining.