Uniform Stability of a Particle Approximation of the Optimal Filter Derivative

Sequential Monte Carlo methods, also known as particle methods, are a widely used set of computational tools for inference in non-linear non-Gaussian state-space models. In many applications it may be necessary to compute the sensitivity, or derivative, of the optimal filter with respect to the static parameters of the state-space model; for instance, in order to obtain maximum likelihood model parameters of interest, or to compute the optimal controller in an optimal control problem. In Poyiadjis et al. [2011] an original particle algorithm to compute the filter derivative was proposed and it was shown using numerical examples that the particle estimate was numerically stable in the sense that it did not deteriorate over time. In this paper we substantiate this claim with a detailed theoretical study. Lp bounds and a central limit theorem for this particle approximation of the filter derivative are presented. It is further shown that under mixing conditions these Lp bounds and the asymptotic variance characterized by the central limit theorem are uniformly bounded with respect to the time index. We demon- strate the performance predicted by theory with several numerical examples. We also use the particle approximation of the filter derivative to perform online maximum likelihood parameter estimation for a stochastic volatility model.

[1]  D. Titterington.  Recursive Parameter Estimation Using Incomplete Data , 1984 .

[2]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[3]  F. LeGland,et al.  Recursive estimation in hidden Markov models , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[4]  A Orman,et al.  Optimization of Stochastic Models: The Interface Between Simulation and Optimization , 2012, J. Oper. Res. Soc..

[5]  M. Pitt,et al.  Filtering via Simulation: Auxiliary Particle Filters , 1999 .

[6]  J. Azéma,et al.  Seminaire de Probabilites XXXIV , 2000 .

[7]  P. Moral,et al.  Branching and interacting particle systems. Approximations of Feynman-Kac formulae with applications to non-linear filtering , 2000 .

[8]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[9]  P. Moral,et al.  Genealogies and Increasing Propagation of Chaos For Feynman-Kac and Genetic Models , 2001 .

[10]  Carlo Novara,et al.  Nonlinear Time Series , 2003 .

[11]  Jianqing Fan.  Nonlinear Time Series , 2003 .

[12]  A. Doucet,et al.  Parameter estimation in general state-space models using particle methods , 2003 .

[13]  P. Moral,et al.  On a Class of Genealogical and Interacting Metropolis Models , 2003 .

[14]  P. Moral.  Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications , 2004 .

[15]  Melvin J. Hinich,et al.  Time Series Analysis by State Space Methods , 2001 .

[16]  Arnaud Doucet,et al.  Particle methods for optimal filter derivative: application to parameter estimation , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[17]  L. Gerencsér,et al.  Recursive estimation of Hidden Markov Models , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[18]  N. Oudjane,et al.  Stability and Uniform Particle Approximation of Nonlinear Filters in Case of Non Ergodic Signals , 2005 .

[19]  Haikady N. Nagaraja,et al.  Inference in Hidden Markov Models , 2006, Technometrics.

[20]  Rémi Munos,et al.  Particle Filter-based Policy Gradient in POMDPs , 2008, NIPS.

[21]  O. Cappé,et al.  Sequential Monte Carlo smoothing with application to parameter estimation in nonlinear state space models , 2006, math/0609514.

[22]  D. Crisan,et al.  Uniform approximations of discrete-time filters , 2008, Advances in Applied Probability.

[23]  Aurélien Garivier,et al.  ON THE FORWARD FILTERING BACKWARD SMOOTHING PARTICLE APPROXIMATIONS OF THE SMOOTHING DISTRIBUTION IN GENERAL STATE SPACES MODELS , 2009, 0904.0316.

[24]  Arnaud Doucet,et al.  An overview of sequential Monte Carlo methods for parameter estimation in general state-space models , 2009 .

[25]  Sumeetpal S. Singh,et al.  A backward particle interpretation of Feynman-Kac formulae , 2009, 0908.2556.

[26]  A. Beskos,et al.  On the stability of sequential Monte Carlo methods in high dimensions , 2011, 1103.3965.

[27]  P. Moral,et al.  Concentration Inequalities for Mean Field Particle Models , 2011, 1211.1837.

[28]  N. Whiteley.  Sequential Monte Carlo Samplers: Error Bounds and Insensitivity to Initial Conditions , 2011, 1103.3970.

[29]  Aurélien Garivier,et al.  Sequential Monte Carlo smoothing for general state space hidden Markov models , 2011, 1202.2945.

[30]  Sumeetpal S. Singh,et al.  Particle approximations of the score and observed information matrix in state space models with application to parameter estimation , 2011 .

[31]  N. Whiteley.  Stability properties of some particle filters , 2011, 1109.6779.

[32]  Arnaud Doucet,et al.  On Particle Methods for Parameter Estimation in State-Space Models , 2014, 1412.8695.

[33]  Ajay Jasra,et al.  On the Behaviour of the Backward Interpretation of Feynman-Kac Formulae Under Verifiable Conditions , 2013, J. Appl. Probab..

[34]  A. Jasra.  On the behaviour of the backward interpretation of Feynman-Kac formulae under verifiable conditions , 2015 .