Sensitivity analysis in HMMs with application to likelihood maximization

This paper considers a sensitivity analysis in Hidden Markov Models with continuous state and observation spaces. We propose an Infinitesimal Perturbation Analysis (IPA) on the filtering distribution with respect to some parameters of the model. We describe a methodology for using any algorithm that estimates the filtering density, such as Sequential Monte Carlo methods, to design an algorithm that estimates its gradient. The resulting IPA estimator is proven to be asymptotically unbiased, consistent and has computational complexity linear in the number of particles. We consider an application of this analysis to the problem of identifying unknown parameters of the model given a sequence of observations. We derive an IPA estimator for the gradient of the log-likelihood, which may be used in a gradient method for the purpose of likelihood maximization. We illustrate the method with several numerical experiments.

[1]  Carlos S. Kubrusly,et al.  Stochastic approximation algorithms and applications , 1973, CDC 1973.

[2]  Paul Glasserman,et al.  Gradient Estimation Via Perturbation Analysis , 1990 .

[3]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .

[4]  G. Kitagawa.  Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[5]  F. Gland,et al.  Exponential Forgetting and Geometric Ergodicity in Hidden Markov Models , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[6]  A Orman,et al.  Optimization of Stochastic Models: The Interface Between Simulation and Optimization , 2012, J. Oper. Res. Soc..

[7]  Laurent Mevel,et al.  Exponential Forgetting and Geometric Ergodicity in Hidden Markov Models , 2000, Math. Control. Signals Syst..

[8]  R. Douc,et al.  Asymptotics of the maximum likelihood estimator for general hidden Markov models , 2001 .

[9]  Geir Storvik,et al.  Particle filters for state-space models with the presence of unknown static parameters , 2002, IEEE Trans. Signal Process..

[10]  Timothy J. Robinson,et al.  Sequential Monte Carlo Methods in Practice , 2003 .

[11]  A. Doucet,et al.  Parameter estimation in general state-space models using particle methods , 2003 .

[12]  P. Moral.  Feynman-Kac Formulae: Genealogical and Interacting Particle Systems with Applications , 2004 .

[13]  Johann Fichou,et al.  Particle-based methods for parameter estimation and tracking: Numerical experiments , 2004 .

[14]  R. Douc,et al.  Limit theorems for weighted samples with applications to sequential Monte Carlo methods , 2005, math/0507042.

[15]  Arnaud Doucet,et al.  Particle methods for optimal filter derivative: application to parameter estimation , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[16]  Eric Moulines,et al.  On the use of particle filtering for maximum likelihood parameter estimation , 2005, 2005 13th European Signal Processing Conference.

[17]  A. Papavasiliou.  A uniformly convergent adaptive particle filter , 2005 .

[18]  Arnaud Doucet,et al.  Particle Filter as A Controlled Markov Chain For On-Line Parameter Estimation in General State Space Models , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[19]  Rogemar S. Mamon,et al.  Hidden Markov Models In Finance , 2007 .

[20]  P. Moral,et al.  Sharp Propagation of Chaos Estimates for Feynman–Kac Particle Models , 2007 .

[21]  Rémi Munos,et al.  Particle Filter-based Policy Gradient in POMDPs , 2008, NIPS.