Forward-reverse EM algorithm for Markov chains

We develop an EM algorithm for estimating parameters that determine the dynamics of a discrete time Markov chain evolving through a certain measurable state space. As a key tool for the construction of the EM method we develop forward-reverse representations for Markov chains conditioned on a certain terminal state. These representations may be considered as an extension of the earlier work [1] on conditional diffusions. We present several experiments and consider the convergence of the new EM algorithm.

[1]  K. Chan,et al.  Monte Carlo EM Estimation for Time Series Models Involving Counts , 1995 .

[2]  B. Delyon,et al.  Simulation of conditioned diffusion and application to parameter estimation , 2006 .

[3]  D. Rubin,et al.  The ECME algorithm: A simple extension of EM and ECM with faster monotone convergence , 1994 .

[4]  John Schoenmakers,et al.  Transition density estimation for stochastic differential equations via forward-reverse representations , 2004 .

[5]  Panos Stinis,et al.  CONDITIONAL PATH SAMPLING FOR STOCHASTIC DIFFERENTIAL EQUATIONS THROUGH DRIFT RELAXATION , 2011 .

[6]  Xiao-Li Meng,et al.  Fitting Full-Information Item Factor Models and an Empirical Investigation of Bridge Sampling , 1996 .

[7]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[8]  G. C. Wei,et al.  A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms , 1990 .

[9]  V. Spokoiny,et al.  Forward and reverse representations for Markov chains ∗ , 2006 .

[10]  Mogens Bladt,et al.  Corrigendum to “Simple simulation of diffusion bridges with application to likelihood inference for diffusions” , 2010, Bernoulli.

[11]  Xiao-Li Meng,et al.  Maximum likelihood estimation via the ECM algorithm: A general framework , 1993 .

[12]  Evaluation of conditional Wiener integrals by numerical integration of stochastic differential equations , 2004 .

[13]  A. Stuart,et al.  Conditional Path Sampling of SDEs and the Langevin MCMC Method , 2004 .

[14]  Lain L. MacDonald,et al.  Hidden Markov and Other Models for Discrete- valued Time Series , 1997 .

[15]  John Schoenmakers,et al.  Simulation of forward-reverse stochastic representations for conditional diffusions , 2013, 1306.2452.

[16]  É. Moulines,et al.  Convergence of a stochastic approximation version of the EM algorithm , 1999 .