On Recursive Estimation in Incomplete Data Models

We consider a new recursive algorithm for parameter estimation from an independent incomplete data sequence. The algorithm can be viewed as a recursive version of the well-known EM algorithm, augmented with a Monte-Carlo step which restores the missing data. Based on recent results on stochastic algorithms, we give conditions for the a.s. convergence of the algorithm. Moreover, asymptotical variance of this estimator is reduced by a simple averaging. Application to finite mixtures is given with a simulation experiment.

[1]  T. Rydén.  Asymptotically efficient recursive estimation for incomplete data models using the observed information , 1998 .

[2]  Gilles Pagès,et al.  CONVERGENCE OF STOCHASTIC ALGORITHMS: FROM THE KUSHNER-CLARK THEOREM TO THE LYAPOUNOV FUNCTIONAL METHOD , 1996 .

[3]  G. Celeux,et al.  Stochastic versions of the em algorithm: an experimental study in the mixture case , 1996 .

[4]  S. Richardson,et al.  Mixtures of distributions: inference and estimation , 1995 .

[5]  Han-Fu Chen,et al.  Asymptotically efficient stochastic approximation , 1993 .

[6]  Boris Polyak,et al.  Acceleration of stochastic approximation by averaging , 1992 .

[7]  W. Qian,et al.  Estimation of parameters in hidden Markov models , 1991, Philosophical Transactions of the Royal Society of London. Series A: Physical and Engineering Sciences.

[8]  G. C. Wei,et al.  A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms , 1990 .

[9]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[10]  R. Redner,et al.  Mixture densities, maximum likelihood, and the EM algorithm , 1984 .

[11]  V. Fabian.  On Asymptotically Efficient Recursive Estimation , 1978 .

[12]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[13]  E. Gassiat,et al.  Testing in locally conic models, and application to mixture models , 1997 .

[14]  Xiao-Li Meng,et al.  The EM Algorithm—an Old Folk‐song Sung to a Fast New Tune , 1997 .

[15]  C. Robert.  Mixtures of Distributions: Inference and Estimation , 1996 .

[16]  O. Brandière,et al.  Les algorithmes stochastiques contournent-ils les pièges? , 1995 .

[17]  Ministerial Meeting,et al.  Summary of the , 1994 .

[18]  Han-Fu Chen,et al.  Convergence and robustness of the Robbins-Monro algorithm truncated at randomly varying bounds , 1987 .

[19]  K. Pearson.  III. Contributions to the mathematical theory of evolution , 1894, Proceedings of the Royal Society of London.