Recursive EM and SAGE-inspired algorithms with application to DOA estimation

This paper is concerned with recursive estimation using augmented data. We study two recursive procedures closely linked with the well-known expectation and maximization (EM) and space alternating generalized EM (SAGE) algorithms. Unlike iterative methods, the recursive EM and SAGE-inspired algorithms give a quick update on estimates given new data. Under mild conditions, estimates generated by these procedures are strongly consistent and asymptotically normally distributed. These mathematical properties are valid for a broad class of problems. When applied to direction of arrival (DOA) estimation, the recursive EM and SAGE-inspired algorithms lead to a very simple and fast implementation of the maximum-likelihood (ML) method. The most complicated computation in each recursion is inversion of the augmented information matrix. Through data augmentation, this matrix is diagonal and easy to invert. More importantly, there is no search in such recursive procedures. Consequently, the computational time is much less than that associated with existing numerical methods for finding ML estimates. This feature greatly increases the potential of the ML approach in real-time processing. Numerical experiments show that both algorithms provide good results with low computational cost.

[1]  J.F. Bohme,et al.  Recursive EM and SAGE algorithms , 2001, Proceedings of the 11th IEEE Signal Processing Workshop on Statistical Signal Processing (Cat. No.01TH8563).

[2]  Meir Feder,et al.  Recursive expectation-maximization (EM) algorithms for time-varying parameters with applications to multiple target tracking , 1999, IEEE Trans. Signal Process..

[3]  D. Brillinger.  Time series - data analysis and theory , 1981, Classics in applied mathematics.

[4]  V. Nollau.  Kushner, H. J./Clark, D. S., Stochastic Approximation Methods for Constrained and Unconstrained Systems. (Applied Mathematical Sciences 26). Berlin‐Heidelberg‐New York, Springer‐Verlag 1978. X, 261 S., 4 Abb., DM 26,40. US $ 13.20 , 1980 .

[5]  D. Titterington.  Recursive Parameter Estimation Using Incomplete Data , 1984 .

[6]  Pei Jung Chung,et al.  Comparative convergence analysis of EM and SAGE algorithms in DOA estimation , 2001, IEEE Trans. Signal Process..

[7]  H. Piaggio.  Mathematical Analysis , 1955, Nature.

[8]  Boris Polyak,et al.  Acceleration of stochastic approximation by averaging , 1992 .

[9]  Tamer Basar,et al.  Analysis of Recursive Stochastic Algorithms , 2001 .

[10]  V. Fabian.  On Asymptotic Normality in Stochastic Approximation , 1968 .

[11]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[12]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[13]  Petre Stoica,et al.  MUSIC, maximum likelihood, and Cramer-Rao bound , 1989, IEEE Transactions on Acoustics, Speech, and Signal Processing.

[14]  Bin Yang,et al.  Projection approximation subspace tracking , 1995, IEEE Trans. Signal Process..

[15]  V. Fabian.  On Asymptotically Efficient Recursive Estimation , 1978 .

[16]  Pei-Jung Chung,et al.  EM and SAGE Algorithms for Towed Array Data , 2004 .

[17]  Petre Stoica,et al.  Performance study of conditional and unconditional direction-of-arrival estimation , 1990, IEEE Trans. Acoust. Speech Signal Process..

[18]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .

[19]  Carlos S. Kubrusly,et al.  Stochastic approximation algorithms and applications , 1973, CDC 1973.

[20]  Alfred O. Hero,et al.  Space-alternating generalized expectation-maximization algorithm , 1994, IEEE Trans. Signal Process..

[21]  David Williams,et al.  Probability with Martingales , 1991, Cambridge mathematical textbooks.

[22]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.

[23]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[24]  J.F. Bohme,et al.  DOA estimation of multiple moving sources using recursive EM algorithms , 2002, Sensor Array and Multichannel Signal Processing Workshop Proceedings, 2002.

[25]  J. Sacks.  Asymptotic Distribution of Stochastic Approximation Procedures , 1958 .

[26]  George Ch. Pflug,et al.  Optimization of Stochastic Models , 1996 .