Convergence behavior of an iterative reweighting algorithm to compute multivariate M-estimates for location and scatter

The iteratively reweighting algorithm is one of the widely used algorithm to compute the M-estimates for the location and scatter parameters of a multivariate dataset. If the M estimating equations are the maximum likelihood estimating equations from some scale mixture of normal distributions (e.g. from a multivariate t-distribution), the iteratively reweighting algorithm is identified as an EM algorithm and the convergence behavior of which is well established. However, as Tyler (J. Roy. Statist. Soc. Ser. B 59 (1997) 550) pointed out, little is known about the theoretical convergence properties of the iteratively reweighting algorithms if it cannot be identified as an EM algorithm. In this paper, we consider the convergence behavior of the iteratively reweighting algorithm induced from the M estimating equations which cannot be identified as an EM algorithm. We give some general results on the convergence properties and, we show that convergence behavior of a general iteratively reweighting algorithm induced from the M estimating equations is similar to the convergence behavior of an EM algorithm even if it cannot be identified as an EM algorithm.

[1]  Xiao-Li Meng,et al.  The EM Algorithm—an Old Folk‐song Sung to a Fast New Tune , 1997 .

[2]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[3]  M. R. Osborne.  Finite Algorithms in Optimization and Data Analysis , 1985 .

[4]  J. Kent,et al.  Convergence Behavior of the em algorithm for the multivariate t -distribution , 1995 .

[5]  David E. Tyler,et al.  Constrained M-estimation for multivariate location and scatter , 1996 .

[6]  D. Ruppert.  Computing S Estimators for Regression and Multivariate Location/Dispersion , 1992 .

[7]  H. P. Lopuhaä.  On the relation between S-estimators and M-estimators of multivariate location and covariance , 1989 .

[8]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[9]  David E. Tyler,et al.  Redescending $M$-Estimates of Multivariate Location and Scatter , 1991 .

[10]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[11]  E. M. L. Beale,et al.  Nonlinear Programming: A Unified Approach. , 1970 .

[12]  R. Maronna.  Robust $M$-Estimators of Multivariate Location and Scatter , 1976 .

[13]  K. Lange,et al.  Normal/Independent Distributions and Their Applications in Robust Regression , 1993 .

[14]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[15]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[16]  David M. Rocke,et al.  Computable Robust Estimation of Multivariate Location and Shape in High Dimension Using Compound Estimators , 1994 .

[17]  Michael J. Black,et al.  On the unification of line processes, outlier rejection, and robust statistics with applications in early vision , 1996, International Journal of Computer Vision.

[18]  David L. Woodruff,et al.  Computation of robust estimates of multivariate location and shape , 1993 .

[19]  Donald B. Rubin.  Iteratively Reweighted Least Squares , 2006 .

[20]  P. K. Sen,et al.  Multivariate Analysis V. , 1982 .

[21]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[22]  David E. Tyler,et al.  A curious likelihood identity for the multivariate t-distribution , 1994 .

[23]  David E. Tyler.  Radial estimates and the test for sphericity , 1982 .