Weighted Majorization Algorithms for Weighted Least Squares Decomposition Models

For many least-squares decomposition models efficient algorithms are well known. A more difficult problem arises in decomposition models where each residual is weighted by a nonnegative value. A special case is principal components analysis with missing data. Kiers (1997) discusses an algorithm for minimizing weighted decomposition models by iterative majorization. In this paper, we for computing a solution. We will show that the algorithm by Kiers is a special case of our algorithm. Here, we will apply weighted majorization to weighted principal components analysis, robust Procrustes analysis, and logistic bi-additive models of which the two parameter logistic model in item response theory is a special case. Simulation studies show that weighted majorization is generally faster than the method by Kiers by a factor one to four and obtains the same or better quality solutions. For logistic bi-additive models, we propose a new iterative majorization algorithm called logistic majorization.

[1]  A. Béguin,et al.  MCMC estimation and some model-fit analysis of multidimensional IRT models , 2001 .

[2]  S. Zamir,et al.  Lower Rank Approximation of Matrices by Least Squares With Any Choice of Weights , 1979 .

[3]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[4]  Melvin R. Novick,et al.  Statistical Theories of Mental Test Scores , 1969 .

[5]  W. J. Krzanowski,et al.  Recent Advances in Descriptive Multivariate Analysis. , 1996 .

[6]  T. Berge.  Least squares optimization in multivariate analysis , 2005 .

[7]  G. J. Mellenbergh,et al.  Generalized linear item response theory. , 1994 .

[8]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[9]  Frederick Mosteller,et al.  Data Analysis and Regression , 1978 .

[10]  Melvin R. Novick,et al.  Some latent train models and their use in inferring an examinee's ability , 1966 .

[11]  D. Hunter,et al.  Optimization Transfer Using Surrogate Objective Functions , 2000 .

[12]  P. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 1999 .

[13]  P. McCullagh,et al.  Generalized Linear Models , 1992 .

[14]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[15]  Jan de Leeuw,et al.  Block-relaxation Algorithms in Statistics , 1994 .

[16]  Henk A. L. Kiers,et al.  Setting up alternating least squares and iterative majorization algorithms for solving various matrix optimization problems , 2002, Comput. Stat. Data Anal..

[17]  J. Leeuw.  Fitting Distances by Least Squares , 1993 .

[18]  H. Kiers.  Weighted least squares fitting using ordinary least squares algorithms , 1997 .

[19]  Willem S. Heiser.  Correspondence analysis with least absolute residuals , 1987 .

[20]  Peter Verboon,et al.  A robust approach to nonlinear multivariate analysis , 1994 .

[21]  Willem J. Heiser,et al.  Resistant orthogonal procrustes analysis , 1992 .

[22]  J. Tukey,et al.  The Fitting of Power Series, Meaning Polynomials, Illustrated on Band-Spectroscopic Data , 1974 .