Maximum likelihood fitting using ordinary least squares algorithms

In this paper a general algorithm is provided for maximum likelihood fitting of deterministic models subject to Gaussian‐distributed residual variation (including any type of non‐singular covariance). By deterministic models is meant models in which no distributional assumptions are valid (or applied) on the parameters. The algorithm may also more generally be used for weighted least squares (WLS) fitting in situations where either distributional assumptions are not available or other than statistical assumptions guide the choice of loss function. The algorithm to solve the associated problem is called MILES (Maximum likelihood via Iterative Least squares EStimation). It is shown that the sought parameters can be estimated using simple least squares (LS) algorithms in an iterative fashion. The algorithm is based on iterative majorization and extends earlier work for WLS fitting of models with heteroscedastic uncorrelated residual variation. The algorithm is shown to include several current algorithms as special cases. For example, maximum likelihood principal component analysis models with and without offsets can be easily fitted with MILES. The MILES algorithm is simple and can be implemented as an outer loop in any least squares algorithm, e.g. for analysis of variance, regression, response surface modeling, etc. Several examples are provided on the use of MILES. Copyright © 2002 John Wiley & Sons, Ltd.

[1]  G. Ewing.  Instrumental methods of chemical analysis , 1954 .

[2]  N. Draper,et al.  Applied Regression Analysis , 1966 .

[3]  Pieter M. Kroonenberg,et al.  Three-mode principal component analysis : theory and applications , 1983 .

[4]  G. Judge,et al.  The Theory and Practice of Econometrics , 1981 .

[5]  Willem S. Heiser.  Correspondence analysis with least absolute residuals , 1987 .

[6]  Henk A. L. Kiers,et al.  Majorization as a tool for optimizing a class of matrix functions , 1990 .

[7]  S. Leurgans,et al.  Multilinear Models: Applications in Spectroscopy , 1992 .

[8]  J. Berge,et al.  Minimization of a class of matrix trace functions by means of refined majorization , 1992 .

[9]  P. Paatero,et al.  Analysis of different modes of factor analysis as least squares fit problems , 1993 .

[10]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[11]  Sue Leurgans,et al.  [27] Component resolution using multilinear models , 1995 .

[12]  P. Groenen,et al.  The majorization approach to multidimensional scaling for Minkowski distances , 1995 .

[13]  Rasmus Bro,et al.  Enzymatic browning of vegetables. Calibration and analysis of variance by multiway methods , 1996 .

[14]  Darren T. Andrews,et al.  Maximum likelihood principal component analysis , 1997 .

[15]  R. Bro.  PARAFAC. Tutorial and applications , 1997 .

[16]  H. Kiers.  Weighted least squares fitting using ordinary least squares algorithms , 1997 .

[17]  Rasmus Bro,et al.  Prediction of Polyphenol Oxidase Activity in Model Solutions Containing Various Combinations of Chlorogenic Acid, (−)-Epicatechin, O2, CO2, Temperature, and pH by Multiway Data Analysis , 1997 .

[18]  N. Sidiropoulos,et al.  Least squares algorithms under unimodality and non‐negativity constraints , 1998 .

[19]  R. Bro.  Exploratory study of sugar production using fluorescence spectroscopy and multi-way analysis , 1999 .

[20]  Peter D. Wentzell,et al.  Maximum likelihood principal component analysis with correlated measurement errors: theoretical and practical considerations , 1999 .

[21]  J. Vandewalle,et al.  An introduction to independent component analysis , 2000 .

[22]  K. Booksh,et al.  Mitigation of Rayleigh and Raman spectral interferences in multiway calibration of excitation-emission matrix fluorescence spectra. , 2000, Analytical chemistry.

[23]  P. Wentzell,et al.  Three-way analysis of fluorescence spectra of polycyclic aromatic hydrocarbons with quenching by nitromethane. , 2001, Analytical chemistry.

[24]  Nikos D. Sidiropoulos,et al.  Identifiability results for blind beamforming in incoherent multipath with small delay spread , 2001, IEEE Trans. Signal Process..

[25]  Nikos D. Sidiropoulos,et al.  Cramer-Rao lower bounds for low-rank decomposition of multidimensional arrays , 2001, IEEE Trans. Signal Process..