Maximum Likelihood Estimation in Random Linear Models: Generalizations and Performance Analysis

We consider the problem of estimating an unknown deterministic parameter vector in a linear model with a Gaussian model matrix. The matrix has a known mean and independent rows of equal covariance matrix. Our problem formulation also allows for some known columns within this model matrix. We derive the maximum likelihood (ML) estimator associated with this problem and show that it can be found using a simple line-search over a unimodal function which can be efficiently evaluated. We then analyze its asymptotic performance using the Cramer Rao bound. Finally, we discuss the similarity between the ML, total least squares (TLS), and regularized TLS estimators

[1]  Amir Beck,et al.  On the Solution of the Tikhonov Regularization of the Total Least Squares Problem , 2006, SIAM J. Optim..

[2]  S. Kay.  Fundamentals of statistical signal processing: estimation theory , 1993 .

[3]  Ami Wiesel,et al.  Maximum likelihood estimation in linear models with a Gaussian model matrix , 2006, IEEE Signal Processing Letters.

[4]  A. Beck,et al.  Hidden convexity based near maximum-likelihood CDMA detection , 2005, IEEE 6th Workshop on Signal Processing Advances in Wireless Communications, 2005..

[5]  G. Forsythe,et al.  On the Stationary Values of a Second-Degree Polynomial on the Unit Sphere , 1965 .

[6]  Gene H. Golub,et al.  Tikhonov Regularization and Total Least Squares , 1999, SIAM J. Matrix Anal. Appl..

[7]  Sabine Van Huffel,et al.  The total least squares problem , 1993 .

[8]  Steven Kay,et al.  Fundamentals Of Statistical Signal Processing , 2001 .

[9]  Sabine Van Huffel,et al.  Total least squares problem - computational aspects and analysis , 1991, Frontiers in applied mathematics.

[10]  Gene H. Golub,et al.  An analysis of the total least squares problem , 1980, Milestones in Matrix Computation.

[11]  Stephen P. Boyd,et al.  Robust solutions to l/sub 1/, l/sub 2/, and l/sub /spl infin// uncertain linear approximation problems using convex optimization , 1998, Proceedings of the 1998 American Control Conference. ACC (IEEE Cat. No.98CH36207).

[12]  K.Venkatesh Prasad,et al.  Fundamentals of statistical signal processing: Estimation theory: by Steven M. KAY; Prentice Hall signal processing series; Prentice Hall; Englewood Cliffs, NJ, USA; 1993; xii + 595 pp.; $65; ISBN: 0-13-345711-7 , 1994 .

[13]  J. Vandewalle,et al.  Analysis and properties of the generalized total least squares problem AX≈B when some or all columns in A are subject to error , 1989 .

[14]  Henry Wolkowicz,et al.  The trust region subproblem and semidefinite programming , 2004, Optim. Methods Softw..

[15]  Laurent El Ghaoui,et al.  Robust Solutions to Least-Squares Problems with Uncertain Data , 1997, SIAM J. Matrix Anal. Appl..