Algebraic connections between the least squares and total least squares problems

SummaryIn this paper the closeness of the total least squares (TLS) and the classical least squares (LS) problem is studied algebraically. Interesting algebraic connections between their solutions, their residuals, their corrections applied to data fitting and their approximate subspaces are proven.All these relationships point out the parameters which mainly determine the equivalences and differences between the two techniques. These parameters also lead to a better understanding of the differences in sensitivity between both approaches with respect to perturbations of the data.In particular, it is shown how the differences between both approaches increase when the equationsAX≈B become less compatible, when the length ofB orX is growing or whenA tends to be rank-deficient. They are maximal whenB is parallel with the singular vector ofA associated with its smallest singular value. Furthermore, it is shown how TLS leads to a weighted LS problem, and assumptions about the underlying perturbation model of both techniques are deduced. It is shown that many perturbation models correspond with the same TLS solution.

[1]  Sabine Van Huffel.  Analysis of the Total Least Squares Problem and its Use in Parameter Estimation , 1987 .

[2]  L. Gleser.  Estimation in a Multivariate "Errors in Variables" Regression Model: Large Sample Results , 1981 .

[3]  G. Stewart.  Collinearity and Least Squares Regression , 1987 .

[4]  Joos Vandewalle,et al.  The Use of Total Linear Least Squares Techniques for Identification and Parameter Estimation , 1985 .

[5]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[6]  Paul P. Gallo.  Consistency of regression estimates when some variables are subject to error , 1982 .

[7]  F. R. Gantmakher.  The Theory of Matrices , 1984 .

[8]  J. Vandewalle,et al.  Algebraic relationships between classical regression and total least-squares estimation , 1987 .

[9]  R. J. Adcock.  A Problem in Least Squares , 1878 .

[10]  B. Moor,et al.  A geometrical approach to the maximal corank problem in the analysis of linear relations , 1986, 1986 25th IEEE Conference on Decision and Control.

[11]  P. Sprent,et al.  A Generalized Least‐Squares Approach to Linear Functional Relationships , 1966 .

[12]  L. Mirsky,et al.  The Theory of Matrices , 1961, The Mathematical Gazette.

[13]  M. Kendall,et al.  The advanced theory of statistics , 1945 .

[14]  M. Levin.  Estimation of a system pulse transfer function in the presence of noise , 1964 .

[15]  Gabrielle E. Kelly,et al.  The influence function in the errors in variables problem , 1981 .

[16]  J. Vandewalle,et al.  Analysis and solution of the nongeneric total least squares problem , 1988 .

[17]  Gene H. Golub,et al.  Matrix computations , 1983 .

[18]  W. M. Patefield.  Multivariate Linear Relationships: Maximum Likelihood Estimation and Regression Bounds , 1981 .

[19]  Geoffrey S. Watson,et al.  Estimation of a linear transformation , 1973 .

[20]  P. Wedin.  Perturbation bounds in connection with singular value decomposition , 1972 .

[21]  Herman Van der Auweraer,et al.  A comparison of some frequency response function measurement techniques , 1986 .

[22]  T. W. Anderson.  Estimating Linear Statistical Relationships , 1984 .

[23]  Park M. Reilly,et al.  A Bayesian Study of the Error-in-Variables Model , 1981 .

[24]  R. R. Hocking.  Developments in Linear Regression Methodology: 1959–l982 , 1983 .

[25]  H. Schwetlick,et al.  Numerical Methods for Estimating Parameters in Nonlinear Models With Errors in the Variables , 1985 .