Total Least Squares and Errors- In-Variables Modeling: Bridging the Gap Between Statistics, Computational Mathematics and Engineering

The main purpose of this paper is to present an overview of the progress of a modeling technique which is known as Total Least Squares (TLS) in computational mathematics and engineering, and as Errors-In- Variables (EIV) modeling or orthogonal regression in the statistical community. The basic concepts of TLS and EIV modeling are presented. In particular, it is shown how the seemingly different linear algebraic approach of TLS, as studied in computational mathematics and applied in diverse engineering fields, is related to EIV regression, as studied in the field of statistics. Computational methods, as well as the main algebraic, sensitivity and statistical properties of the estimators, are discussed. Furthermore, generalizations of the basic concept of TLS and EIV modeling, such as structured TLS, Lp approximations, nonlinear and polynomial EIV, are introduced and applications of the technique in engineering are overviewed.

[1]  D. York.  Least-squares fitting of a straight line. , 1966 .

[2]  G. Golub,et al.  Regularized Total Least Squares Based on Quadratic Eigenvalue Problem Solvers , 2004 .

[3]  James Durbin,et al.  Errors in variables , 1954 .

[4]  P. Sprent.  Models in regression and related topics , 1971 .

[5]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[6]  T. W. Anderson.  Estimating Linear Statistical Relationships , 1984 .

[7]  S. S. Wilks,et al.  Linear Regression Analysis of Economic Time Series. , 1938 .

[8]  Sabine Van Huffel,et al.  Recent advances in total least squares techniques and errors-in-variables modeling , 1997 .

[9]  I. Markovsky,et al.  Consistent estimation in the bilinear multivariate errors-in-variables model , 2003 .

[10]  P. Sprent,et al.  A Generalized Least‐Squares Approach to Linear Functional Relationships , 1966 .

[11]  K. Fernando,et al.  Identification of linear systems with input and output noise: the Koopmans-Levin method , 1985 .

[12]  L. Gleser.  Estimation in a Multivariate "Errors in Variables" Regression Model: Large Sample Results , 1981 .

[13]  D. Ruppert,et al.  Measurement Error in Nonlinear Models , 1995 .

[14]  D. V. Lindley,et al.  Regression Lines and the Linear Functional Relationship , 1947 .

[15]  Gene H. Golub,et al.  Matrix computations , 1983 .

[16]  M. Levin.  Estimation of a system pulse transfer function in the presence of noise , 1964 .

[17]  S. Huffel,et al.  Total Least Squares and Errors-in-Variables Modeling : Analysis, Algorithms and Applications , 2002 .

[18]  Herman Van der Auweraer,et al.  A comparison of some frequency response function measurement techniques , 1986 .

[19]  K. S. Arun,et al.  A Unitarily Constrained Total Least Squares Problem in Signal Processing , 1992, SIAM J. Matrix Anal. Appl..

[20]  Darren T. Andrews,et al.  Maximum likelihood principal component analysis , 1997 .

[21]  Sabine Van Huffel,et al.  Total least squares problem - computational aspects and analysis , 1991, Frontiers in applied mathematics.

[22]  Eric M. Dowling,et al.  The Data Least Squares Problem and Channel Equalization , 1993, IEEE Trans. Signal Process..

[23]  Eric M. Dowling,et al.  The equivalence of the total least squares and minimum norm methods [signal processing] , 1991, IEEE Trans. Signal Process..

[24]  R. J. Adcock.  Note on the Method of Least Squares , 1877 .

[25]  D. O’Leary,et al.  Implementation of the regularized structured total least squares algorithms for blind image deblurring , 2004 .

[26]  Sabine Van Huffel,et al.  Consistency of elementwise-weighted total least squares estimator in a multivariate errors-in-variables model AX=B , 2004 .

[27]  A. Madansky.  The fitting of straight lines when both variables are subject to error , 1959 .

[28]  Jerry M. Mendel,et al.  The constrained total least squares technique and its applications to harmonic superresolution , 1991, IEEE Trans. Signal Process..

[29]  T. Söderström,et al.  Bias correction in least-squares identification , 1982 .

[30]  J. Vandewalle,et al.  Analysis and solution of the nongeneric total least squares problem , 1988 .

[31]  Gene H. Golub,et al.  Regularization by Truncated Total Least Squares , 1997, SIAM J. Sci. Comput..

[32]  Sudhir Gupta,et al.  Statistical Regression With Measurement Error , 1999, Technometrics.

[33]  R. J. Adcock.  A Problem in Least Squares , 1878 .