Level choice in truncated total least squares

The method of truncated total least squares (TTLS) is an alternative to the classical truncated singular value decomposition (TSVD) used for the regularization of ill-conditioned linear systems. Truncation methods aim at limiting the contribution of noise or rounding errors by cutting off a certain number of terms in an expansion such as the singular value decomposition. To this end a truncation level k must be carefully chosen. The TTLS solution becomes more significantly dominated by noise or errors when the truncation level k is overestimated than the TSVD solution does. Model selection methods that are often applied in the context of the TSVD are modified to be applied in the context of the TTLS. The proposed modified generalized cross validation (GCV) combined with the TTLS method performs better than the classical GCV combined with the TSVD, especially, when both the coefficient matrix and the right-hand side are contaminated by noise.

[1]  J. Varah.  Pitfalls in the Numerical Solution of Linear Ill-Posed Problems , 1981 .

[2]  Zdenek Strakos,et al.  Core Problems in Linear Algebraic Systems , 2005, SIAM J. Matrix Anal. Appl..

[3]  John E. Moody,et al.  The Effective Number of Parameters: An Analysis of Generalization and Regularization in Nonlinear Learning Systems , 1991, NIPS.

[4]  Sabine Van Huffel,et al.  Total least squares problem - computational aspects and analysis , 1991, Frontiers in applied mathematics.

[5]  Gene H. Golub,et al.  Regularization by Truncated Total Least Squares , 1997, SIAM J. Sci. Comput..

[6]  Gene H. Golub,et al.  An analysis of the total least squares problem , 1980, Milestones in Matrix Computation.

[7]  Rudolf Beran,et al.  ADAPTIVE ESTIMATORS OF A MEAN MATRIX: TOTAL LEAST SQUARES VERSUS TOTAL SHRINKAGE , 2008, Econometric Theory.

[8]  H. Akaike.  A new look at the statistical model identification , 1974 .

[9]  Gene H. Golub,et al.  Matrix computations , 1983 .

[10]  Gene H. Golub,et al.  Generalized cross-validation as a method for choosing a good ridge parameter , 1979, Milestones in Matrix Computation.

[11]  Per Christian Hansen,et al.  REGULARIZATION TOOLS: A Matlab package for analysis and solution of discrete ill-posed problems , 1994, Numerical Algorithms.

[12]  G. Kitagawa,et al.  Generalised information criteria in model selection , 1996 .

[13]  Per Christian Hansen,et al.  Rank-Deficient and Discrete Ill-Posed Problems , 1996 .

[14]  Per Christian Hansen,et al.  Analysis of Discrete Ill-Posed Problems by Means of the L-Curve , 1992, SIAM Rev..

[15]  Teresa Reginska,et al.  A Regularization Parameter in Discrete Ill-Posed Problems , 1996, SIAM J. Sci. Comput..

[16]  Thomas Kailath,et al.  Detection of signals by information theoretic criteria , 1985, IEEE Trans. Acoust. Speech Signal Process..

[17]  P. Hansen,et al.  Exploiting Residual Information in the Parameter Choice for Discrete Ill-Posed Problems , 2006 .

[18]  P. Hansen.  Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion , 1987 .

[19]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[20]  V. Morozov.  On the solution of functional equations by the method of regularization , 1966 .

[21]  Sabine Van Huffel,et al.  The total least squares problem , 1993 .

[22]  G. Wahba.  Spline models for observational data , 1990 .