Structured condition numbers of structured Tikhonov regularization problem and their estimations

Both structured componentwise and structured normwise perturbation analysis of the Tikhonov regularization are presented. The structured matrices under consideration include: Toeplitz, Hankel, Vandermonde, and Cauchy matrices. Structured normwise, mixed and componentwise condition numbers for the Tikhonov regularization are introduced and their explicit expressions are derived. For the general linear structure, based on the derived expressions, we prove structured condition numbers are smaller than their corresponding unstructured counterparts. By means of the power method and small sample statistical condition estimation (SCE), fast condition estimation algorithms are proposed. Our estimation methods can be integrated into Tikhonov regularization algorithms that use the generalized singular value decomposition (GSVD). For large scale linear structured Tikhonov regularization problems, we show how to incorporate the SCE into the preconditioned conjugate gradient (PCG) method to get the posterior error estimations. The structured condition numbers and perturbation bounds are tested on some numerical examples and compared with their unstructured counterparts. Our numerical examples demonstrate that the structured mixed condition numbers give sharper perturbation bounds than existing ones, and the proposed condition estimation algorithms are reliable. Also, an image restoration example is tested to show the effectiveness of the SCE for large scale linear structured Tikhonov regularization problems.

[1]  A. Laub,et al.  Statistical Condition Estimation for Linear Least Squares , 1998, SIAM J. Matrix Anal. Appl..

[2]  A. N. Tikhonov,et al.  REGULARIZATION OF INCORRECTLY POSED PROBLEMS , 1963 .

[3]  Yimin Wei,et al.  On mixed and componentwise condition numbers for Moore-Penrose inverse and linear least squares problems , 2006, Math. Comput..

[4]  Yimin Wei,et al.  Condition numbers and perturbation analysis for the Tikhonov regularization of discrete ill‐posed problems , 2011, Numer. Linear Algebra Appl..

[5]  Hua Xiang,et al.  Structured mixed and componentwise condition numbers of some structured matrices , 2007 .

[6]  Nicholas J. Higham,et al.  FORTRAN codes for estimating the one-norm of a real or complex matrix, with applications to condition estimation , 1988, TOMS.

[7]  G. Wahba.  Spline models for observational data , 1990 .

[8]  Haesun Park,et al.  Schur-Type Methods for Solving Least Squares Problems with Toeplitz Structure , 2000, SIAM J. Sci. Comput..

[9]  Albrecht Böttcher,et al.  Structured condition numbers of large Toeplitz matrices are rarely better than usual condition numbers , 2005, Numer. Linear Algebra Appl..

[10]  Alan J. Laub,et al.  Small-Sample Statistical Condition Estimates for General Matrix Functions , 1994, SIAM J. Sci. Comput..

[11]  P. Hansen.  Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion , 1987 .

[12]  Serge Gratton,et al.  Using dual techniques to derive componentwise and mixed condition numbers for a linear function of a linear least squares solution , 2009 .

[13]  Hiroshi Sekigawa,et al.  The Ratio Between the Toeplitz and the Unstructured Condition Number , 2010 .

[14]  P. Hansen.  Regularization,GSVD and truncatedGSVD , 1989 .

[15]  Nicholas J. Higham,et al.  Backward Error and Condition of Structured Linear Systems , 1992, SIAM J. Matrix Anal. Appl..

[16]  Per Christian Hansen,et al.  Rank-Deficient and Discrete Ill-Posed Problems , 1996 .

[17]  P. Hansen.  Discrete Inverse Problems: Insight and Algorithms , 2010 .

[18]  Serge Gratton,et al.  A Partial Condition Number for Linear Least Squares Problems , 2007, SIAM J. Matrix Anal. Appl..

[19]  Yimin Wei,et al.  Perturbation Identities for Regularized Tikhonov Inverses and Weighted Pseudoinverses , 2000 .

[20]  Israel Koltracht,et al.  Mixed componentwise and structured condition numbers , 1993 .

[21]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[22]  C. Loan.  Generalizing the Singular Value Decomposition , 1976 .

[23]  Thorkell Gudmundsson.  Small-Sample Statistical Estimates for the Sensitivity of Eigenvalue Problems , 1997 .

[24]  Siegfried M. Rump,et al.  Structured Perturbations Part I: Normwise Distances , 2003, SIAM J. Matrix Anal. Appl..

[25]  M. Ng.  Iterative Methods for Toeplitz Systems , 2004 .

[26]  V. A. Morozov,et al.  Methods for Solving Incorrectly Posed Problems , 1984 .

[27]  Alexander N. Malyshev.  A Unified Theoryof Conditioning for Linear Least Squares and Tikhonov Regularization Solutions , 2003, SIAM J. Matrix Anal. Appl..

[28]  Fu-Rong Lin,et al.  BTTB preconditioners for BTTB least squares problems , 2011 .

[29]  Daniel B. Szyld,et al.  An introduction to iterative Toeplitz solvers , 2009, Math. Comput..

[30]  Nicholas J. Higham,et al.  Experience with a Matrix Norm Estimator , 1990, SIAM J. Sci. Comput..

[31]  Raymond H. Chan,et al.  Conjugate Gradient Methods for Toeplitz Systems , 1996, SIAM Rev..

[32]  Julien Langou,et al.  Computing the conditioning of the components of a linear least-squares solution , 2009, Numer. Linear Algebra Appl..

[33]  Lars Eldén,et al.  An Efficient Algorithm for the Regularization of Ill-Conditioned Least Squares Problems with Triangular Toeplitz Matrix , 1984 .

[34]  L. Eldén,et al.  Stability analysis and fast algorithms for triangularization of Toeplitz matrices , 1997 .

[35]  J. Nagy,et al.  Circulant Preconditioned Toeplitz Least Squares Iterations , 1994, SIAM J. Matrix Anal. Appl..

[36]  Per Christian Hansen,et al.  Deconvolution and Regularization with Toeplitz Matrices , 2002, Numerical Algorithms.

[37]  J. Nagy,et al.  FFT-based preconditioners for Toeplitz-block least squares problems , 1993 .

[38]  D. Fasino,et al.  A Priori Estimates on the Structured Conditioning of Cauchy and Vandermonde Matrices , 2010 .

[39]  Yimin Wei,et al.  Condition Numbers for Structured Least Squares Problems , 2006 .

[40]  Jianlin Xia,et al.  Fast Condition Estimation for a Class of Structured Eigenvalue Problems , 2008, SIAM J. Matrix Anal. Appl..

[41]  Michael K. Ng.  Iterative Methods for Toeplitz Systems (Numerical Mathematics and Scientific Computation) , 2004 .

[42]  Sven G. Bartels,et al.  The structured sensitivity of Vandermonde-like systems , 1992 .

[43]  Jianlin Xia,et al.  Statistical Condition Estimation for the Roots of Polynomials , 2008, SIAM J. Sci. Comput..

[44]  Felipe Cucker,et al.  Mixed and componentwise condition numbers for rectangular structured matrices , 2007 .

[45]  Alexander Graham,et al.  Kronecker Products and Matrix Calculus: With Applications , 1981 .

[46]  Jianlin Xia,et al.  Applications of statistical condition estimation to the solution of linear systems , 2008, Numer. Linear Algebra Appl..

[47]  Per Christian Hansen,et al.  Regularization Tools version 4.0 for Matlab 7.3 , 2007, Numerical Algorithms.

[48]  Raymond H. Chan,et al.  An Introduction to Iterative Toeplitz Solvers (Fundamentals of Algorithms) , 2007 .

[49]  Dario Bini,et al.  Numerical Methods for Structured Matrices and Applications , 2010 .

[50]  Michael K. Ng,et al.  Preconditioned Iterative Methods for Weighted Toeplitz Least Squares Problems , 2005, SIAM J. Matrix Anal. Appl..

[51]  Siegfried M. Rump,et al.  Structured Perturbations Part II: Componentwise Distances , 2003, SIAM J. Matrix Anal. Appl..

[52]  P. Hansen.  Perturbation bounds for discrete Tikhonov regularisation , 1989 .

[53]  Nicholas J. Higham,et al.  INVERSE PROBLEMS NEWSLETTER , 1991 .

[54]  Alan J. Laub,et al.  Statistical Condition Estimation for Linear Systems , 1998, SIAM J. Sci. Comput..