Adaptive choice of the Tikhonov regularization parameter to solve ill-posed linear algebraic equations via Liapunov Optimizing Control

Numerical problems, including linear algebraic equations, are frequently ill-posed, which means that small perturbations in the observation data can have large effects on the computed solutions. In these cases some kind of regularization method must be used to find a feasible solution. Regularization implies the modification of the original ill-posed problem, so that it becomes well-posed, and the Tikhonov regularization method is the most often used. This method requires the choice of a regularization parameter, and this choice is a crucial issue. Recently, several papers were presented proposing adaptive Tikhonov regularization parameters, instead of constant ones. This paper revisits all these adaptive choices and shows that they can be interpreted in a unified and systematic manner using the Liapunov Optimizing Control (LOC) method, which, in addition, opens up the possibility of new choices of the regularization parameter. Examples of matrices from real applications show that one of the methods designed by the LOC method, called the OVM method by its discoverer, can even outperform the CG method for some matrices, although the CG method performs better for a larger class of matrices.

[1]  Mark A. Lukas,et al.  Comparing parameter choice methods for regularization of ill-posed problems , 2011, Math. Comput. Simul..

[2]  Chein-Shan Liu,et al.  Optimally scaled vector regularization method to solve ill-posed linear problems , 2012, Appl. Math. Comput..

[3]  Gene H. Golub,et al.  Generalized cross-validation as a method for choosing a good ridge parameter , 1979, Milestones in Matrix Computation.

[4]  Per Christian Hansen,et al.  REGULARIZATION TOOLS: A Matlab package for analysis and solution of discrete ill-posed problems , 1994, Numerical Algorithms.

[5]  Paul Tseng.  Control perspectives on numerical algorithms and matrix problems , 2008, Math. Comput..

[6]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[7]  ProblemsPer Christian HansenDepartment.  The L-curve and its use in the numerical treatment of inverse problems , 2000 .

[8]  Eugenius Kaszkurewicz,et al.  Iterative methods as dynamical systems with feedback control , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[9]  U. Ascher,et al.  On fast integration to steady state and earlier times , 2008 .

[10]  Jin Cheng,et al.  Regularization Methods for Ill-Posed Problems , 2015, Handbook of Mathematical Methods in Imaging.

[11]  Dianne P. O'Leary,et al.  Deblurring Images: Matrices, Spectra, and Filtering (Fundamentals of Algorithms 3) (Fundamentals of Algorithms) , 2006 .

[12]  Anil K. Jain.  Fundamentals of Digital Image Processing , 2018, Control of Color Imaging Systems.

[13]  S. Atluri,et al.  Novel Algorithms Based on the Conjugate Gradient Method for Inverting Ill-Conditioned Matrices, and a New Regularization Method to Solve Ill-Posed Linear Systems , 2010 .

[14]  U. Ascher,et al.  Esaim: Mathematical Modelling and Numerical Analysis Gradient Descent and Fast Artificial Time Integration , 2022 .

[15]  Chein-Shan Liu.  A Dynamical Tikhonov Regularization for Solving Ill-posed Linear Algebraic Systems , 2013 .

[16]  Dianne P. O'Leary,et al.  The Use of the L-Curve in the Regularization of Discrete Ill-Posed Problems , 1993, SIAM J. Sci. Comput..

[17]  H. Gfrerer.  An a posteriori parameter choice for ordinary and iterated Tikhonov regularization of ill-posed problems leading to optimal convergence rates , 1987 .

[18]  Eugenius Kaszkurewicz,et al.  Control Perspectives on Numerical Algorithms And Matrix Problems (Advances in Design and Control) (Advances in Design and Control 10) , 2006 .

[19]  Marek Rudnicki,et al.  Regularization Parameter Selection in Discrete Ill-Posed Problems — The Use of the U-Curve , 2007, Int. J. Appl. Math. Comput. Sci..

[20]  Misha Elena Kilmer,et al.  Choosing Regularization Parameters in Iterative Methods for Ill-Posed Problems , 2000, SIAM J. Matrix Anal. Appl..

[21]  Pierre-Alexandre Bliman,et al.  Control-theoretic design of iterative methods for symmetric linear systems of equations , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[22]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[23]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[24]  Mostafa Kaveh,et al.  Regularization and image restoration using differential equations , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[25]  S. Atluri,et al.  An Iterative Method Using an Optimal Descent Vector, for Solving an Ill-Conditioned System B x = b , Better and Faster than the Conjugate Gradient Method , 2011 .

[26]  Dianne P. O'Leary,et al.  Deblurring Images: Matrices, Spectra and Filtering , 2006, J. Electronic Imaging.