Hybrid and Iteratively Reweighted Regularization by Unbiased Predictive Risk and Weighted GCV for Projected Systems

Tikhonov regularization for projected solutions of large-scale ill-posed problems is considered. The Golub-Kahan iterative bidiagonalization is used to project the problem onto a subspace and regularization then applied to find a subspace approximation to the full problem. Determination of the regularization parameter using the method of unbiased predictive risk estimation is considered and contrasted with the generalized cross validation and discrepancy principle techniques. Examining the unbiased predictive risk estimator for the projected problem, it is shown that the obtained regularized parameter provides a good estimate for that to be used for the full problem with the solution found on the projected space. The connection between regularization for full and projected systems for the discrepancy and generalized cross validation estimators is also discussed and an argument for the weight parameter in the weighted generalized cross validation approach is provided. All results are independent of whether systems are over or underdetermined, the latter of which has not been considered in discussions of regularization parameter estimation for projected systems. Numerical simulations for standard one dimensional test problems and two dimensional data for both image restoration and tomographic image reconstruction support the analysis and validate the techniques. The size of the projected problem is found using an extension of a noise revealing function for the projected problem. Furthermore, an iteratively reweighted regularization approach for edge preserving regularization is extended for projected systems, providing stabilization of the solutions of the projected systems with respect to the determination of the size of the projected subspace.

[1]  Misha Elena Kilmer,et al.  A Framework for Regularization via Operator Approximation , 2015, SIAM J. Sci. Comput..

[2]  Zhongxiao Jia,et al.  Some results on the regularization of LSQR for large-scale discrete ill-posed problems , 2015, 1503.01864.

[3]  Esa Niemi,et al.  Tomographic X-ray data of a walnut , 2015, 1502.04064.

[4]  Marco Donatelli,et al.  Square smoothing regularization matrices with accurate boundary conditions , 2014, J. Comput. Appl. Math..

[5]  Rosemary A. Renaut,et al.  Application of the χ2 principle and unbiased predictive risk estimator for determining the regularization parameter in 3-D focusing gravity inversion , 2014, 1408.0712.

[6]  Per Christian Hansen,et al.  A computationally efficient tool for assessing the depth resolution in large-scale potential-field inversion , 2014 .

[7]  R. Renaut,et al.  Regularization parameter estimation for underdetermined problems by the χ 2 principle with application to 2D focusing gravity inversion , 2014, 1402.3365.

[8]  Rosemary A. Renaut,et al.  Automatic estimation of the regularization parameter in 2-D focusing gravity inversion: an application to the Safo manganese mine in northwest of Iran , 2013, ArXiv.

[9]  Lothar Reichel,et al.  Tikhonov regularization based on generalized Krylov subspace methods , 2012 .

[10]  Dianne P. O'Leary,et al.  Windowed Spectral Regularization of Inverse Problems , 2011, SIAM J. Sci. Comput..

[11]  Michiel E. Hochstenbach,et al.  An iterative method for Tikhonov regularization with a general linear regularization operator , 2010 .

[12]  Michael A. Saunders,et al.  LSMR: An Iterative Algorithm for Sparse Least-Squares Problems , 2010, SIAM J. Sci. Comput..

[13]  Zdeněk Strakoš,et al.  The regularizing effect of the Golub-Kahan iterative bidiagonalization and revealing the noise level in the data , 2009 .

[14]  Brendt Wohlberg,et al.  An Iteratively Reweighted Norm Algorithm for Minimization of Total Variation Functionals , 2007, IEEE Signal Processing Letters.

[15]  Dianne P. O'Leary,et al.  Deblurring Images: Matrices, Spectra and Filtering , 2006, J. Electronic Imaging.

[16]  James G. Nagy,et al.  Iterative Methods for Image Deblurring: A Matlab Object-Oriented Approach , 2004, Numerical Algorithms.

[17]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[18]  Richard G. Baraniuk,et al.  ForWaRD: Fourier-wavelet regularized deconvolution for ill-conditioned systems , 2004, IEEE Transactions on Signal Processing.

[19]  Misha Elena Kilmer,et al.  Choosing Regularization Parameters in Iterative Methods for Ill-Posed Problems , 2000, SIAM J. Matrix Anal. Appl..

[20]  Michael S. Zhdanov,et al.  Focusing geophysical inversion images , 1999 .

[21]  Per Christian Hansen,et al.  REGULARIZATION TOOLS: A Matlab package for analysis and solution of discrete ill-posed problems , 1994, Numerical Algorithms.

[22]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[23]  M. Saunders,et al.  Towards a Generalized Singular Value Decomposition , 1981 .

[24]  Gene H. Golub,et al.  Generalized cross-validation as a method for choosing a good ridge parameter , 1979, Milestones in Matrix Computation.

[25]  KEIJO HÄMÄLÄINEN,et al.  Sparse Tomography , 2013, SIAM J. Sci. Comput..

[26]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[27]  P. Hansen,et al.  Noise propagation in regularizing iterations for image deblurring , 2008 .

[28]  Per Christian Hansen,et al.  A Matlab Package for Analysis and Solution of Discrete Ill-Posed Problems , 2008 .

[29]  J. Nagy,et al.  A weighted-GCV method for Lanczos-hybrid regularization. , 2007 .

[30]  Jianhong Shen,et al.  Deblurring images: Matrices, spectra, and filtering , 2007, Math. Comput..

[31]  Per Christian Hansen,et al.  Rank-Deficient and Discrete Ill-Posed Problems , 1996 .

[32]  Per Christian Hansen,et al.  Regularization methods for large-scale problems , 1993 .

[33]  P. Hansen.  Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion , 1987 .

[34]  C. Vogel.  Computational Methods for Inverse Problems , 1987 .

[35]  M. Saunders,et al.  Algorithm 583: LSQR: Sparse Linear Equations and Least Squares Problems , 1982, ACM Trans. Math. Softw..

[36]  V. Morozov.  On the solution of functional equations by the method of regularization , 1966 .

[37]  Xudong Sun,et al.  Regularization parameter estimation for large-scale Tikhonov regularization using a priori information , 2010, Comput. Stat. Data Anal..