Efficient Minimization Methods of Mixed l2-l1 and l1-l1 Norms for Image Restoration

Image restoration problems are often solved by finding the minimizer of a suitable objective function. Usually this function consists of a data-fitting term and a regularization term. For the least squares solution, both the data-fitting and the regularization terms are in the $\ell$2 norm. In this paper, we consider the least absolute deviation (LAD) solution and the least mixed norm (LMN) solution. For the LAD solution, both the data-fitting and the regularization terms are in the $\ell$1 norm. For the LMN solution, the regularization term is in the $\ell$1 norm but the data-fitting term is in the $\ell$2 norm. Since images often have nonnegative intensity values, the proposed algorithms provide the option of taking into account the nonnegativity constraint. The LMN and LAD solutions are formulated as the solution to a linear or quadratic programming problem which is solved by interior point methods. At each iteration of the interior point method, a structured linear system must be solved. The preconditioned conjugate gradient method with factorized sparse inverse preconditioners is employed to solve such structured inner systems. Experimental results are used to demonstrate the effectiveness of our approach. We also show the quality of the restored images, using the minimization of mixed $\ell$2-$\ell$1 and $\ell1$-$\ell$1 norms, is better than that using only the $\ell$2 norm.

[1]  M. Saunders,et al.  Solution of Sparse Indefinite Systems of Linear Equations , 1975 .

[2]  N. Abdelmalek,et al.  Digital image restoration using quadratic programming. , 1980, Applied optics.

[3]  G Eichmann,et al.  Superresolving image restoration using linear programming. , 1982, Applied optics.

[4]  N. Abdelmalek.  Restoration of images with missing high-frequency components using quadratic programming. , 1983, Applied optics.

[5]  W. Steiger,et al.  Least Absolute Deviations: Theory, Applications and Algorithms , 1984 .

[6]  N. Otsu,et al.  Restoration of images with missing high-frequency components by minimizing the L1 norm of the solution vector. , 1985, Applied optics.

[7]  Y. Saad,et al.  GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems , 1986 .

[8]  A. Dax.  A row relaxation method for large l1 problems , 1991 .

[9]  Achiya Dax.  On Regularized Least Norm Problems , 1992, SIAM J. Optim..

[10]  Richard J. Mammone,et al.  Image restoration by convex projections using adaptive constraints and the L1 norm , 1992, IEEE Trans. Signal Process..

[11]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[12]  C. Loan.  Computational Frameworks for the Fast Fourier Transform , 1992 .

[13]  Toyokatsu Miyashita.  Superresolved Image Restoration of Holographic Images by L1-Norm Minimization with Clutter Rejection , 1992 .

[14]  Stefano Alliney,et al.  Digital filters as absolute norm regularizers , 1992, IEEE Trans. Signal Process..

[15]  L. Kolotilina,et al.  Factorized Sparse Approximate Inverse Preconditionings I. Theory , 1993, SIAM J. Matrix Anal. Appl..

[16]  Achiya Dax,et al.  On Row Relaxation Methods for Large Constrained Least Squares Problems , 1993, SIAM J. Sci. Comput..

[17]  Stefano Alliney,et al.  An algorithm for the minimization of mixed l1 and l2 norms with application to Bayesian estimation , 1994, IEEE Trans. Signal Process..

[18]  Achiya Dax,et al.  A row relaxation method for large ℓp least norm problems , 1994, Numer. Linear Algebra Appl..

[19]  Nikolas P. Galatsanos,et al.  Regularized constrained total least-squares image restoration , 1994, Other Conferences.

[20]  Yuying Li,et al.  A computational algorithm for minimizing total variation in image restoration , 1996, IEEE Trans. Image Process..

[21]  Michele Benzi,et al.  A Sparse Approximate Inverse Preconditioner for the Conjugate Gradient Method , 1996, SIAM J. Sci. Comput..

[22]  K. Kunisch,et al.  Regularization of linear least squares problems by total bounded variation , 1997 .

[23]  Stephen J. Wright.  Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[24]  Stefano Alliney,et al.  A property of the minimum vectors of a regularizing functional defined by means of the absolute norm , 1997, IEEE Trans. Signal Process..

[25]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[26]  K. Kunisch,et al.  An active set strategy based on the augmented Lagrangian formulation for image restoration , 1999 .

[27]  Raymond H. Chan,et al.  A Fast Algorithm for Deblurring Models with Neumann Boundary Conditions , 1999, SIAM J. Sci. Comput..

[28]  Wei-Pai Tang,et al.  Toward an Effective Sparse Approximate Inverse Preconditioner , 1999, SIAM J. Matrix Anal. Appl..

[29]  M. Ng,et al.  A new approach to constrained total least squares image restoration , 2000 .

[30]  J. Nagy,et al.  Quasi-Newton approach to nonnegative image restorations , 2000 .

[31]  Delores M. Etter,et al.  Introduction to MATLAB 6 , 2001 .

[32]  Patrick L. Combettes,et al.  An adaptive level set method for nondifferentiable constrained image recovery , 2002, IEEE Trans. Image Process..

[33]  Mila Nikolova,et al.  Minimizers of Cost-Functions Involving Nonsmooth Data-Fidelity Terms. Application to the Processing of Outliers , 2002, SIAM J. Numer. Anal..

[34]  Dianne P. O'Leary,et al.  Blind Deconvolution Using a Regularized Structured Total Least Norm Algorithm , 2002, SIAM J. Matrix Anal. Appl..

[35]  Gene H. Golub,et al.  A Preconditioner for Generalized Saddle Point Problems , 2004, SIAM J. Matrix Anal. Appl..

[36]  M. Nikolova.  A Variational Approach to Remove Outliers and Impulse Noise , 2004 .

[37]  Karl Kunisch,et al.  Total Bounded Variation Regularization as a Bilaterally Constrained Optimization Problem , 2004, SIAM J. Appl. Math..

[38]  A. Guitton,et al.  Adaptive subtraction of multiples using the L1‐norm , 2004 .

[39]  Karl Kunisch,et al.  Denoising of Smooth Images Using L1-Fitting , 2005, Computing.

[40]  J. Barlow,et al.  A regularized structured total least squares algorithm for high-resolution image reconstruction , 2004 .

[41]  D. O’Leary,et al.  Implementation of the regularized structured total least squares algorithms for blind image deblurring , 2004 .

[42]  M. Nikolova.  An Algorithm for Total Variation Minimization and Applications , 2004 .

[43]  Michael K. Ng,et al.  Factorized Banded Inverse Preconditioners for Matrices with Toeplitz Structure , 2005, SIAM J. Sci. Comput..

[44]  Tony F. Chan,et al.  Aspects of Total Variation Regularized L[sup 1] Function Approximation , 2005, SIAM J. Appl. Math..