Parameter selection for total-variation-based image restoration using discrepancy principle

There are two key issues in successfully solving the image restoration problem: 1) estimation of the regularization parameter that balances data fidelity with the regularity of the solution and 2) development of efficient numerical techniques for computing the solution. In this paper, we derive a fast algorithm that simultaneously estimates the regularization parameter and restores the image. The new approach is based on the total-variation (TV) regularized strategy and Morozov's discrepancy principle. The TV norm is represented by the dual formulation that changes the minimization problem into a minimax problem. A proximal point method is developed to compute the saddle point of the minimax problem. By adjusting the regularization parameter adaptively in each iteration, the solution is guaranteed to satisfy the discrepancy principle. We will give the convergence proof of our algorithm and numerically show that it is better than some state-of-the-art methods in terms of both speed and accuracy.

[1]  Junfeng Yang,et al.  ALTERNATING DIRECTION ALGORITHMS FOR TOTAL VARIATION DECONVOLUTION IN IMAGE RECONSTRUCTION , 2009 .

[2]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[3]  Mila Nikolova,et al.  Efficient Minimization Methods of Mixed l2-l1 and l1-l1 Norms for Image Restoration , 2005, SIAM J. Sci. Comput..

[4]  C. Atkinson METHODS FOR SOLVING INCORRECTLY POSED PROBLEMS , 1985 .

[5]  Stéphane Mallat,et al.  A Wavelet Tour of Signal Processing, 2nd Edition , 1999 .

[6]  Ke Chen,et al.  A Nonlinear Multigrid Method for Total Variation Minimization from Image Restoration , 2007, J. Sci. Comput..

[7]  D. Krishnan,et al.  An Efficient Operator-Splitting Method for Noise Removal in Images , 2006 .

[8]  P. L. Combettes,et al.  Solving monotone inclusions via compositions of nonexpansive averaged operators , 2004 .

[9]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[10]  M. Hanke Limitations of the L-curve method in ill-posed problems , 1996 .

[11]  Junfeng Yang,et al.  An Efficient TVL1 Algorithm for Deblurring Multichannel Images Corrupted by Impulsive Noise , 2009, SIAM J. Sci. Comput..

[12]  Per Christian Hansen,et al.  Analysis of Discrete Ill-Posed Problems by Means of the L-Curve , 1992, SIAM Rev..

[13]  C. Vogel Non-convergence of the L-curve regularization parameter selection method , 1996 .

[14]  Mário A. T. Figueiredo,et al.  Fast frame-based image deconvolution using variable splitting and constrained optimization , 2009, 2009 IEEE/SP 15th Workshop on Statistical Signal Processing.

[15]  E. Beckenbach CONVEX FUNCTIONS , 2007 .

[16]  P. Tseng Applications of splitting algorithm to decomposition in convex programming and variational inequalities , 1991 .

[17]  Dianne P. O'Leary,et al.  The Use of the L-Curve in the Regularization of Discrete Ill-Posed Problems , 1993, SIAM J. Sci. Comput..

[18]  R. Rockafellar Monotone Operators and the Proximal Point Algorithm , 1976 .

[19]  Emmanuel J. Candès,et al.  NESTA: A Fast and Accurate First-Order Method for Sparse Recovery , 2009, SIAM J. Imaging Sci..

[20]  R. Tyrrell Rockafellar,et al.  Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming , 1976, Math. Oper. Res..

[21]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[22]  C. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[23]  Dimitri P. Bertsekas,et al.  Convex Analysis and Optimization , 2003 .

[24]  P. Laguna,et al.  Signal Processing , 2002, Yearbook of Medical Informatics.

[25]  Gene H. Golub,et al.  Generalized cross-validation as a method for choosing a good ridge parameter , 1979, Milestones in Matrix Computation.

[26]  Tony F. Chan,et al.  Modular solvers for image restoration problems using the discrepancy principle , 2002, Numer. Linear Algebra Appl..

[27]  Gene H. Golub,et al.  A Nonlinear Primal-Dual Method for Total Variation-Based Image Restoration , 1999, SIAM J. Sci. Comput..

[28]  D. Varberg Convex Functions , 1973 .

[29]  P. Hall,et al.  Common Structure of Techniques for Choosing Smoothing Parameters in Regression Problems , 1987 .

[30]  Marc Teboulle,et al.  A proximal-based decomposition method for convex minimization problems , 1994, Math. Program..

[31]  Junfeng Yang,et al.  A Fast Algorithm for Edge-Preserving Variational Multichannel Image Restoration , 2009, SIAM J. Imaging Sci..

[32]  Jérôme Darbon,et al.  Image Restoration with Discrete Constrained Total Variation Part I: Fast and Exact Optimization , 2006, Journal of Mathematical Imaging and Vision.

[33]  Truong Q. Nguyen,et al.  An Augmented Lagrangian Method for Total Variation Video Restoration , 2011, IEEE Transactions on Image Processing.

[34]  Aggelos K. Katsaggelos,et al.  Parameter Estimation in TV Image Restoration Using Variational Distribution Approximation , 2008, IEEE Transactions on Image Processing.

[35]  Raymond H. Chan,et al.  Conjugate Gradient Methods for Toeplitz Systems , 1996, SIAM Rev..

[36]  S. Osher,et al.  Nonlinear inverse scale space methods , 2006 .

[37]  S. Mallat A wavelet tour of signal processing , 1998 .

[38]  Aggelos K. Katsaggelos,et al.  Variational Bayesian Blind Deconvolution Using a Total Variation Prior , 2009, IEEE Transactions on Image Processing.

[39]  José M. Bioucas-Dias,et al.  Adaptive total variation image deconvolution: A majorization-minimization approach , 2006, 2006 14th European Signal Processing Conference.

[40]  Gilles Aubert,et al.  Efficient Schemes for Total Variation Minimization Under Constraints in Image Processing , 2009, SIAM J. Sci. Comput..

[41]  H. Engl,et al.  Using the L--curve for determining optimal regularization parameters , 1994 .

[42]  Mingqiang Zhu,et al.  An Efficient Primal-Dual Hybrid Gradient Algorithm For Total Variation Image Restoration , 2008 .

[43]  Raymond H. Chan,et al.  A Fast Algorithm for Deblurring Models with Neumann Boundary Conditions , 1999, SIAM J. Sci. Comput..

[44]  José M. Bioucas-Dias,et al.  An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems , 2009, IEEE Transactions on Image Processing.

[45]  Fang Li,et al.  Selection of regularization parameter in total variation image restoration. , 2009, Journal of the Optical Society of America. A, Optics, image science, and vision.

[46]  T. Chan,et al.  Fast numerical algorithms for total variation based image restoration , 2008 .

[47]  Mohamed-Jalal Fadili,et al.  A Proximal Iteration for Deconvolving Poisson Noisy Images Using Sparse Representations , 2008, IEEE Transactions on Image Processing.

[48]  Curtis R. Vogel,et al.  Iterative Methods for Total Variation Denoising , 1996, SIAM J. Sci. Comput..

[49]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[50]  Lothar Reichel,et al.  A new zero-finder for Tikhonov regularization , 2008 .

[51]  M. Nikolova An Algorithm for Total Variation Minimization and Applications , 2004 .

[52]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[53]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[54]  Ke Chen,et al.  An Optimization-Based Multilevel Algorithm for Total Variation Image Denoising , 2006, Multiscale Model. Simul..

[55]  P. L. Combettes,et al.  A proximal decomposition method for solving convex variational inverse problems , 2008, 0807.2617.

[56]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[57]  Michael K. Ng,et al.  Solving Constrained Total-variation Image Restoration and Reconstruction Problems via Alternating Direction Methods , 2010, SIAM J. Sci. Comput..

[58]  Jérôme Darbon,et al.  A Fast and Exact Algorithm for Total Variation Minimization , 2005, IbPRIA.

[59]  Guy Demoment,et al.  Image reconstruction and restoration: overview of common estimation structures and problems , 1989, IEEE Trans. Acoust. Speech Signal Process..

[60]  Guy Gilboa,et al.  Constrained and SNR-Based Solutions for TV-Hilbert Space Image Denoising , 2006, Journal of Mathematical Imaging and Vision.

[61]  José M. Bioucas-Dias,et al.  Adaptive total variation image deblurring: A majorization-minimization approach , 2009, Signal Process..

[62]  B. R. Hunt,et al.  Digital Image Restoration , 1977 .

[63]  D. Calvetti,et al.  Tikhonov Regularization of Large Linear Problems , 2003 .

[64]  Michael K. Ng,et al.  Fast Image Restoration Methods for Impulse and Gaussian Noises Removal , 2009, IEEE Signal Processing Letters.

[65]  Brendt Wohlberg,et al.  UPRE method for total variation parameter selection , 2010, Signal Process..

[66]  Zita Poracká-Diviš Existence Theorem and Convergence of Minimizing Sequences in Extremum Problems , 1971 .

[67]  M. Mohammadzadeh,et al.  Global optimization of the generalized cross-validation criterion , 2000, Stat. Comput..

[68]  D. M. Titterington,et al.  Choosing the regularization parameter in image restoration , 1991 .

[69]  Nikolas P. Galatsanos,et al.  Methods for choosing the regularization parameter and estimating the noise variance in image restoration and their relation , 1992, IEEE Trans. Image Process..

[70]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.