Solving Robust Regularization Problems Using Iteratively Re-weighted Least Squares

Many computer vision problems are formulated as an objective function consisting of a sum of functions. In the case of ill-constrained problems, regularization terms are included in the objective function to reduce the ambiguity and noise in the solution. The most commonly used regularization terms are the L2 norm and the L1 norm. Since the last two decades, the class of regularized problems, especially the L1-regularized problems, has received much attention but still many regularized problems are either difficult to solve, or require complex optimization techniques. We propose a method based on an Iteratively Re-weighted Least Squares approach to minimize an objective function comprising a mixture of m-estimator regularization terms. In addition to the proof of convergence of the algorithm to the desired minimum, we show the applicability of the proposed algorithm by solving the problems of edge-preserved image denoising and image super-resolution. In both the cases, our experimental results show that the proposed algorithm gives superior results to the state-of-art regularization methods.

[1]  L. Shao,et al.  From Heuristic Optimization to Dictionary Learning: A Review and Comprehensive Comparison of Image Denoising Algorithms , 2014, IEEE Transactions on Cybernetics.

[2]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[3]  Jack Brimberg,et al.  Global Convergence of a Generalized Iterative Procedure for the Minisum Location Problem with lp Distances , 1993, Oper. Res..

[4]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[5]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[6]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[7]  Xue-Cheng Tai,et al.  A Convex and Exact Approach to Discrete Constrained TV-L1 Image Approximation , 2011 .

[8]  Tony F. Chan,et al.  A General Framework for a Class of First Order Primal-Dual Algorithms for Convex Optimization in Imaging Science , 2010, SIAM J. Imaging Sci..

[9]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[10]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[11]  David G. Luenberger,et al.  Linear and Nonlinear Programming: Second Edition , 2003 .

[12]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[13]  H. Engl,et al.  Regularization of Inverse Problems , 1996 .

[14]  Jochen Trumpf,et al.  Generalized Weiszfeld Algorithms for Lq Optimization , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Suyash P. Awate,et al.  Image denoising with unsupervised, information-theoretic, adaptive filtering , 2004 .

[16]  Shmuel Peleg,et al.  Robust super-resolution , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[17]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[18]  A. N. Tikhonov,et al.  Solutions of ill-posed problems , 1977 .

[19]  Jochen Trumpf,et al.  $${L_q}$$Lq-Closest-Point to Affine Subspaces Using the Generalized Weiszfeld Algorithm , 2015, International Journal of Computer Vision.

[20]  J. Brimberg.  FURTHER NOTES ON CONVERGENCE OF THE WEISZFELD ALGORITHM , 2003 .

[21]  Damien Garcia,et al.  Robust smoothing of gridded data in one and higher dimensions with missing values , 2010, Comput. Stat. Data Anal..

[22]  Gerard de Haan,et al.  An Overview and Performance Evaluation of Classification-Based Least Squares Trained Filters , 2008, IEEE Transactions on Image Processing.

[23]  Klamer Schutte,et al.  Robust Fusion of Irregularly Sampled Data Using Adaptive Normalized Convolution , 2006, EURASIP J. Adv. Signal Process..

[24]  Bernard Ghanem,et al.  ℓ0TV: A new method for image restoration in the presence of impulse noise , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Xuelong Li,et al.  Multi-scale dictionary for single image super-resolution , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Norbert Wiener,et al.  Extrapolation, Interpolation, and Smoothing of Stationary Time Series , 1964 .

[27]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[28]  Reuven Chen,et al.  A note on convergence in the single facility minisum location problem , 1998 .

[29]  Michael Elad,et al.  A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur , 2001, IEEE Trans. Image Process..

[30]  M. Shirosaki.  Another proof of the defect relation for moving targets , 1991 .

[31]  Terrance E. Boult,et al.  Efficient super-resolution via image warping , 2000, Image Vis. Comput..

[32]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[33]  Richard I. Hartley,et al.  Convergence of Iteratively Re-weighted Least Squares to Robust M-Estimators , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[34]  Nirmal K. Bose,et al.  Recursive implementation of total least squares algorithm for image reconstruction from noisy, undersampled multiframes , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[35]  Robert L. Stevenson,et al.  Super-resolution from image sequences-a review , 1998, 1998 Midwest Symposium on Circuits and Systems (Cat. No. 98CB36268).

[36]  Lei Zhang,et al.  Sparsity-based image denoising via dictionary learning and structural clustering , 2011, CVPR 2011.

[37]  Adi Ben-Israel,et al.  Generalized inverses: theory and applications , 1974 .

[38]  Stephen J. Wright,et al.  Duality-based algorithms for total-variation-regularized image restoration , 2010, Comput. Optim. Appl..

[39]  Pierrick Coupé,et al.  An Optimized Blockwise Nonlocal Means Denoising Filter for 3-D Magnetic Resonance Images , 2008, IEEE Transactions on Medical Imaging.