An augmented Lagrangian approach to general dictionary learning for image denoising

This paper presents an augmented Lagrangian (AL) based method for designing of overcomplete dictionaries for sparse representation with general l"q-data fidelity term (q=<2). In the proposed method, the dictionary is updated via a simple gradient descent method after each inner minimization step of the AL scheme. Besides, a modified Iterated Shrinkage/Thresholding Algorithm is employed to accelerate the sparse coding stage of the algorithm. We reveal that the dictionary update strategy of the proposed method is different from most of existing methods because the learned dictionaries become more and more complex regularly. An advantage of the iterated refinement methodology is that it makes the method less dependent on the initial dictionary. Experimental results on real image for Gaussian noise removal (q=2) and impulse noise removal (q=1) consistently demonstrate that the proposed approach can efficiently remove the noise while maintaining high image quality.

[1]  Qiegen Liu,et al.  A novel predual dictionary learning algorithm , 2012, J. Vis. Commun. Image Represent..

[2]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[3]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[4]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[5]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[6]  Michael Elad,et al.  Efficient Implementation of the K-SVD Algorithm using Batch Orthogonal Matching Pursuit , 2008 .

[7]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[8]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[9]  Qiegen Liu,et al.  An augmented Lagrangian multi-scale dictionary learning algorithm , 2011, EURASIP J. Adv. Signal Process..

[10]  P. Vandergheynst,et al.  Sparse Approximation by Linear Programming using an L1 Data-Fidelity Term , 2005 .

[11]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[12]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[13]  Brendt Wohlberg,et al.  An efficient algorithm for sparse representations with lp data fidelity term , 2008 .

[14]  Àlex Haro,et al.  A Parameterization Method for the Computation of Invariant Tori and Their Whiskers in Quasi-Periodic Maps: Explorations and Mechanisms for the Breakdown of Hyperbolicity , 2008 .

[15]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[16]  Michael Elad,et al.  Sparse and Redundant Modeling of Image Content Using an Image-Signature-Dictionary , 2008, SIAM J. Imaging Sci..

[17]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[18]  José M. Bioucas-Dias,et al.  Fast Image Recovery Using Variable Splitting and Constrained Optimization , 2009, IEEE Transactions on Image Processing.

[19]  Wotao Yin,et al.  Bregman Iterative Algorithms for (cid:2) 1 -Minimization with Applications to Compressed Sensing ∗ , 2008 .

[20]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[21]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[22]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[23]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[24]  Jean-Yves Tourneret,et al.  Bayesian Orthogonal Component Analysis for Sparse Representation , 2009, IEEE Transactions on Signal Processing.

[25]  Peyman Milanfar,et al.  Robust Kernel Regression for Restoration and Reconstruction of Images from Sparse Noisy Data , 2006, 2006 International Conference on Image Processing.

[26]  Kjersti Engan,et al.  Frame based signal compression using method of optimal directions (MOD) , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[27]  Kjersti Engan,et al.  Recursive Least Squares Dictionary Learning Algorithm , 2010, IEEE Transactions on Signal Processing.

[28]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[29]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[30]  Joseph F. Murray,et al.  Dictionary Learning Algorithms for Sparse Representation , 2003, Neural Computation.

[31]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[32]  Tieyong Zeng,et al.  A Predual Proximal Point Algorithm Solving a Non Negative Basis Pursuit Denoising Model , 2009, International Journal of Computer Vision.

[33]  Guy Gilboa,et al.  Nonlinear Inverse Scale Space Methods for Image Restoration , 2005, VLSM.

[34]  Stanley Osher,et al.  Iterative Regularization and Nonlinear Inverse Scale Space Applied to Wavelet-Based Denoising , 2007, IEEE Transactions on Image Processing.

[35]  Joel A. Tropp,et al.  ALGORITHMS FOR SIMULTANEOUS SPARSE APPROXIMATION , 2006 .

[36]  Bhaskar D. Rao,et al.  An affine scaling methodology for best basis selection , 1999, IEEE Trans. Signal Process..

[37]  Guillermo Sapiro,et al.  Non-Parametric Bayesian Dictionary Learning for Sparse Image Representations , 2009, NIPS.

[38]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[39]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[40]  Brendt Wohlberg,et al.  Efficient Minimization Method for a Generalized Total Variation Functional , 2009, IEEE Transactions on Image Processing.

[41]  Guillermo Sapiro,et al.  Online dictionary learning for sparse coding , 2009, ICML '09.