Constrained analysis dictionary learning with the ℓ1/2-norm regularizer

Sparse representation has been proven to be a powerful tool for analysis and processing of signals and images. Whereas the most existing sparse representation methods are based on the synthesis model, this paper addresses sparse representation with the so-called analysis model. The ℓ1/2-norm regularizer theory in compressive sensing (CS) shows that the ℓ1/2-norm regularizer can yield stronger sparsity-promoting solutions than the ℓ1-norm regularizer. In this paper, we propose a novel and efficient algorithm for analysis dictionary learning problem with ℓ1/2-norm regularizer as sparsity constraint, which includes two stages: the analysis sparse coding stage and the analysis dictionary update stage. In the analysis sparse coding stage, adaptive half-thresholding is employed to solve the ℓ1/2-norm regularizer problem. In the analysis dictionary update stage, the solution can be straightforwardly obtained by solving the related least square problem followed by a projection. According to our simulation study, the main advantage of the proposed algorithms is its greater learning efficiency in different cosparsities.

[1]  Ender M. Eksioglu,et al.  Overcomplete sparsifying transform learning algorithm using a constrained least squares approach , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Wei Hu,et al.  Image inpainting via sparse representation , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[3]  Michael Elad,et al.  Cosparse analysis modeling - uniqueness and algorithms , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[5]  Ender M. Eksioglu,et al.  K-SVD Meets Transform Learning: Transform K-SVD , 2014, IEEE Signal Processing Letters.

[6]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[7]  Klaus Diepold,et al.  Analysis Operator Learning and its Application to Image Reconstruction , 2012, IEEE Transactions on Image Processing.

[8]  H L Fields,et al.  Science in China , 1996, Nature.

[9]  A. Bruckstein,et al.  K-SVD : An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation , 2005 .

[10]  Shuxue Ding,et al.  Dictionary learning with ℓ1/2 regularizer for sparsity based on proximal operator , 2015, 2015 IEEE 7th International Conference on Awareness Science and Technology (iCAST).

[11]  Zongben Xu,et al.  $L_{1/2}$ Regularization: A Thresholding Representation Theory and a Fast Solver , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[12]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[13]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[14]  Zhenni Li,et al.  A Fast Algorithm for Learning Overcomplete Dictionary for Sparse Representation Based on Proximal Operators , 2015, Neural Computation.

[15]  Michael Elad,et al.  Analysis K-SVD: A Dictionary-Learning Algorithm for the Analysis Sparse Model , 2013, IEEE Transactions on Signal Processing.

[16]  Wei Dai,et al.  Simultaneous Codeword Optimization (SimCO) for Dictionary Update and Learning , 2011, IEEE Transactions on Signal Processing.

[17]  Wei Dai,et al.  Analysis SimCO Algorithms for Sparse Analysis Model Based Dictionary Learning , 2016, IEEE Transactions on Signal Processing.

[18]  Yoram Bresler,et al.  Learning Sparsifying Transforms , 2013, IEEE Transactions on Signal Processing.

[19]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[20]  Mike E. Davies,et al.  Dictionary Learning for Sparse Approximations With the Majorization Method , 2009, IEEE Transactions on Signal Processing.

[21]  Michael Elad,et al.  K-SVD and its non-negative variant for dictionary design , 2005, SPIE Optics + Photonics.

[22]  Ke Huang,et al.  Sparse Representation for Signal Classification , 2006, NIPS.

[23]  Michael Elad,et al.  The Cosparse Analysis Model and Algorithms , 2011, ArXiv.

[24]  Yoram Bresler,et al.  Structured Overcomplete Sparsifying Transform Learning with Convergence Guarantees and Applications , 2015, International Journal of Computer Vision.

[25]  Yoram Bresler,et al.  Learning overcomplete sparsifying transforms for signal processing , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[26]  Rémi Gribonval,et al.  Constrained Overcomplete Analysis Operator Learning for Cosparse Signal Modelling , 2012, IEEE Transactions on Signal Processing.

[27]  Rémi Gribonval,et al.  Analysis operator learning for overcomplete cosparse representations , 2011, 2011 19th European Signal Processing Conference.

[28]  Anamitra Makur,et al.  Image denoising via sparse representations over Sequential Generalization of K-means (SGK) , 2013, 2013 9th International Conference on Information, Communications & Signal Processing.

[29]  Kjersti Engan,et al.  Method of optimal directions for frame design , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).