Non-negative sparse decomposition based on constrained smoothed ℓ0 norm

Sparse decomposition of a signal over an overcomplete dictionary has many applications including classification. One of the sparse solvers that has been proposed for finding the sparse solution of a spare decomposition problem (i.e., solving an underdetermined system of equations) is based on the Smoothed L0 norm (SL0). In some applications such as classification of visual data using sparse representation, the coefficients of the sparse solution should be in a specified range (e.g., non-negative solution). This paper presents a new approach based on the Constrained Smoothed L0 norm (CSL0) for solving sparse decomposition problems with non-negative constraint. The performance of the new sparse approach is evaluated on both simulated and real data. For the simulated data, the mean square error of the solution using the CSL0 is comparable to state-of-the-art sparse solvers. For real data, facial expression recognition via sparse representation is studied where the recognition rate using the CSL0 is better than other solver methods (in particular is about 4% better than the SL0).

[1]  Michael Zibulevsky,et al.  Underdetermined blind source separation using sparse representations , 2001, Signal Process..

[2]  Mehmet Türkan,et al.  Sparse approximation with adaptive dictionary for image prediction , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[3]  Mohamed-Jalal Fadili,et al.  Inpainting and Zooming Using Sparse Representations , 2009, Comput. J..

[4]  Michael Elad,et al.  On the Uniqueness of Nonnegative Sparse Solutions to Underdetermined Systems of Equations , 2008, IEEE Transactions on Information Theory.

[5]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[6]  Jian Li,et al.  Linear Systems, Sparse Solutions, and Sudoku , 2010, IEEE Signal Processing Letters.

[7]  Michael Elad,et al.  Image Denoising Via Learned Dictionaries and Sparse representation , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[8]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[10]  P. Ekman,et al.  Constants across cultures in the face and emotion. , 1971, Journal of personality and social psychology.

[11]  Lu Yang,et al.  Sparse representation and learning in visual recognition: Theory and applications , 2013, Signal Process..

[12]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[13]  Albert Wang,et al.  The In-Crowd Algorithm for Fast Basis Pursuit Denoising , 2011, IEEE Transactions on Signal Processing.

[14]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[15]  Christian Jutten,et al.  Robust-SL0 for stable sparse representation in noisy settings , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[16]  Rémi Gribonval,et al.  A survey of Sparse Component Analysis for blind source separation: principles, perspectives, and new challenges , 2006, ESANN.

[17]  Shemels Geletaw Penalty and Augmented Lagrangian Methods to Solve Nonlinear Optimization Prob- lems , 2016 .

[18]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[19]  Thomas Kailath,et al.  Linear Systems , 1980 .

[20]  D. Donoho,et al.  Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA) , 2005 .

[21]  Daniel W. C. Ho,et al.  Underdetermined blind source separation based on sparse representation , 2006, IEEE Transactions on Signal Processing.

[22]  Onur G. Guleryuz,et al.  Sparse orthonormal transforms for image compression , 2008, 2008 15th IEEE International Conference on Image Processing.

[23]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[24]  S. Liberty,et al.  Linear Systems , 2010, Scientific Parallel Computing.

[25]  Edoardo Amaldi,et al.  On the Approximability of Minimizing Nonzero Variables or Unsatisfied Relations in Linear Systems , 1998, Theor. Comput. Sci..

[26]  Y. C. Pati,et al.  Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[27]  D. Donoho,et al.  Sparse nonnegative solution of underdetermined linear equations by linear programming. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[28]  Takeo Kanade,et al.  The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops.

[29]  Salvatore Tabbone,et al.  Author manuscript, published in "IEEE International Conference on Image Processing- ICIP'2011 (2011)" EDGE NOISE REMOVAL IN BILEVEL GRAPHICAL DOCUMENT IMAGES USING SPARSE REPRESENTATION , 2011 .

[30]  Franz Pernkopf,et al.  Sparse nonnegative matrix factorization with ℓ0-constraints , 2012, Neurocomputing.

[31]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[32]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[33]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[34]  Christian Jutten,et al.  A Fast Approach for Overcomplete Sparse Decomposition Based on Smoothed $\ell ^{0}$ Norm , 2008, IEEE Transactions on Signal Processing.

[35]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares , 2007, IEEE Journal of Selected Topics in Signal Processing.

[37]  Brendt Wohlberg,et al.  Noise sensitivity of sparse signal representations: reconstruction error bounds for the inverse problem , 2003, IEEE Trans. Signal Process..