Overcomplete Dictionary Learning for Nonnegative Sparse Representation with an ℓ_p-Norm Constraint Based on Majorize-Minimization

This paper is for addressing the nonnegative sparse representation problem, i.e. to represent a nonnegative matrix as an over complete nonnegative dictionary times a nonnegative coefficient matrix. Many data in the real world can be represented sparsely by combinations of typical features. Moreover, large family of nonnegative data, such as image pixels, word frequency, power spectrum etc., are in great demand for engineering problems. Nonnegative Sparse Representation (NSR) is attractive to nonnegative data analysis. In this study, Overcomplete dictionary learning for NSR with an ℓ_p-norm (0 < p <1) is proposed and the ℓ_p-norm is expected for leading a sparser solution than ℓ_1-norm. An ℓ_p-norm is non-convex, then ℓ_p-norm was approximated by weighted ℓ_1-norm for convex optimization. We performed experiments about recovering accuracies of dictionary and coefficients. The recovering ratio is evaluated for various sparse levels of coefficients. The average of recovery ratios by proposed method for each sparse level were higher compared with an unweighted ℓ_1-norm. It improved +15.37% at most. We have shown that the proposed method also has advantages in recovering sparseness, ℓ_2 relative error and support distance of coefficients etc.

[1]  D. Hunter,et al.  A Tutorial on MM Algorithms , 2004 .

[2]  Mark W. Schmidt,et al.  Fast Optimization Methods for L1 Regularization: A Comparative Study and Two New Approaches , 2007, ECML.

[3]  Lucas C. Parra,et al.  Nonnegative matrix factorization for rapid recovery of constituent spectra in magnetic resonance chemical shift imaging of the brain , 2004, IEEE Transactions on Medical Imaging.

[4]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[5]  Daniela M Witten,et al.  Extensions of Sparse Canonical Correlation Analysis with Applications to Genomic Data , 2009, Statistical applications in genetics and molecular biology.

[6]  Constantine Kotropoulos,et al.  Music Genre Classification Using Locality Preserving Non-Negative Tensor Factorization and Sparse Representations , 2009, ISMIR.

[7]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[8]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[9]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[10]  M. Elad,et al.  $rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation , 2006, IEEE Transactions on Signal Processing.

[11]  Michael Elad,et al.  K-SVD and its non-negative variant for dictionary design , 2005, SPIE Optics + Photonics.

[12]  Thomas S. Huang,et al.  Image Super-Resolution Via Sparse Representation , 2010, IEEE Transactions on Image Processing.

[13]  Ke Huang,et al.  Sparse Representation for Signal Classification , 2006, NIPS.

[14]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[15]  Damiana Lazzaro,et al.  A fast algorithm for nonconvex approaches to sparse recovery problems , 2013, Signal Process..

[16]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[17]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Andrzej Cichocki,et al.  New Algorithms for Non-Negative Matrix Factorization in Applications to Blind Source Separation , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[19]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[20]  Rayan Saab,et al.  Stable sparse approximations via nonconvex optimization , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  Guillermo Sapiro,et al.  Sparse Representation for Computer Vision and Pattern Recognition , 2010, Proceedings of the IEEE.

[22]  Wei Hu,et al.  Image inpainting via sparse representation , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[23]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[24]  Hirokazu Kameoka,et al.  Formulations and algorithms for multichannel complex NMF , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[25]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.