Non-negative Matrix Factorization with Sparseness Constraints

Non-negative matrix factorization (NMF) is a recently developed technique for finding parts-based, linear representations of non-negative data. Although it has successfully been applied in several applications, it does not always result in parts-based representations. In this paper, we show how explicitly incorporating the notion of 'sparseness' improves the found decompositions. Additionally, we provide complete MATLAB code both for standard NMF and for our extension. Our hope is that this will further the application of these methods to solving novel data-analysis problems.

[1]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[2]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[3]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[4]  David J. Field,et al.  What Is the Goal of Sensory Coding? , 1994, Neural Computation.

[5]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[6]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[7]  Peter Földiák,et al.  SPARSE CODING IN THE PRIMATE CORTEX , 2002 .

[8]  S. Thorpe.  Localized versus distributed representations , 1998 .

[9]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[10]  Andreas Ziehe,et al.  Unmixing Hyperspectral Data , 1999, NIPS.

[11]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[12]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[13]  Patrik O. Hoyer,et al.  Non-negative sparse coding , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.

[14]  G. Buchsbaum,et al.  Color categories revealed by non-negative matrix factorization of Munsell color spectra , 2002, Vision Research.

[15]  James V. Stone,et al.  Spatiotemporal Independent Component Analysis of Event-Related fMRI Data Using Skewed Probability Density Functions , 2002, NeuroImage.

[16]  Mark D. Plumbley.  Algorithms for Non-Negative Independent Component Analysis , 2002 .

[17]  Mark D. Plumbley.  Algorithms for nonnegative independent component analysis , 2003, IEEE Trans. Neural Networks.

[18]  Nanning Zheng,et al.  Non-negative matrix factorization for visual coding , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[19]  Patrik O. Hoyer,et al.  Modeling Receptive Fields with Non-Negative Sparse Coding , 2002, Neurocomputing.

[20]  Philip M. Kim,et al.  Subsystem identification through dimensionality reduction of large-scale gene expression data. , 2003, Genome research.

[21]  Victoria Stodden,et al.  When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts? , 2003, NIPS.

[22]  Eun Yi Kim,et al.  Automatic Text Extraction for Content-Based Image Indexing , 2004, PAKDD.

[23]  Pablo Tamayo,et al.  Metagenes and molecular pattern discovery using matrix factorization , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Erkki Oja,et al.  Blind Separation of Positive Sources by Globally Convergent Gradient Search , 2004, Neural Computation.